### Appendix F: Critique of Syngenta Corporation’ s Probabilistic Implementation of CASM ATZ 

### for Ecological Risk Assessment

(30 Pages)Page 1 of 29



# Critique of Syngenta Corporation's Probabilistic Implementation of CASM ATZ 

# for Ecological Risk Assessment of Atrazine

Russell Erickson U.S. Environmental Protection Agency Mid-Continent Ecology Division Duluth, Minnesota

July 27, 2009Page 2 of 29 

### 1 Introduction

This document critiques the merits of the atrazine risk assessment methodology proposed by Syngenta Crop Protection, Inc. during public commen ts at the May 12-14, 2009 FIFRA Scientific Advisory Panel (SAP) Open Meeting regard ing the "The Ecological Significance of Atrazine Effects on Primary Producers in Surface Wa ter Streams in the Corn and Sorghum Growing Region of the United States". This proposa l was in response to EPA presentations at this meeting and to the EPA briefing document provi ded for this meeting (hereafter referred to as the "SAP Briefing Document").

The material from Syngenta examined for this critiq ue included:

(1) Copies of the Powerpoint slides in the present ation given on May 12, 2009 at the SAP meeting by David Volz (Syngenta) and Steven Bartell (E2 Consulting Engineers under contract to Syngenta) entitled "Probabilistic CASM ATZ2 : Refined Aquatic Ecological Risk Assessments for Atrazine."

(2) A June 1, 2009 report from Steven Bartell and Shyam Nair of E2 Consulting Engineers entitled "Distributions of Environmental Input Data , Toxicity Data, and Bioenergetic Parameter Values for the Probabilistic CASM ATZ2 Presented at the 2009 FIFRA SAP Meeting, " which summarizes the procedures used to develop parameter sets for probabilistic CASM ATZ . (3) Electronic files containing the 200 parameter sets used in Syngenta's probabilistic CASM ATZ analyses, provided by David Volz of Syngenta. Due to an error by EPA regarding toxicity values and an error by Syngenta regarding light int ensities in these sets, these sets were modified in coordination with Syngenta and E2. These new, m odified sets will be used for the analyses in this critique. Consequently, model results will no longer match those presented at the SAP meeting.

In addition, exchanges of information were made bet ween EPA and E2 Consulting Engineers to establish consistency in model inputs and verify id entical outputs when using the same inputs. As part of this exchange, it was also determined th at E2 was not using the final experimental ecosystem exposure time-series EPA included with th e SAP Briefing Document, and a consistent, updated set of time-series for model in put was provided by EPA to E2, resulting in additional differences in results from those presen ted by Syngenta at the May SAP.

This document is organized into three sections:

(1) To better understand the review comments on Syn genta's proposed methodology, the general assessment problem and framework for atrazine asses sments is described.

(2) Various shortcomings of Syngenta's methodology and analyses are identified.

(3) Important implications of Syngenta's methodolog y to atrazine risk assessments are discussed.Page 3 of 29 

### 2 Framework for Atrazine Assessments

Appendix I provides the problem definition and gene ral strategy for EPA's atrazine assessments, as presented by EPA at the SAP meeting and in the S AP Briefing Document. This assessment framework must be understood to provide necessary c ontext for the critique of Syngenta's proposed methodology. In addition, some aspects of this framework have been a source of confusion and merit some additional explanation her e:
2.1 Extrapolation of Atrazine Effects Information Across Exposure Time-Series

A major element of this framework is the need to ex trapolate effects across different exposure time-series, which involves the use of an effects m odel that can provide information on such extrapolations. Figure 1 depicts a simple example of this extrapolation problem and how it is addressed.

In this example, an experimental ecosystem establis hes the level of concern (LOC) for a 30-day exposure to be 20 μ g/L. For the purposes of this example, the effect of concern is specified to be a 20% reduction in average total plant biomass for some appropriate measurement period. However, the exact nature and magnitude of this eff ect is not important to the general problem definition and the conceptual model for extrapolati on being explained here, so other types and/or magnitudes of effect could have been used here inst ead.

This experimental ecosystem-based LOC is assumed to be relevant to the natural ecosystem; i.e., the natural ecosystem would experience a comparable effect if exposed to 20 μ g/L for 30 days. However, this long experimental exposure is quite d ifferent than the exposure of interest in the natural ecosystem, which is specified for this exam ple to be a 3-day, 100 μ g/L exposure. The basic question is whether this shorter, more intens e exposure exceeds an LOC; i.e., if the

Figure 1. A simple example of problem definition f or extrapolating an LOC based on an experimental mesocosm to a natural ecosystem with a different ex posure time-series.

Experimental Ecosystem 20 μ g/L atrazine determined to be LOC (20% effect) for a 30-day exposure Natural Ecosystem Exposure of interest is 100 μ g/Lfor 3-days; this exceeds modelextrapolated LOC of 80 μ g/L.

Mathematical Effects Model 20 μ g/L, 30-day exposure produces a 30% effect (=model effects index LOC) Mathematical Effects Model To produce the 30% effect for a 3-day exposure requires 80 μ g/L atrazine Uncertain Extrapolation to 3-day Exposure

Determine Model Effects LOC Based on Experimental System

Use Model to Extrapolate Between Exposures Apply Model Extrapolation to Natural SystemPage 4 of 29

experimental ecosystem was tested with this shorter exposure, would effects exceed the 20% benchmark? Unfortunately, the experimental ecosyst em study does not have information pertinent to this shorter, more-intense exposure.

The extrapolation model is intended to bridge this gap between the experimental ecosystem and natural ecosystem exposures by providing a means to assess the effects of both exposure timeseries for the same (model) system. The model need not exactly descri be either the experimental or natural ecosystem, or even produce the same impa irment at the same exposure. For example, Figure 1 shows the model to produce a 30% effect at 20 μ g/L, in contrast to the 20% effect in the experimental ecosystem. What the model does need t o provide is an extrapolation between these two exposure time-series that is informative of wha t would be expected in the experimental ecosystem if it was tested at the shorter duration.

In the example here, the model is first used to det ermine that there is a 30% modeled effect at the LOC (20 μ g/L) established for the 30-day exposure in the exp erimental ecosystem. This 30% effect is designated the LOC MEI – the value for a model effects index (MEI) at the LOC concentration established in the experimental ecosy stem study. The model is then used to estimate that reaching this LOC MEI for a 3-day exposure requires 4-times the concentr ation (80 ug/L).

Under the assumption that this concentration shift is relevant to the experimental ecosystem, the LOC concentration for the experimental ecosystem wo uld also be 80 μ g/L for a 3-day exposure. And because it is already assumed that the experime ntal ecosystem is relevant to the natural ecosystem, the LOC for a 3-day exposure would also be 80 μ g/L for this system as well. Thus, the 3-day, 100 μ g/L exposure in the natural ecosystem does exceed t he LOC.

The above simple example establishes the basic natu re of the assumptions being made, the extrapolations being conducted, and the (limited) r ole of the extrapolation model. However, the assessment framework must accommodate more complexi ties regarding effects and exposure information than in Figure 1. This more complex si tuation is described in Appendix I and only summarized here. There is not just one, definitive experimental ecosystem that defines an LOC, but rather a large number of experimental ecosystem studies with exposures that are simply classified as exceeding or not exceeding an effects level of concern (Figure I-1). There is not just one exposure time-series in the experimental e cosystems and one in the natural ecosystems, but rather many different exposure time-series shap es (e.g., Figure I-2). Nonetheless, the concepts in the above simple example still apply. As described in Appendix I, the model is run for each experimental ecosystem exposure to determi ne an LOC MEI that best discriminates exposures with and without an effect of concern (Fi gure I-3). Once the LOC MEI is determined, the model can be run for any natural ecosystem expo sure to determine if the modeled effects exceed the LOC MEI , and also to determine the factor (risk factor) by which the exposure must be reduced to produce modeled effects exactly equal th e LOC MEI (Figure I-4).

To end this section, it should be reiterated and em phasized that the effects model is not being asked to predict specific responses in the various ecosystems of interest (experimental or natural ecosystems); rather, these various systems are bein g used to define the variety of exposure timeseries over which the effects model is to assess re lative effects. The main requirement for thePage 5 of 29

model is that the relative model-calculated effects among exposure time-series be acceptable estimates of the relative effects among time-series in the real systems, so that the effects of concern in the experimental ecosystems can be extra polated to exposure time-series in natural ecosystems. An analogous situation would be if the natural ecosystem exposures were all at lower temperature than the experimental ecosystems, and that independent information was available that this temperature difference would re duce sensitivity by two-fold. It is not necessary for this information on temperature effec ts to pertain directly to the absolute effects in any of these systems, but rather just that the two- fold relative effect of temperature be relevant to the extrapolation of effects from the experimental ecosystems to the natural ecosystems.

2.2 Generic Nature of Time-Series Extrapolation

One option for the extrapolation effects model that was proposed by Syngenta and evaluated by EPA was a version of the mathematical aquatic commu nity simulation model CASM (Comprehensive Aquatic Simulation Model) developed by Dr. Steven Bartell of E2 Consulting Engineers for atrazine effects in a generic U.S. Mi dwest corn belt stream (CASM ATZ ). This model has a large number of parameters and input va riables regarding community structure/bioenergetics and physicochemical conditi ons that must be provided specific values. The values of these parameters/variables would be q uite different among the different experimental ecosystems and the different natural e cosystem systems of interest.

However, the very nature of the time-series extrapo lation explained above requires that the model not have one parameterization for one time-se ries and another parameterization for another time-series, because then the relative effe ct is not just of the time-series. Rather, CASM ATZ parameterization must be generic – meaning that th e same parameter and input variable values are used for all exposures, regardl ess of the nature of the system for which the exposure was observed. For a probabilistic assessm ent, this means that there would be multiple parameterization sets selected from ranges for the parameters/variables encompassing the systems of interest, but that all exposures (whethe r they be from an experimental ecosystem or a natural ecosystem site) would be analyzed with the same parameterization sets. Again, the model is just addressing the relative effects of di fferent exposure time-series, not trying to simulate specific systems. The real ecosystems (ex perimental or natural) provide the exposure time-series across which extrapolations are desired , not specific community structures and physicochemical conditions that the model must simu late effects for.

Thus, with regard to CASM ATZ , the assessment thus would not be site-specific ex cept with respect to the characteristics of the exposure time -series. However, values for the various bioenergetics parameters and physicochemical condit ions required by CASM ATZ must still be specified, and the selection of these values should be based (generically) on the experimental and natural ecosystems being assessed so that the extra polations are as relevant as possible. Furthermore, variability of risk estimators due to the ranges from which these values might be selected from should be assessed. If this variabil ity is significant, there must be consideration of where risk decisions are positioned within this var iability, but this generic application will not attempt to adjust risk estimates based on system-sp ecific attributes.Page 6 of 29

To end this section, it should be noted that more s ite-specific uses of CASM ATZ are conceivable, in which CASM would not just address relative effec ts of different exposure time-series, but would be expected to predict absolute effects of at razine not just as a function of exposure timeseries but other system attributes as well. This w ould require creating individual versions CASM ATZ tailored to each experimental and natural ecosyste m, including relevant uncertainty analysis for each version and validation that these individual versions adequately simulate community dynamics with and without atrazine exposu res. The modeled effects for the experimental ecosystems still would be used to esta blish an LOC MEI based on the observed effects in the experimental ecosystem, and the mode led effects in natural ecosystems would then be compared to this LOC MEI . However, such an effort is not feasible.

2.3 Assessing Uncertainty and Variability

As EPA discussed at the SAP meeting and in the SAP Briefing Document, this framework also allows for uncertainty analyses of the final risk d eterminations (e.g., whether an exposure exceeds the LOC; the value of the risk factor), by conducting Monte Carlo analyses in which various data inputs to the assessment are varied ac cording to information on their individual uncertainties. In particular, parameters and input variables for the extrapolation model could be varied to determine how uncertain/variable these ex trapolations might be. In the SAP Briefing Document, EPA described in general how such uncerta inty analyses should be done, and provided some example calculations of uncertainties , although the uncertainty analyses were not fully developed.

CASM ATZ requires specifying a large number of parameters a nd input variables, and EPA conducted sensitivity analyses to determine how ris k assessments would vary with the values specified for certain of these parameters/variables . Based on these sensitivity analyses, EPA concluded there would be too much uncertainty in th e risk assessments, and thus rejected CASM ATZ as the extrapolation model and adopted a different approach (which will not be discussed here). The limited uncertainty analysis provided by EPA in the SAP briefing document also demonstrated how the variable results from CASM ATZ would result in conservative restrictions on atrazine of questionab le validity.

Syngenta disagreed with EPA's conclusions regarding the utility of CASM. They asserted that what EPA treated as uncertainty should rather be vi ewed as meaningful variability and would not preclude CASM's use in the assessment, and that the sensitivity analysis conducted by EPA was too incomplete (with respect to the identity, distr ibution, and correlations of parameters/variables being varied) to support EPA's conclusions. Syngen ta therefore conducted probabilistic analyses similar to EPA's but involving more extensive param eter variations, and proposed how these should be incorporated into the risk assessments.

Three categories of model parameters and inputs wer e varied by Syngenta.

(a) Toxicity relationship EC50s and slopes were var ied in accord with procedures developed by EPA. The relationships developed by Syngenta were based on erroneous mean EC50s for some taxonomic groups in the Briefing Document, but stil l supported a useful demonstration of their proposed methodology. This critique will present a nalyses using the correct mean EC50s.Page 7 of 29

(b) Field data on temperature, light, nutrients, su spended solids, and water depth/velocity were analyzed by Syngenta to develop distributions for t hese variables, including certain correlations. This information was used to generate hypothetical input time-series for the model, although there were errors in the maximum light values used for these series.

(c) Selected bioenergetics parameters were assigned distributions based on the variability of values reported in the literature.

Based on this information, Syngenta developed 200 s ets of model parameters and input variables, differing with respect to toxicity and b ioenergetics parameters, and environmental conditions (these input sets have subsequently been updated to correct for errors noted above). For each input set, Syngenta's analyses were conduc ted using EPA's assessment framework as described above and in Appendix I. Syngenta summar ized the results of their 200 analyses in a distribution of the LOC MEI s for the 200 cases and distributions of the MEIs f or each field exposure time-series of interest. Risk was determi ned simply by whether the median of the these distributions differed; i.e., if the median of the MEIs for a particular exposure time-series exceeded the median of the LOC MEI s. This differed from EPA analyses, in which risk was characterized by the risk factor described above.Page 8 of 29



### 3 Shortcomings of Syngenta Methodology and Analyse s

It first should be first noted that Syngenta's prop osed probabilistic methodology is very similar to the uncertainty analysis described by EPA, in that both methods (a) use Monte Carlo analysis, modifying model parameters and input variables to g enerate probability distributions of effect outcomes and (b) follow the basic extrapolation met hodology from experimental ecosystems to field exposures described in Appendix I for each Mo nte Carlo case. The only differences in the methodologies presented at the SAP meeting regarded (a) how risks were characterized based on these probability distributions and (b) various spe cifics of model parameter/input variable modifications. This section of the critique will f irst address these two areas of differences, and then address the issue of ecological realism in pro babilistic CASM ATZ results.

3.1 Risk Characterization Statistic

Syngenta determined whether the LOC was exceeded ba sed simply on comparison, for the 200 Monte Carlo cases, of the median LOC MEI to the median MEI for a field exposure of interest . In contrast, EPA based their assessment on risk factor s and addressed percentiles other than the median. Some deficiencies regarding Syngenta's ris k characterization are:

3.1.1 Syngenta's methodology ignores the pairwise nature of the data.

By modifying model parameters and input variables, the 200 probabilistic cases define 200 different model configurations that vary markedly i n their sensitivity to atrazine. The MEI for a specific probabilistic case should be compared only to the LOC MEI for that case; i.e., it is inappropriate to compare the MEI for an atrazine-to lerant configuration to the LOC MEI determined for an atrazine-sensitive configuration. Consequently, comparing the medians (or any other percentile) of the separate MEI and LOC MEI distributions actually does not have a defined meaning with regard to how often and how ma rkedly the MEI exceeds the LOC MEI . For example, it is possible for the LOC to be exceeded a large fraction of the time, and the medians of these distributions to still indicate that the L OC is not exceeded. In contrast, the risk factors used by EPA as the basis of the assessment inherent ly address the pairwise nature of the data because they are based on adjusting exposure so tha t the MEI for a probabilistic case equals the LOC MEI for that case

. Instead of the risk characterization procedure used by Syngenta, a single risk statistic needs to be derived based on the MEI and LOC MEI for each case, thus recognizing the pairwise natur e of this analysis. The distribution of this single risk stat istic should be examined, not the separate distributions of the MEI and LOC MEI . Three possibilities for this statistic are:

(a) A simple yes/no regarding whether the MEI exceeds t he LOC

MEI . This can then be summarized as a probability of the MEI exceeding the LOC MEI based on all 200 cases. This ignores information on the severity of exceedences, but at least provides a legitimate expression of risk for a specific exposu re, because the frequency with which the LOC is exceeded across the 200 probabilistic ca ses is actually determined, unlike Syngenta's methodology.Page 9 of 29

(b) The ratio of the MEI to the LOC MEI ( or other measure of this deviation). Such an effect ratio retains information on the relative severity of the exceedence and risk can be summarized as a distribution of these ratios. T he percentiles of this distribution would provide useful summary information (e.g., wha t ratio is exceeded in 50% of the cases? In 10% of the cases?). This distribution co uld also be used to summarize how often a particular value for the ratio is exceeded (e.g., how often is the ratio above 1 to indicate some effect, or above 2 to indicate a majo r effect?).

(c) The risk factor as described in Section 2 and in Ap pendix I. This changes the perspective to the concentration scale rather than the effects scale and is analogous to risk quotients commonly used in risk assessment, wh ich contrast an exposure concentration to an effects concentration. Like th e effects ratio in option (b) above, this provides information on the severity of the exceede nce and can be summarized based on its distributional properties. EPA elected to f ocus its analyses on this statistic because risks defined using the concentration scale can be more directly related to management needs.

3.1.2 Syngenta methodology bases risk just on the median.

Basing risk characterization on median sensitivitie s constitutes a risk management decision for a lower level of protection than typical of OPP asses sments. When the median result shows the exposure to be just above the LOC, this is tantamou nt to allowing the exposure to exceed the LOC 50% of the time (often by a substantial amount, given the variability of CASM results). This is true whether the variability of CASM-based results is considered to be uncertainty or meaningful variability among sites. If uncertainty , using the median rather than a confidence limit allows a high rate of error. If meaningful v ariability, using the median allows a substantial percentage of sites to be at risk.

3.1.3 The statistical test for the difference betw een the distributions is inappropriate.

These model-based distributions involve an arbitrar y number of simulation cases. A statistical test is not appropriate to address an inadequate nu mber of model simulations. Enough simulations should be run to characterize the media n well enough without a statistical test. Furthermore, even if a statistical test were approp riate, the null hypothesis used by Syngenta is not, because it imposes a antiprotective bias in th at the MEI must exceed the LOC MEI by a certain amount before it is concluded to exceed the LOC MEI at all.

3.1.4 Syngenta's methodology is not a probabilisti c risk assessment.

Although based on a probabilistic analysis, Syngent a's methodology does not support meaningful probability statements regarding whether a given ex posure exceeds a level of concern (i.e., risk = probability of effect). Syngenta's simple comparis on of median MEIs and LOC MEI s has uncertain meaning because it ignores the pairwise n ature of the data, and comparisons based just on the median provide little information beyond tha t from a single, deterministic risk assessment. In contrast, probabilistic analyses using the risk statistics options discussed above provide a more meaningful characterization of risk, and allow usef ul statements about percentiles other than thePage 10 of 29

median. Figure 2 illustrates how these statistics would support an actual probabilistic risk assessment, in contrast to Syngenta's methodology. This figure is based on new EPA analyses using CASM ATZ2 code that was confirmed to be functionally the sam e as that used by Syngenta for the SAP meeting, with updates to the 200 sets o f parameters/input variables and experimental ecosystem exposure time-series to address errors in the original sets discussed earlier.

The first (top) panel of Figure 2 compares the LOC MEI distribution to the MEI distribution for four of the example chemographs in Figure I-2 of Ap pendix I. These chemographs were selected to provide MEI distributions that were shifted, rel ative to the LOC MEI distribution, far the left, slightly to the left, slightly to the right, and fa r to the right. This is the type of comparison proposed by Syngenta, with the risk characterizatio n simply being whether the median of the MEI distribution is shifted to the left or right of the median for the LOC MEI distribution. Such a comparion provides incomplete and misleading inform ation about whether the LOC is exceeded. As discussed above, the magnitude of the shift betw een the distributions has uncertainty meaning because it doesn't address the pairwise nature of t he data. These figures often provide a misleading impression that the MEI is either above or below the LOC MEI for all 200 cases, when in fact this is far from true, as the rest of Figur e 2 illustrates. Even if direct comparison of thes e distributions was meaningful, Syngenta also ignores the degree to which the distributions overlap. In general, such a comparison is not info rmative about the frequency with which MEIs exceed their corresponding LOC MEI s, or the actual magnitude of these exceedences.

The second panel of Figure 2 shows the cumulative p robability distribution of the effects ratio (option b above), which provides well-defined infor mation on both (a) the magnitude of the MEIs relative to their corresponding LOC MEI s and (b) how frequently the LOC (or some multiple of the LOC) is exceeded. The median effect ratios are 0.16, 0.74, 1.10, and 2.48 for chemographs #10, 2, 6, and 16, respectively, which provides a better-defined statistic regarding the degree with which the MEI exceeds the LOC for t he median case. More importantly, this distribution of effects ratios provides information other than whether a simple median shift occurs. For example, Chemograph 2 would be charact erized as not posing a risk by Syngenta, but in 36% of the cases, the MEI exceeds the LOC MEI (effect ratio > 1), and in 25% of the cases this effect ratio exceeds 2. Such information warr ants consideration in characterizing and managing risk, and should not be ignored as it is i n Syngenta's approach.

The third panel of Figure 2 shows the distribution of risk factors (option c above), which also provides useful information on the magnitude and pr obability of effects, similar to that for the effects ratios. In fact, both the effects ratio an d the risk factors have identical cumulative probabilities when their value is 1.0 (i.e., equal probabilities of the MEI exceeding the LOC MEI ). The median risk factors are 0.28, 0.75, 1.08, and 3 .09, differing somewhat from the median effect ratios because of the nonlinear relationship between effects and exposure concentration.

The bottom panel of Figure 2 summarizes the distrib utions of the effect ratios and risk factors as box plots. This format will be used for subsequent analyses in this critique because it allows more effective comparison of many assessments, and more clearly shows the consequences of different risk management decisions with respect to the frequency and magnitude of LOC exceedences. In Figure 2, this format makes clear how, although the majority of the MEIs do notPage 11 of 29

Figure 2. Top panel: comparison of LOC MEI and MEI distributions for four example chemographs , based on simulations with 200 different cases for CASM param eters/input variables. Second panel: cumulative probability distributions for effect ratios. Third panel: cumulative probability distributions for ri sk factors Bottom panel: probability distributions for effects ratios and risk factors summarized as box plots (b ox specifies 25 th , 50 th , and 75 th percentiles; whiskers specify 10 th and 90 th percentiles; dots specify 5 th and 95 th percentiles).

Model Effects Index 0.0 0.2 0.4 0.6Cumulative Probability

0.0 0.2 0.4 0.6 0.8 1.0

LOC MEI Chemograph 16 Chemograph 2 Chemograph 6 Chemograph 10

Effects Ratio 0.2 0.5 2 5 0.1 1 10Cumulative Probability

0.0 0.2 0.4 0.6 0.8 1.0

Risk Factor

0.2 0.5 2 5 0.1 1 10Cumulative Probability

0.0 0.2 0.4 0.6 0.8 1.0

Chemograph # 16 6 2 10 16 6 2 10Risk Factor

0.05 0.2 0.5 2 5 20

0.1 1 10 Effect Ratios Risk FactorsPage 12 of 29

exceed the LOC MEI

for Chemograph 2, this exposure could be considere d to entail considerable risk because the variability/uncertainty of probabi listic CASM results in substantial exceedences of the LOC MEI

for a substantial fraction of the cases.

3.2 Nature of Model Parameter and Input Variable M odifications

Several differences existed in the variation of mod el parameters/input variables in the probabilistic analyses presented by Syngenta and EP A in the SAP meeting materials.

(a) For bioenergetics, Syngenta varied several sel ected parameters, based on the range of values reported in reviewed literature. EPA did no t vary any bioenergetic parameters in their sensitivity or uncertainty analyses. Analyse s for this critique use the 200 bioenergetics data sets developed by Syngenta for the analyses th ey presented at the SAP meeting.

(b) For toxicity, EPA established distributions fo r the EC50s and slopes of effectconcentration relationships for four major plant ta xonomic groups (vascular plants, diatoms, green algae, blue-green algae) and randomly selecte d EC50s and slopes from these distributions for each case in its sensitivity and uncertainty analyses. Syngenta used these same distributions, except toxicity parameters used by Syngenta were in error due to EPA misreporting the mean EC50s for the taxonomic group s at one place in the SAP Briefing Document. Analyses for this critique use new toxic ity parameter sets reflecting correct taxa

means.

(c) For environmental conditions, EPA started with the default time-series in the CASM ATZ code for each variable (developed by Syngenta based on their analysis of field data) and changed the entire time-series based on just two ra ndomly selected factors – one that shifted all the default values of the time-series up and do wn and one that changed variability around the 30-day running averages of these default values . This allowed the development of timeseries that differed in their mean and in their var iability, while still retaining seasonal trends and auto-correlation of daily values similar to the default time-series. Syngenta used the original data for the default time-series to develo p monthly distributions for each variable, including simple correlations among the variables. They then randomly generated individual daily values from these distributions to develop different environmental parameter time-series, without considering temporal autocorrelation except at the monthly time scale. As noted above, the time-series for li ght were based on erroneous upper limits, and Syngenta developed 200 new environmental time-s eries sets to correct this error. The analyses in this critique used these new sets, but also included some variations to illustrate some shortcomings of Syngenta's methodology.

(d) EPA modified the exposure start date (default = Julian day 105) for model simulations using experimental ecosystem exposures and demonstr ated that this had some effect on the risk determination. Syngenta did not consider this parameter in its probabilistic analyses.Page 13 of 29

Some shortcomings of Syngenta's approach and analys es are:
3.2.1 The set of model parameters/input variable b eing modified is inadequately defined.

Although EPA manipulated only a limited number of m odel parameters and input variables in their sensitivity and uncertainty analyses, these a nalyses were only intended to illustrate that there was sufficient uncertainty to contraindicate the use of CASM. A more extensive and refined modification of model parameters/input vari ables would not change this conclusion and was therefore not necessary.

However, if CASM is to be used for a probabilistic risk assessment, as proposed by Syngenta, there needs to be a complete effort to identify wha t parameters/variables need to be manipulated and to appropriately define their distributions and interactions for the Monte Carlo analysis. In this respect, Syngenta's efforts are inadequate. T here was no effort to demonstrate (through sensitivity analyses) which parameters/variables sh ould be included, so that it is unknown whether other CASM model parameters/variables shoul d be included in the probabilistic manipulations and whether the parameters/variables that were manipulated could actually be omitted. One major omission of Syngenta, as noted above, was not considering the effect of exposure start date for the experimental ecosystem exposure simulations used to establish the LOC MEI . Other notable omissions were the degree-day para meters used to determine when each species becomes biologically active, the initial as sumed biomasses which affect eventual biomass potential and competitive relationships, an d light attenuation parameters, which is a major factor in probabilistic CASM's daily primary production. There is also the issue of the community structure in CASM ATZ – how sensitive are results to this and how well d oes it encompass the diversity of sites to be assessed?

3.2.2 Parameter/variable distributions are inadequ ately justified.

Syngenta's environmental variable distributions and correlations are based on only two Ohio streams. Although these streams are not dissimilar to many in the Midwest corn growing region, they would not fully capture the variation and inte ractions of environmental parameters that should be considered. The distributional shape (un iform or log uniform) also needs better evaluation and justification. A defensible probabi listic implementation of CASM applicable to the region would need considerably more development in this regard.

Regarding the bioenergetics parameters, the sources of the distributions are inadequately characterized, without citations and with reference to professional judgment in parameter distribution development. Any actual implementatio n would require specification of all information sources considered, and better establis hed and defined decision criteria.

Although Syngenta's probabilistic analyses used dis tributions for toxicity parameters developed by EPA, if these are to be used for actual risk ass essments (rather than just to establish CASM as too variable for use as EPA analyses concluded), mo re development is needed regarding these distributions. There needs to be more consideratio n of what toxicity data are most applicable to each CASM species, with possible development of dif ferent distributions for smaller or different taxonomic groups. In addition, the probabilistic a nalyses need to consider not just randomPage 14 of 29

selection from a distribution given a nominal mean and standard deviation, but also the uncertainties of these means and standard deviation s. In particular, the implications of limited data for the diatoms and blue-green algae should be addressed. EPA addressed this uncertainty issue in their analyses of the effects model they p roposed as an alternative to CASM.

3.2.3 Correlations among parameters and variables are inadequately developed.

Although Syngenta criticized EPA for not considerin g correlations among parameters, they only considered simple linear correlations and only with in their limited environmental data set, and did not demonstrate how effectively variability/unc ertainty in model results was addressed by this treatment. Considering the limited available data, such shortcomings in the analysis are understandable and largely unavoidable, but still r esult in weaknesses and uncertainties in the overall procedure that affect model credibility and warrant better consideration. Again, considerable more development work and documentatio n in this regard would be needed if this methodology is to be justified and implemented. On e important consideration would be whether plant community structure and bioenergetics should vary with nutrient, temperature, light, etc. This would be expected to be the case and EPA's sen sitivity analysis in the SAP briefing document raised this issue. As difficult as it wou ld be to accomplish, this needs to be addressed in some manner and degree to establish the credibil ity of the probabilistic analyses. Another important consideration would be the shape of relat ionships between parameters other than the currently-assumed linear relationship.

3.2.4. Environmental variable time-series have que stionable characteristics.

Syngenta's procedure for developing environmental t ime-series resulted in questionable characteristics for the time-series. These problem s arose due to combining data from multiple sites/years into distributions for each month, and then using the distributions to randomly select daily values without regard to how daily values wou ld vary due to temporal autocorrelation and the differences between years and sites. This resu lts in three problems in the time-series, illustrated in Figure 3 using the first five Syngen ta Monte Carlo cases for temperature.

(a) Because each case uses, for each month, a rand om sample of 28-31 daily values from the same distribution, monthly means are virtually identical among the different cases (upper panel of Figure 3). Syngenta's probabilisti c analysis thus does not account for variation in average temperatures across sites and years, which is important if this analysis is to credibly describe variability across the Midwest corn belt. Rather, the cases differ just in the random variations within a month for a single set of monthly means. For contrast, the upper panel of Figure 3 also shows th e range over which 30-day running average temperatures varied in EPA's analyses, cove ring a range of possibilities for variation among sites and years.

(b) Because temporal effects other than monthly me ans are not considered (especially autocorrelation among days) and because the contrib ution of variability between sites and years are treated as part of the daily variation, r andom day-to-day variations are unrealistically extreme – up to 15 ° C between adjacent days (middle panel of Figure 3), in contrast to the more realistic default temperature time-series.Page 15 of 29

Figure 3. Temperature inputs for simulations. In top panel, solid lines with numbered circles denote monthly means for first five cases of Syngenta's probabilis tic analysis, and dashed lines denote range of 30-d ay running averages in EPA's uncertainty analysis. In middle and bottom panels, the variability for the default time-series (bold line) is contrasted with that for the first f ive cases of Syngenta's probabilistic analysis (num bered lines on middle panel, dotted lines on bottom panel).

Month 1 2 3 4 5 6 7 8 9 10 11 12Average Monthly Temperature

0 5 10 15 20 25 30 35

1 1 1 1 1 1 1 1

1

1

1

1

2 2 2 2 2 2 2 2

2

2

2

2 3 3 3 3 3 3 3 3
3

3

3

3 4 4 4 4 4 4 4 4

4

4

4

4 5 5 5 5 5 5 5 5

5

5

5

5

Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Daily Temperature

0 5 10 15 20 25 30 Julian Day 90 95 100 105 110 115 120Daily Temperature

0 5 10 15 20 25 30

1 1
1 1

1 1 1 1 1 1

1

1 1 1

1 1 1

1 1

1

1 1

1 1

1 1 1 1

1 1 1 2 2 2

2 2

2 2 2 2

2 2

2 2 2 2

2
2 2

2 2 2 2 2

2 2

2 2 2 2 2 2

3 3

3 3 3 3

3

3 3

3 3 3

3 3 3

3 3

3 3 3

3 3

3 3

3 3

3 3

3 3

3

4 4 4 4 4

4 4

4 4 4

4 4 4 4 4

4 4 4

4

4 4 4 4 4

4 4

4

4

4 4

4 5

5 5

5 5 5

5

5 5 5 5 5

5

5 5

5 5

5 5 5

5 5 5 5

5 5 5 5

5 5

5Page 16 of 29

(c) Using simple monthly stratification results in a disjoint, "stair-step" time-series (bottom panel of Figure 3). This again should be c ontrasted to the more realistic variations of the default temperature time-series ( bold line).

What value and legitimacy does Syngenta's probabili stic analysis have if it is based on unrealistic temperature fluctuations and does not c over the range of environmental conditions that likely occur in the assessment region? And si milar problems exist for other environmental variables – Figure 4 shows even more extreme and un realistic variations for suspended solids compared to the default time-series. The default s eries provides a more reasonable temporal pattern, suggesting events that last for a few to s everal days rather than wild day-to-day variations. However, even this default series warr ants some further evaluation because by using average values over multiple years it has too many events and the events would be less pronounced than original due to averaging. In any event, more realistic time-series are needed, because primary productivity responds greatly to th is variable in a highly nonlinear way.

Figure 4. Total suspended solids inputs for simula tions. The variability for the default time-series (bold line) is contrasted with that for the first five cases of Sy ngenta's probabilistic analysis (numbered lines on top panel, dotted lines on bottom panel).

Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Total Suspended Solids

10 100 1000 Julian Day 90 95 100 105 110 115 120Total Suspended Solids

10 100 1000

1 1 1 1 1

1 1 1

1 1 1 1

1 1

1

1 1 1

1 1 1

1 1

1 1

1

1 1 1

1 1 2
2

2 2 2 2

2

2 2 2 2 2

2 2

2 2

2 2

2

2 2

2

2 2

2 2 2

2 2

2

2

3 3

3 3

3 3

3 3 3

3 3 3

3 3

3
3

3 3 3

3 3

3 3

3 3

3 3 3

3

3 3 4

4 4 4

4

4 4 4

4 4

4 4

4 4

4 4

4 4 4 4

4 4

4

4 4 4

4 4 4 4

4

5 5 5

5 5

5

5 5

5 5

5 5 5 5 5

5 5 5

5 5 5 5

5 5 5

5

5 5

5 5 5Page 17 of 29

3.3 Ecological Realism of CASM ATZ Simulations

A potential benefit of CASM is that it might introd uce more ecological realism into atrazine risk assessments. However, although CASM does include c redible descriptions of certain ecological processes, it does not necessarily follow that thes e provide meaningful and significant inferences regarding atrazine risk assessments. As George Box said, "all models are wrong, some are useful." CASM is necessarily a gross simplificatio n of an actual ecosystem, and whether its inferences about how atrazine toxicity is translate d into community effects as a function of model parameters and input variables are "useful" i s far from established. This section of the critique will discuss various aspects of this issue , some of which apply to CASM ATZ generally and some of which are more specific to Syngenta's p robabilistic approach.

This question of ecological realism is especially i mportant given the large variation of risk estimates from Syngenta's probabilistic CASM method ology. Figure 5 summarizes the variability of risk factors for all 16 example chem ographs from Figure I-2 across the 200 (new) probabilistic cases. The difference between the 5 th and 95 th percentiles varies from 6 to 50 depending on the chemograph, and would probably be larger if these cases addressed the deficiencies discussed in Section 3.2. If CASM ATZ is "ecologically realistic" the implication of this is that atrazine risk varies with how system p roperties differ across the probabilistic cases. This has great implications to risk assessments, as discussed in Section 4, dependent on how meaningful this variation actually is. For such a large variability to be accepted in the risk assessments, there must be confidence that this CAS M-predicted variability is meaningful.

3.3.1 Simplifications in CASM ATZ have uncertain effect on assessment realism.

CASM ATZ involves various simplifications that conceivably can affect the ecological realism of the simulations and thus atrazine effects assessmen ts. At a minimum, there must be some demonstration that these simplifications do not mat erially affect assessments. Some examples of these simpliccations are: Figure 5. Box plots of the distributions of risk f actors for sixteen example chemographs, based on us ing probabilistic CASM ATZ with Syngenta's 200 new cases. The box denotes th e 25 th , 50 th , and 75 th percentiles; the bars denote the 10 th and 90 th percentiles; and the dots denote the 5 th and 95 th percentiles.

Chemograph # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16Risk Factor

0.2 0.5 2 5

0.1 1 10Page 18 of 29

(a) Although 26 plant species are a lot for such a model, this is far more simple than natural ecosystems and often results in time periods in whi ch only 1-3 species are significant. Plant community dynamics thus might be unduly affected by the toxicity parameters assigned to certain critical species, which is not necessarily true to the same extent in natural systems. This is especially a concern given uncertainties regardi ng what the toxicity parameters for individual species should be.

(b) CASM is a single-compartment (zero-dimensional) model trying to describe a complex threedimensional system. Although it does address certa in depth effects (such as light attenuation), each species type is assigned a specific depth and the light at that depth is applied to all the bioenergetic calculations for the species. One spe cific ramification of this for streams is that, under turbid conditions, benthic plants are often u niformly subject to enough light attenuation to make primary production negligible, without conside ration of plants growing at different depths, including shallow depths were sufficient light exis ts for primary production.

(c) CASM does not consider diurnal variations in co mmunity dynamics and other conditions. This might or might not affect estimations of atraz ine risks, but still is an ecologically unrealistic element of CASM that warrants better evaluation wit h regard to whether its results are unduly distorted by this simplification.

3.3.2 Mass balance is not satisfied in CASM ATZ . CASM originated as a lentic model in which horizont al exchanges of material are much slower and more limited than in streams. For the stream i mplementation of CASM ATZ , advective mass balance is not handled properly. Partly for this r eason, phytoplankton were dropped from CASM ATZ . However, for Sygenta's methodology, CASM still m odels nutrient dynamics despite these mass balance problems, and there are question able depletions of nutrients within the stream reach being simulated that affect community dynamic s. In addition, field nutrient concentrations are treated as inputs to the simulated compartment (stream reach), when they might arguably be better treated as the actual ambient concentrations , so that their dynamics should not be simulated. Due to concerns raised by EPA, CASM ATZ does have the option of not simulating nutrient dynamics, which EPA has used in some analy ses, demonstrating that these two different options can produce substantially different referen ce simulations and risk calculations (see SAP Briefing Document). Such different outcomes unders cores uncertainties of CASM ATZ that cast doubt on its ecological realism. Which option, if either, provides realistic nutrient concentrations to drive the dynamics of the plant community and it s susceptibility to atrazine?

3.3.3 Effects of specific parameters and variables are inadequately defined and validated.

As already noted, Syngenta failed to provide sensit ivity analyses that define which model parameters and input variables are actually needed in the probabilistic analysis. Another reason for such sensitivity analyses is to understand what parameters/variables are most important for the large variability of risk estimates in Figure 5 . If some credibility of the nature and magnitude of the effects of specific parameters/variables can be established, this would support the overall credibility of the probabilistic CASM ATZ results. EPA's sensitivity analysis do this to so me degree. However, rather than supporting CASM ATZ credibility, such analyses often revealPage 19 of 29

shortcomings. For example, EPA's sensitivity analy sis showed that temperature can have a large effect on CASM-computed risks. Is there a reasonab le expectation that risks in real ecosystems would be similarly temperature-dependent, or is thi s an artifact of the model that stems from incomplete consideration of important temperature-r elated factors? As noted by EPA, this temperature sensitivity raises questions about corr elations between temperature and community structure/bioenergetics, exposure timing, toxicity, etc., that are not addressed in Syngenta's probabilistic analyses. Overall, there is no empir ical or theoretical basis for saying this variability is meaningful, and it would be necessar y to more fully explore Syngenta's results in this regard.

3.3.4 Probabilistic parameter modifications produc e unrealistic reference simulations.

Considerable attention has been paid to whether CAS M produces realistic reference (i.e., no atrazine exposure) simulations of biomass time-seri es for a midwestern stream, using the default bioenergetics parameters and environmental variable time-series. To some degree, the default CASM parameters were tailored to produce credible r eference simulations. If this probabilistic analysis is to be meaningful, the reference simulat ions for all 200 probabilistic cases also need to be realistic.

Figure 4 shows the variability of the reference bio mass time-series for different plant taxa for Syngenta's first ten probabilitic cases, compared t o the reference simulation for the default case. In some of the probabilistic cases, taxa biomasses don't even increase beyond their initial assumed values. Among these cases, peak taxa bioma sses vary by up to three orders of magnitude. If they do not provide more realistic r eference simulations than these, what is the value of this probabilistic variation of bioenerget ics parameters and environmental variables?

Another aspect of the reference simulations (includ ing for the default situation) that must be questioned before accepting CASM ATZ as a foundation for risk assessments is the extend ed period of constant plant biomass early in the year (controlled by a degree-day parameter for plant dormancy), and the significant declines for most ta xa following this flat portion before significant growth occurs. This behavior is, per s e, unrealistic. How much this behavior, and the associated selection of values for initial biomass and the degree-day parameter, might affect the risk assessments must be addressed.

3.3.5 Risk factors are overly sensitive to methodo logy for environmental time-series.

It was already noted that Syngenta's time-series fo r environmental variables were unrealistic in that they involved excessive day-to-day variations around the same monthly mean, rather than providing appropriately autocorrelated day-to-day c hanges and means that covered the range in the Midwest corn belt. That the nature of these ti me-series can affect risk assessments is illustrated on Figure 7. These boxplots summarize analyses of the 200 probabilistic cases in which toxicity and bioenergetics values are unchang ed, but environmental variables are treated in four different ways – (a) the 200 (new) cases us ing Syngenta's approach, (b) fixed at the daily averages of Syngenta's time-series, (c) fixed at th e default values, and (d) 200 cases using EPA's approach as described in the SAP briefing document.Page 20 of 29

Figure 7 demonstrates considerable variability in r isk estimates depending on how the environmental time-series are treated. For a parti cular chemograph, the four alternative treatments have medians that vary by up to 6-fold ( typically be at least 2-fold), and can have greatly different variabilities. Most notably, for all the chemographs, Syngenta's 200 environmental time-series result in risks markedly less than those for the average of these timeseries. This is not necessarily inappropriate, but underscores that the nature of the probabilistic time-series will affect risk assessments, so that i t is important to have something more realistic than what Syngenta's methodology provides. Interes tingly, the default time-series often produce the lowest risk estimates. The point here is not t hat one of these alternatives is better than the others, but rather that the sensitivity of risk est imates to assumptions about environmental variabilities requires that there be more effort to have better estimates for these variables. Figure 6. Variability of reference simulations in Sygenta probabilistic analysis. Bold lines depict default environmental conditions and bioenergetics paramete rs; other lines depict first 10 of 200 probabilisti c cases.

Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Green Algae Biomass

0.005 0.02 0.05 0.2 0.5 2 5 20 50

0.01 0.1 1 10

Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Bluegreen Algae Biomass

0.005 0.02 0.05 0.2 0.5 2 5 20 50

0.01 0.1 1 10 Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Diatom Biomass

0.005 0.02 0.05 0.2 0.5 2 5 20 50

0.01 0.1 1 10

Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Other Algae Biomass

0.005 0.02 0.05 0.2 0.5 2 5 20 50

0.01 0.1 1 10 Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Total Plant Biomass

0.005 0.02 0.05 0.2 0.5 2 5 20 50

0.01 0.1 1 10

Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Vascular Plants

0.005 0.02 0.05 0.2 0.5 2 5 20 50

0.01 0.1 1 10Page 21 of 29

Figure 7. Distributions of risk assessments for al l 16 example chemographs using four alternatives fo r environmental variable time-series (still using Syn genta's 200 probabilistic sets for bioenergetics an d toxicity parameters). Numbers without a letter suffix denot e simulations with chemograph using Syngenta's 200 probabilistic environmental time-series; suffix A denotes using average of these time-series; suffix D denotes using default time-series; suffix E denotes using 2 00 probablistic environmental time-series using EPA approach.

Chemograph # 1 1A 1D 1E 2 2A 2D 2E 3 3A 3D 3E 4 4A 4D 4ERisk Factor

0.2 0.5 2 5 20

0.1 1 10

Chemograph # 5 5A 5D 5E 6 6A 6D 6E 7 7A 7D 7E 8 8A 8D 8ERisk Factor

0.2 0.5 2 5 20

0.1 1 10

Chemograph # 9 9A 9D 9E 10 10A 10D 10E 11 11A 11D 11E 12 12A 12D 12ERisk Factor

0.2 0.5 2 5 20

0.1 1 10

Chemograph # 13 13A 13D 13E 14 14A 14D 14E 15 15A 15D 15E 16 16A 16D 16ERisk Factor

0.2 0.5 2 5 20

0.1 1 10Page 22 of 29



### 4 Implications of Sygenta Methodology to Risk Asse ssments

Section 3 discussed a variety of issues regarding e stablishing CASM ATZ as a credible tool for atrazine risk assessments that would require a trem endous effort, and might not be possible for some of the specifics. However, assuming that this could be done, the risk estimates of CASM ATZ would still range widely depending on how the mode l is parameterized, which would have implications to risk assessments that would st ill contraindicate its use.

4.1 Syngenta method risk characterizations are exc essively uncertain.

At the SAP meeting, both Syngenta and some SAP pane l members objected to EPA characterizing the range of CASM results as uncerta inty, when it could to some degree represents meaningful model inferences regarding ho w atrazine risk varies with respect to system characteristics.

However, probabilistic CASM's variability can be ar gued to be predominantly uncertainty. A large fraction of this variability is due to uncert ainty regarding what toxicity parameter to assign CASM species. As noted earlier, this involves even more uncertainty that isn't yet accounted for in probabilistic CASM. Even if the toxicity parame ter distributions could be better resolved for particular taxa, there is also uncertainty about wh ether individual CASM species should be treated as just representing particular species, or are surrogates for a broader set of species with certain bioenergetic properties. For example, EPA' s sensitivity analysis demonstrated that changing the nutrient status shifts the late-season dominant species from blue-green algae to what is based on a euglenoid. Thus, with respect t o toxicity parameters, variability must be treated as uncertainty. The same is also true for the bioenergetics parameters, which reflect the variability of values reported in the literature wi thout any clear links to system characteristics.

With respect to the environmental variables, there is a potential for CASM results to reflect some meaningful differences between systems (e.g., lower versus higher nutrients, temperatures, etc.). However, as discussed earlier, this potential is no t realized in the current probabilistic CASM because the environmental time-series are very clos e regarding their monthly means and just differ by unrealistic daily variations about those means. Even if this shortcoming of probabilistic CASM was corrected, there is inadequate substantiat ion that CASM variability is a meaningful function of system attributes. As discussed earlie r, EPA sensitivity analysis reported in the SAP Briefing Document did show risk to vary as a functi on of mean temperature, but various factors and interactions not considered in probabilistic CA SM make this effect of uncertain validity.

However, even if there are some elements of probabi listic CASM that reflect meaningful variation of risk due to system characteristics, th is doesn't matter because both uncertainty and meaningful variability have the same implications t o risk assessment for this generic assessment. If the variability is uncertainty, risk needs to be managed based on some location in this range of uncertainty. If the variability is meaningful, the implication is that some systems are more at risk than others, but generic risk then must be managed based on a similar location in the distributions of systems to be protected.Page 23 of 29

4.2 Risk managment using Syngenta methodology will be unjustifiably conservative.

As noted above, whether probabilistic CASM variabil ity is uncertainty or variability, it requires a decision regarding where risk is managed within thi s variability. The median is arguably inappopriate, because it allows a 50% change of a s ystem not being protected. Furthermore, given the wide variability of CASM ATZ risk estimates (Figure 5), this also involves cons iderable risk of substantial effects, not just minor risks. Use of the median is contrary to current OPP practice of using more sensitive endpoints.

Table 1 lists the 50 th and 90 th risk factor percentiles for all 16 example chemogr aphs from Figure 5. These represent the factor by which exposure wo uld need to be reduced so that the MEI is at the LOC MEI

for the selected percentage of probabilistic cases . Using the 90 th percentile to provide moderate conservatism in face of probabilis itic CASM variability/uncertainty results in 15 of these chemographs exceeding a level of concer n (as opposed to 8 if the median was used), and would result in restricting exposure by 2.0- to 3.5-fold more than if the median was used.

This more restrictive risk management is a necessar y consequence of the variability/uncertainty of probabilistic CASM. However, whether these risk restrictions can be justified must be questioned given various shortcomings and insuffici ent validation of CASM for this application. This creates an untenable situation for EPA – of su ggesting conservative regulations that cannot really be defended and of not being able to justify less conservative CASM-based risk assessments because of various uncertainties about this model.

4.3 Syngenta methodology requires site-specific ap plication that is uncertain and impractical.

A final point that should be made regarding the imp lications of the proposed Syngenta methodology to risk assessments is that, if probabi listic CASM ATZ variability is truly a meaningful function of definable system attributes, then the generic assessment approach should be dropped. If CASM ATZ can truly be trusted to provide meaningful informa tion on the relationship of atrazine risk to system attributes, then a site specific approach should be used as discussed at the beginning of this critique, in whi ch a version of CASM is developed for each experimental and natural ecosystem of concern and t he MEIs/LOC MEI s and risk factors reflect these system differences, not just model extrapolat ions across the time-series of concerns. Even if possible, this is not practical regarding the le vel-of-effort, money, and time needed to address the many development issues/questions with CASM for so many systems. The successful uses of CASM have involved sufficient efforts to tailor CASM to answer assessment questions for specific sites. The problems discussed here with p robabilistic CASM underscore how it is not justified for the desired generic atrazine assessme nts. Table 1. 50 th and 90 th percentile risk factors for CASM-based assessment of 16 example chemographs.

Chemograph # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 50 th Pct Risk Factor 2.0 0.85 0.68 0.78 0.66 1.27 1.49 1.41 1.62 0.32 0.64 0.59 1.33 1.61 4.1 3.3 90 th Pct Risk Factor 7.0 2.1 1.82 2.1 1.76 3.4 3.5 3.5 3.8 0.64 1.33 1.22 3.1 3.6 9.5 8.1Page 24 of 29 

### Appendix I. Atrazine Assessment Framework

1. Problem Definition

A set of microexperimental ecosystem and mesoexperi mental ecosystem (experimental ecosystem) studies with atrazine serve are to serve as the foundation for determining LOCs for atrazine exposures in natural freshwater systems. Figure I-1 summarizes these data. Each datum on Figure I-1 is characterized by the exposure dura tion at which effects were assessed, the average exposure concentration over this duration, and an effects score (Brock score) using the system of Brock et al. (2000). A fundamental assum ption in using these experimental ecosystem data is that they collectively describe a relations hip of effects to exposure that is relevant to natural freshwater systems. In other words, natura l aquatic plant communities will react adversely if subjected to the same exposures that e licited significant adverse effects in the experimental ecosystem studies. This assumption is inherent in any assessment that extrapolates toxicity experiments to the field. The use of expe rimental ecosystems arguably provides a better basis for such extrapolations than do single-specie s toxicity tests.

An important consideration in using the experimenta l ecosystem data to establish LOCs is the diversity of exposure concentration time-series. N ot only do the experimental ecosystem studies have different durations, but the exposure concentr ations within these durations can range from being roughly constant to declining substantially w ith a half-life as short as 3 days. This variability makes if difficult to compare effects a mong different experimental ecosystem studies, but exposure variability is an even greater problem for extrapolating these effects to the field. For natural freshwater systems, atrazine exposures tend to be even more variable than in the

Figure I-1 Brock scores in atrazine cosm exposures as a function of exposure duration and average concentration. Dashed lines denote the 10- 20 μg/L LOC concentration range designated by the 2003 IRED.

Exposure Duration (days)

2 5 20 50 200 10 100Average Concentration (μg/L Atrazine)

0.1 1 10 100 1000 10000

5 4 3 2 1 Brock ScorePage 25 of 29

Figure I-2. Examples of atrazine exposure time-ser ies for natural freshwater systems.

0 50 100 150 200 250 300 350 0 50 100 150 200 250
IN11_05_1Peak

0 50 100 150 200 250 300 350 0 10 20 30 40 50 0 50 100 150 200 250 300 350 0 5 10 15 20 25 0 50 100 150 200 250 300 350 0 2 4 6 8 10 0 50 100 150 200 250 300 350 0 10 20 30 40 50 0 50 100 150 200 250 300 350 0 10 20 30 40 50

0 50 100 150 200 250 300 350 0 10 20 30 40

Julian Day 0 50 100 150 200 250 300 350 0 10 20 30 40 50 0 50 100 150 200 250 300 350 0 10 20 30 40 0 50 100 150 200 250 300 350 0 5 10 15 20 25 0 50 100 150 200 250 300 350Concentration (μg/L Atrazine)

0 10 20 30 40 50 0 50 100 150 200 250 300 350 0 10 20 30 40 0 50 100 150 200 250 300 350 0 10 20 30 40

0 50 100 150 200 250 300 350 0 50 100 150 200 RegA_Res_1Peak

IL08_06_HiLo IL08_06_HiHi

IL08_06_LoHi Sandusky_HiLo

Sandusky_HiMed IN03_05_Orig_HiMedLo

IN03_05_HiMedLo IN03_05_MedHiLo

Maumee_MedHiLo Sandusky_MedHiLo

RegG_Res_longdur MO01_05_MO01 1

3

5

8

11

13 14 12 10 6 4 2

16 15Page 26 of 29

experimental ecosystems and do not have a fixed dur ation. Atrazine enters natural freshwater systems largely in rainfall-driven runoff, resultin g in highly variable and episodic exposures (Figure I-2) that depend on rainfall distribution, atrazine application patterns, topography, and soil properties. Therefore, a primary goal for thi s assessment methodology is to address the diversity of exposure time-series both in the exper imental experimental ecosystems and natural systems. The experimental ecosystem data do not pro vide sufficient information for defining such time effects because the scoring system is sem iquantitative and the study conditions and responses do not fully address the issues of durati on and time-variability. Thus, another issue facing this methodology is to base time effects on considerations other than the experimental ecosystem data.

Atrazine LOCs are to be based on discriminating the Brock score group of 3 to 5 (i.e., significant effects) from the score group of 1 or 2 (i.e., no t o slight effects). As shown in Figure I-1, these two score groups are largely separated, although so me overlap also exists, particularly over the 10-20 μ g/L range.. Some of this overlap is due to the exp osure time-variability issue (which cannot be fully addressed in the two-dimensional gr aphic format of Figure I-1); for example, the two lowest squares involve exposures with fast deca ys, so that the plotted average exposure concentrations are much lower than the peak concent rations. However, this overlap is also reflective of other differences among the experimen tal ecosystems. Unfortunately, these differences are not sufficiently understood to supp ort methods for better discriminating the score groups. In any event, this overlap means that any LOC methodology will need to address the occurrence of false negatives (i.e., Brock scores o f 3 to 5 which lie below the LOC) and false positives (i.e., Brock scores of 1 or 2 which lie a bove the LOC). The positioning of the LOC relative to the number of false positives and negat ives is a key risk management decision.

To illustrate this decision, Figure I-1 shows horiz ontal lines at 10 and 20 μ g/L (the LOC range specified in the 2003 IRED). The upper line corres ponds to false positives and false positives being about equal, while the lower line has conside rably more false positives than negatives. It is important to emphasize that these horizontal LOC lines are presented here only to illustrate the concept of using the experimental ecosystem data to position the LOC, and that the methods described in this document will not produce simple, fixed-concentration LOCs.

Therefore, the general problem is to extrapolate ef fects observed under one exposure time-series to another time-series. This extrapolation will al low effects observed for different experimental ecosystem exposures to be compared and synthesized into an LOC, and for this LOC to be used to assess the types of exposures observed in natura l systems. For example, if 20 μ g/L atrazine is an LOC for a 60-day constant exposure, what would b e the LOC for a 30-day constant exposure? Or what would be the LOC based on the highest peak (or some average concentration) for a chemograph with a complex shape, such as those in F igure I-2?

The LOC thus must be a function that can be compute d for the entire range of exposure timeseries encountered in experimental ecosystem studie s and in natural freshwater systems of interest. This requires a model that can provide, for any exposure time-series, a comparable, quantitative measure of expected effects, which wil l be referred to as the model effects index (MEI). For example, this might be a model-calculat ed percent reduction in total plant biomass over a specified duration.Page 27 of 29

By correlating the MEI to the Brock scores for the various experimental ecosystem exposures, an LOC for this MEI ( LOC MEI ) can be determined, so that the LOC is on a modeled effects scale rather than on a concentration scale . Evaluation of the exposure time-series in natura l freshwater systems is then simply done by computing the MEIs for these time-series and comparing them to the LOC MEI . One consequence of such a "calibration" of the L OC MEI to the experimental ecosystem data is that the model is no t required to make absolute predictions of effects in an actual aquatic system, but rather is used to support comparisons of the relative severity of different exposures time-series. This general strategy for assessments is described in further detail below.

Because the experimental ecosystem data do not cont ain sufficient information on timedependence of effects, this model also must provide an acceptable basis for this timedependence. In addition, the model should provide extrapolations among different exposure time-series that are not adjusted for physical, che mical, and biological properties attributes (other than the exposure time-series) of the experimental ecosystems and natural systems of interest. Even if a model is a function of such attributes, i t must be applied with generic values for the attributes because there is insufficient empirical information on the relationship of effects to such system attributes to validate model predictions and because the model must be applicable to systems for which these system attributes are not d efined. With the exception of exposure variability, the influence of system attributes on differences among experimental ecosystem studies and on extrapolations from experimental eco system studies to natural systems cannot be addressed.

2. General Strategy for Assessment Methodology

This section describes the general strategy for atr azine assessments in terms of two major steps: (1) establishing an LOC MEI based on experimental ecosystem data and (2) using the LOC MEI to evaluate exposure time-series for natural freshwate r systems. This general strategy can accommodate MEIs of widely different natures and is described here without specifying the exact nature of the MEI. However, for illustrative purposes, it is useful to provide actual values for some variables, so some calculations were made for the description. The effects model is only characterized as being able to convert an expo sure time-series into a time-series of "percent effects". The MEI is set to the maximum 90-day ave rage of these daily percent effects, this 90- day assessment period being an arbitrary example. The LOC MEI is determined by equalizing false positives and negatives, and the LOC MEI value is for illustrative purposes only.

Establishing a LOC MEI Based on Experimental ecosystem Data

(1) The MEI is determined for each of the atrazine exposures in the experimental ecosystem data set. This MEI must be for a fixed assessment period to provide comparability among different exposures (i.e., effects confined to a short exposu re period cannot be equated to effects extending over a longer exposure period; rather, average effe cts of each exposure over the same period should be compared). The selected assessment perio d must be appropriate to the nature of the MEI, the experimental ecosystem studies, and risk a ssessment goals.Page 28 of 29

(2) The observed Brock scores (1 through 5) for ea ch experimental ecosystem treatment are correlated to their respective MEIs, to determine w hat levels of the MEI are associated with observed effects in the experimental ecosystem. An LOC MEI is determined that best discriminates between the score groups (i.e., 1 or 2 vs. 3 to 5), based on a desired relative incidence of false positives and negatives. Note t hat the LOC is now on the modeled effects scale rather than an exposure concentration scale. Figure I-3 depicts this process. The top panel sho ws the experimental ecosystem data and highlights one of the data points, whose exposure t ime-series is shown in the left middle panel. The effects model converts this exposure data to da ily percent effects in the right middle panel and these daily values are in turn processed into 9 0-day running averages for this example. The MEI is set to the maximum (9.3%) of these running a verages. In the lower panel, the Brock scores versus the MEIs are plotted for all of the e xperimental ecosystem treatments. The LOC MEI is set to the MEI value that equalizes false posit ives and negatives (vertical line at 1.9%).

Using a LOC

MEI to Evaluate Exposure Time-Series from Natural Fres hwater Systems

(3) For each exposure time-series from natural fres hwater systems of interest, the MEI is calculated and compared to the LOC MEI to see if it is exceeded, and by how much. Figure I-3. Strategy for establishing the level of concern for a model effects index based on cosm data

Exposure Duration (days) 2 5 20 50 200 500 10 100Average Concentratio(μg/L Atrazine)

0.1 1 10 100 1000 10000

Time from Start of Cosm Exposure (d) 0 20 40 60 80 100Concentration

0 10 20 30 40 50

Time from Start of Cosm Exposure (d) 0 20 40 60 80 100Perce nt Effect

0 5 10 15 20 25 Daily Value

90-Day Running Average

Model Effects Index 0.2 0.5 2 5 20 50

1 10 100Brock Score

1 2 3 4 5 LOC MEI =1.9 Effects Model Model Effects Index = 9.3 5 4 3
2
1 Brock ScorePage 29 of 29

(4) If desired, iterative model calculations can be conducted to determine the factor ( "risk factor" ) by which the exposure concentration time-series w ould need to be decreased for the MEI to exactly equal the LOC MEI , thereby providing further information for risk as sessment.

Figure I-4depicts this process. The top left panel provides an example field exposure time-series. The effects model converts this exposure data to da ily percentage effects in the top right panel and these daily values are in turn processed into 9 0-day running averages, the maximum of which (6.3%) is the MEI. In this case, the MEI sub stantially exceeds the LOC MEI (1.9% average effect). The bottom panels illustrate the process by which a risk factor can be determined. The model is iteratively applied to scale the exposure time-series (left bottom panel) so that the MEI exactly equals the LOC MEI (right bottom panel). The ratio of the original e xposure time-series to the adjusted one equals the risk factor, which is 3 .5 in this case.

For an exposure time-series of this exact shape, th e bottom left panel of Figure I-4 defines an LOC on a concentration scale basis, and the original exposure time-series would need to be reduced by 3.5-fold to be at the LOC. This concent ration-based LOC can be characterized by any single (e.g. the peak) concentration or any ave rage concentration over the time-series, but if the relative shape of the time-series changes, this concentration-based LOC would not be valid and new model calculations would be needed to deter mine when exposure is at the LOC MEI . Figure I-4. Strategy for using the level of concer n for a model effects index to evaluate field exposures

Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Concentration

0 20 40 60 160 180

Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Concentration

0 10 20 30 40 50

Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Percent Effect

0 2 4 6 8 10 12 14 16 Daily Value

90-Day Running Average LOC MEI Julian Day 0 30 60 90 120 150 180 210 240 270 300 330 360Percent Effect

0 5 10 15 20 25 50 55

Daily Value

90-Day Running Average LOC MEI

Risk Factor is Ratio of Exposures Risk Factor = 3.5

Iteratively Apply Model to Scale Exposure so MEI=LOC MEI Apply Effects Model to Determine if LOC MEI Exceeded

MEI = LOC MEI =1.9% MEI(=6.3%) > LOC MEI (=1.9%)