Approaches to Evalu

Approaches to Eval

Water Sampling Stra

Water Sampling Str

and Frequency of M

and Frequency of M

Presentation to Science Advis

April 26-29, 2010

Nelson Thurman and Mary Fra

Office of Pesticide Progr

Michael Messner, Ph.D

Office of Ground Water and Drin
Why We’re Visiting Sampling 

re Visiting Sampling 

z Based on the best available data for the 20

IRED, EPA identified a human health level on a 90-day exposure period. If a shorter dexposure is considered:

 How well will the existing CWS monitoring desig

shorter durations of exposure?

 Does the existing monitoring program need to i

sampling or is it sufficient for a shorter duration 

z For atrazine and other pesticides in drinkin

 How much can we derive from existing, less fre How frequent does sampling need to be in orde

characterize various exposure durations?
Overview: Sampling Freq

Overview: Sampling Freq

z Context for water sampling frequen

evaluations

 Considerations in evaluating monitoring and s

frequencies

 Overview of approaches for analyzing monito

z Examples of approaches / applicatio

 Evaluating the Performance of Sampling Strat

Estimating Maximum Concentrations of Differ

 Artificial Neural Network Modeling of Atrazine 

Patterns 

z What’s next?

 Planned approach to evaluate methods / mod Issues for the SAP
Nature of Pesticide Occurrenc

Nature of Pesticide Occurren

z Complex spatial patterns

 Related to pesticide use, rainfall patterns, soil/h

vulnerabilities

z Complex temporal patterns

 Variability within seasons Year-to-year patterns

z Temporal autocorrelation – pesticide conc

measured on one day are related to the exmeasured on the previous and following d

z Censored data - pesticide concentrations b

LOD/LOQ

z Infrequent sampling
Nature of Nature of Atrazine Atrazine Occurrenc

Occurrenc

Spatial Patterns Spatial Patterns
MO-01, 2008

0

20406080

100

4/1 4/15 4/29 5/13 5/27 6/10 6/24 7/8 7/22 

Date

Atrazine, ug/L

Temporal patterns: within a si

Temporal patterns: within a si

z Sin

pulsho

z Cen

(beLOQ
Temporal patterns: different y

Temporal patterns: different y

MO-01, 2007

0

20406080

100

4/1 4/15 4/29 5/13 5/27 6/10 6/24 7/8 7/22 

Date

Atrazine, ug/L

z Tw

pultail

 D

ape

z Aut
MO-02, 2006

0

20406080

100

4/1 4/15 4/29 5/13 5/27 6/10 6/24 7/8 7/22 8

Date

Atrazine, ug/L

Temporal patterns: different y

Temporal patterns: different y

z Bro

pultail

z Mu
MO-01, 2004-2009

0

20406080

100120

J-04 J-05 J-06 J-07 J-08 J-0

Date

Atrazine, ug/L

Temporal patterns: year Temporal patterns: year-to-yea

yea

MO-02, 2004-2009

0

20406080

100120

J-04 J-05 J-06 J-07 J-08 

Date

Atrazine, ug/L
Sampling Frequency Consid

Sampling Frequency Consid

z Accounts for spatial, temporal patterns in e

 Targeted monitoring focuses on where the pest

it is used

 Accounts for fate/transport of pesticide, nature 

weather patterns, water body/source

z Exposure duration of concern for adverse h

and the concentrations / durations associa

 More intensive sampling for shorter exposure du

z Consequences of false negatives (failing to 

exposures) and false positives (false alarm

z Cost
Previous Recommendations o

Previous Recommendations o

Frequency Frequency

z ILSI (1999) recommendations for estimat

exposures:

 Minimum of 100 samples per year targeting th

seasons (beginning of application, maximum uapplication) at a single CWS

 Less frequent sampling, more CWS for regiona

assessments

z Crawford (2004) evaluated various sampl

recommended concentrating samples durrunoff period

z Both noted the number, frequency of sam

on the time frame of concern
Analyzing and Interpreting Mon

Analyzing and Interpreting Mon

z Basic approaches

 Distribution of measured values only Linear or stair-step interpolation between sa

z Statistical methods

 Bootstrapping Artificial neural networks Kriging / nearest neighbor

z Modeling methods

 Regression-based or deterministic
Interpreting Between Sampl

Interpreting Between Sampl

MO-01, 2007

0

20406080

100

4/1 4/15 4/29 5/13 5/27 6/10 6/24 7/8 7/22 8/5 

Date

Atrazine, ug/L

Original chemogr

MO-01, 2007

0

20406080

100

4/1 4/15 4/29 5/13 5/27 6/10 6/24 7/8 7/22 8

Date

Atrazine, ug/L

MM

Sampleday int

MO-01, 2007

0

20406080

100

4/1 4/15 4/29 5/13 5/27 6/10 6/24 7/8 7/22 

Date

Atrazine, ug/L

Stairinterbetw

MO-01, 2007

0

20406080

100

4/1 4/15 4/29 5/13 5/27 6/10 6/24 7/8 7/22 

Date

Atrazine, ug/L

Linintbe
Broader Sampling Inter

Broader Sampling Inter

MO-01, 2007

0

20406080

100

4/1 4/15 4/29 5/13 5/27 6/10 6/24 7/8 7/22 

Date

Atrazine, ug/L

MO7-dMO

Sampl14-day interva

MO-01, 2007

0

20406080

100

4/1 4/15 4/29 5/13 5/27 6/10 6/24 7/8 7/22 

Date

Atrazine, ug/L

MO7-dMOline

Loss inforwith inter
Basic Interpolation Appro

Basic Interpolation Appro

z Advantages

 Quick, simple May be all that’s needed for certain exposure durat

frequencies

z Disadvantages

 Assumes no concentrations greater than the maxim Likely to underestimate peak or short-term exposur

frequency, may overestimate longer-term exposure

 Use only the distribution of monitoring data or fill in 

z How well they characterize actual concentration p

timing, sampling frequency, shape of the true che
Statistical Analyses Statistical Analyses

MO-01, 2007

0

20406080

100

4/1 4/15 4/29 5/13 5/27 6/10 6/24 7/8 7/2

Date

Atrazine, ug/L
Kriging Kriging / Nearest Neighbor 

/ Nearest Neighbor 

z Kriging

 Based on variance at different lag 

times

 linear interpolation with 

appropriate weighting from nearest neighbors 

z Nearest Neighbor

 Evaluation of correlation (or 

variance) of pesticide concentration at different lag separation times (days)

Variogra

0

0.0020.0040.006

1 2 

ppb2
Preliminary Evaluations with Preliminary Evaluations with 

Visual Inspection of Data

Descriptive Statistics

Variogram Analysis

Cross-Validation

Kriging Stochastic 

Simulation

Julian 

80 100 120 140 160 

Concentration (ug/L)

0123456
Statistical Approache

Statistical Approache

z Advantages

 Can be used to provide probabilistic distributions

bounds around existing monitoring data

 Do not necessarily assume greatest exposure ha Non-subjective estimate of missing values; can a

temporal correlation

z Disadvantages

 More complex modeling than basic interpolations May underestimate high exposures with data sm May not capture underlying explanatory variable

z Utility may depend on both quality/frequency o

monitoring data, toxicity exposure duration of c
Modeling Methods Modeling Methods

z Regression-based models

 Watershed Regression on Pesticides (WAR

using atrazine monitoring data to predict pesticin unmonitored/ inadequately monitored streamestimates of selected percentiles, annual mean, and annual maximum moving-average (21-, 60-durations) concentrations.

z Deterministic models

 For the 2007 SAP on the atrazine ecological exp

program (AEEMP), Syngenta investigated the poof AEEMP chemographs with edge of field exposPesticide Root Zone Model (PRZM)

z Flow-adjustments or combination of flow / 

planting / rainfall events
Examples of Approaches / Ap

Examples of Approaches / Ap

z Evaluating the Performance of Sampling 

for Estimating Maximum Concentrations Durations – Mary Frankenberry

 Use of bootstrapping methods to evaluate d

frequencies with different pesticide concentr

z Artificial Neural Network Modeling of Atr

Occurrence Patterns – Michael Messner

 Use of ANN to simulate pesticide concentrat

evaluate monitoring strategies
Evaluating the Performan

Evaluating the Performan

Sampling Strategies for E

Sampling Strategies for E

Maximum Concentrations 

Maximum Concentrations 

Durations Durations

Presentation to Science Advi

April 26-30, 2010

Mary J. Frankenberr

Environmental Fate & Effect

Office of Pesticide Prog
Introduction Introduction

z Recent work by Crawford (2004) a

has evaluated the performance of sampling strategies for estimating percentiles of “true” chemograph 

z In the SAP issue paper OPP carried 

bootstrapping methods that were on the Crawford approach, one of be presented today. 
Background Background

z Crawford’s work created “true

chemographs using linear inteto augment less than daily sam365-day profiles.

z OPP chose results from 30 Co

Water Systems (CWS) which wsampled weekly or biweekly aaugmented linearly to 365 day
Background Background

z While earlier work looked at the u

percentiles of single day concentrais concerned with the maximum vaseveral different durations, ranginsingle day values to 3-, 7-, or 90-daverages. 

z Similar to earlier methods, OPP loo

performance of different sampling for computing estimates of the “trfor these maximum running avera
Methodology Methodology

z Sampling was carried out using the inte

profiles according to schedules which raquarterly to sampling every 4 days.

z Each sampling strategy was implemente

times with replacement following a randEach resample was interpolated to 365 maximum single day, maximum 3-day, average of each run was compiled, with and percentiles of these maximum value
Matrix of Sampling Strategy v. Co

Matrix of Sampling Strategy v. Co

Duration for Simulations Duration for Simulations

Concentration Duration

Single Day peak

3-day average 7-day a

Sampling IntervalDaily Seasonal XX Every 4 days XX Every 7 days XX Biweekly XX Weekly/ Biweekly Seas.

XX 

Quarterly XX 
CWS Chemograph Chemograph with a Moderate Dura

with a Moderate Dura

extended tail extended tail
Percentiles of Maximum Daily Concentratio

Percentiles of Maximum Daily Concentrati

Sampled at 4 Sampled at 4-, 7-, or 14 , or 14-day Intervals day Intervals
Percentiles of Maximum 7 Percentiles of Maximum 7-Day Average Con

Day Average Con

CWS #21 Sampled at 4 CWS #21 Sampled at 4-, 7-, or 14 , or 14-day Inter

day Inter
Results for CWS #21 Results for CWS #21

z Results in these tables show that f

#21 (and consistent with all syste

 Accuracy is best for strategies emplo

frequent sampling

 Accuracy generally improves for long

averages (e.g., 7 days or 90 days) ovterm averages--single day or 3–day, 
CWS Chemograph Chemograph with a Single, relative

with a Single, relative

Duration (narrow) peak Duration (narrow) peak
Chemograph Chemograph with a Broad or Extended 

with a Broad or Extended 
Maximum 3 Maximum 3-Day Average Concentrations f

Day Average Concentrations f

Sampled at 4 Sampled at 4-day Intervals day Intervals
Maximum 3 Maximum 3-Day Averages for All CWS Sampling Day Averages for All CWS Sampling Every D

Every D

Every 14 Days (balance of year) Every 14 Days (balance of year)
Maximum 90 Maximum 90-Day Averages for All CWS Sampling Day Averages for All CWS Sampling Every 7 D

Every 7 D

Every 14 Days (balance of year) Every 14 Days (balance of year)
What do these indicate? What do these indicate?

z As expected, an endpoint with a s

duration will require more frequenthan a longer one to capture corremonitoring values with similar acc

z The current sampling scheme of 

weekly/biweekly monitoring appeaperform adequately for estimating averages with reasonable to good 
Indications (cont. Indications (cont.

’)

z Seasonally targeted sampling strat

generally perform better than allocsamples equally throughout the ye

z Sampling strategies appear to perf

differently according to the shape chemograph. 

z It may be useful to examine the s

properties of annual profiles in the CWS with the highest peaks over t
Atrazine Atrazine Monitoring Program Data for Monitoring Program Data for A

A

Raw Water* Raw Water*

Year

Tot. Samples/Tot. CWS

Ind Peaks > 12 ppb

Ind Peaks > 20 ppb

Ind Peaks > 30 ppb

CWS wpeak(s) 12 ppb

2004 4429/ 141 19

71301049

6 2 15 

2005 4779/ 142 22 7 27 2006 4817/ 151 11 2 20 2007 4521/ 137 1 0 7 

2008 3927/ 134 19 10 23 

* Source: Raw data submitted by Syngenta annually to EPA
Characterizing Uncertainty Characterizing Uncertainty

z The Agency seeks the Panel’s 

on methods for characterizing uncertainty:

 for the population of systems f

year

 for individual systems within a g

with missing values from less thsampling
Summary of Results Summary of Results

z Results of this preliminary exercise 

Crawford and others in pointing to sampling strategies that can targespatial and temporal patterns amoCWS

z More intense seasonal sampling am

higher concentration systems in vuareas may be a good allocation of resources
Artificial Artificial Neural Netw

Netw

Modeling Modeling of Atrazine AtrazineOccurrence Occurrence Patterns Patterns

Presentation to Science Advisory PanApril 26, 2010Michael J. Messner, Ph.D.Standards and Risk Reduction BrancOffice of Ground Water and Drinking 
• Mathematical Form of ANN mo• Fitting Occurrence Data• Simulation to Assess Performa• Some Pros & Cons

Outline Outline
Mathematical Model Mathematical Model

• Above, the “smooth” part of a four-node • The first parameter, b0, establishes the 

concentration – before any node begins effect

• Each fraction is called a “node,” and two 

to describe a peak (one for the increase for the decrease in concentration)

• Parameters in numerators are the total effec• Parameters in the denominators determine w

year) and how quickly (slope) the increases occur

lnCday b0 b1

1 e

− b5 b6 day ⋅−

+

+

b2

1 e

− b7 b8 day ⋅−

+

+

b3

1 e

− b9 b1

−

+

+

:=
Estimating Model Paramete

Estimating Model Paramete

1. With basic understanding of the p

set initial values.

2. Calculate log likelihood, given init3. Use some algorithm to maximize 

likelihood.

– MS Excel’s Solver Add-In– MathCAD’s Maximize function– R function mle in stats4 package

lnCday b0 b1

1 e

− b5 b6 day ⋅−

+

+

b2

1 e

− b7 b8 day ⋅−

+

+

b3

1 e

− b9 b1

−

+

+

:=
Finding Initial Values for Maum

Finding Initial Values for Maum
Maumee 1998, a Best Maumee 1998, a Best-Fit A

Fit A
Maumee 1998 Three Nodes Maumee 1998 Three Nodes AutoC

AutoC

Function (ACF) Function (ACF)
Model for Maumee 1998 Model for Maumee 1998

z Three nodes z Include autocorrelation (AR2)

Predday b0

b1

1 e

− b4 b5 day ⋅−

+

+

b2

1 e

− b6 b7 day ⋅−

+

+

:=

+

Var := 0.05895229

α

0 := 0.3807437

α

1 := 0
Summary Statistics Summary Statistics

z Given the model parameters, can 

summary statistics.

 Mean Median  Other percentiles of interest

z But what if we’re concerned about 

(false pos and false neg) when occmuch higher or lower than this??? 

model.
Adjusting the Model Adjusting the Model

z Adding a constant K to b0

 Daily concentrations increase by fact If K < 0, the result is an decrease.  If K > 0, the result is an increase. Mean, median and other percentiles 

exp(K)

 Error structure (lognormal with serial 

is preserved
Illustration: One Illustration: One Sim Ye

(same random number seed, but differe

(same random number seed, but differ
Many (10,000) Many (10,000) Sim Years

z Simulate one year of occurrence, a

each year, apply the monitoring st

 Select concentrations for days on wh

monitoring is planned and

 Employ the rule for deciding whether 

is high or low

z 10,000 -> Good estimates of erroz Repeat, using different levels to “t

positive 
Error Rates as Functions of Cr

Error Rates as Functions of Cr
This kind of outcome is This kind of outcome is 

“No

z Decision-makers initially express great concern for bo

errors, then find that

 Rates can be difficult to meet with an affordable strateg Satisfying initial objectives may require large number o

z When this happens, need to reconsider things.

 Can we live with greater F(+) and F(-) error rates? Was the simulated occurrence pattern “typical,” or chal

challenging, what are F(+) and F(-) rates for more typipatterns? Can we live with that?
Summary Summary

z ANN Models can be used to simulate oc

patterns of concern...

 Low occurrence levels, for which concern is  High occurrence levels, with concern for F(-) High-variability patterns that are difficult to 

samples.

 Low-variability patterns that are easy to asse

samples.

z ... and assess candidate monitoring stra

evaluating in terms of their F(+) and F(
Some Advantages and Disadva

Some Advantages and Disadva

Using ANN Models Using ANN Models
z Context for water sampling frequen

evaluations

 Considerations in evaluating monitoring and s

frequencies

 Overview of approaches for analyzing monito

z Examples of approaches / applicatio

 Evaluating the Performance of Sampling Strat

Estimating Maximum Concentrations of Differ

 Artificial Neural Network Modeling of Atrazine 

Patterns 

z What’s next?

 Planned approach to evaluate methods / mod Issues for the SAP

Overview: Sampling Freq

Overview: Sampling Freq
What We Know So F

What We Know So F

z Pesticide concentrations vary spatially and z Timing and frequency of sampling needed 

pesticide exposures at a given location deon the temporal patterns but also on the duration for the toxicological endpoint of 

z The shape of the exposure chemograph a

the frequency of sampling needed

z The complexity of the approach needed to 

exposure estimates depends on the qualitmonitoring data, exposure duration of concertainty needed
Planned Approach to Evaluat

Planned Approach to Evaluat

z Use the robust monitoring data sets (with 

intensive sampling during use/runoff seasrepresent the three general exposure shamoderate-, and long-duration/multiple pe

z Sample these data sets at different interv

14, 30- day sampling intervals).

z Use the various approaches described in S

estimate the exposures between sampling each of the sampled intervals.

z Generate estimated exposures for varying 

average periods (1, 3, 7, 14, 30, 60, 90 dcompare those estimates against the actu
Ultimate Focus Ultimate Focus

z How do we use this in drinking water exp

assessments?

z Taking into account...

 The spatial and temporal nature of pesticide c

drinking water sources

 The duration of exposure for the toxicological 

concern

z How frequent does sampling need to be i

adequately characterize various exposure a specified confidence interval?

z How much can we derive from existing, le

sampling?
Issues for the SAP Issues for the SAP

z Given the nature of the temporal patter

pesticide occurrence in surface waters, best determine confidence bounds on exestimates from monitoring data?

 Converse of this: how do you design a moni

provide exposure estimates with a given leve

z What are the advantages and disadvant

various approaches for estimating pesticconcentrations in drinking water over shmoderate, and long durations?

 When is ‘simple’ good enough and when do 

at more complex methods?