## Uncertainty Analysis in Ecological Risk Assessments of Pesticides Dwayne R.J. Moore – Intrinsik Environmental Sciences Inc., New Gloucester , ME April 22 nd , 2014

# Why **Uncertainty Analysis?**

• Explicitly acknowledges uncertainty that we know exists • Comprehensive characterization of distribution risk • Not conservatively biased • Better understanding of system, available data, and importance of data gaps • Has been recommended for refined risk assessments by numerous expert panels including NAS Panel

# Types of Uncertainty: Variability

• Arises from natural stochasticity • Parameter variability due to

– Spatial distribution of pesticide concentration

– Temporal fluctuations in rainfall

– Genetic differences among individuals 

# • Not reducible by empirical effort, but can be better understood

# Types of Uncertainty: Incertitude

• Arises from incomplete knowledge • Parameter uncertainty due to

– Limited sample size, presence of non - detects

– Measurement error, systematic error

– Use of surrogate data, poor system understanding 

# • Reducible with empirical effort

# QUA Methods

• Variety of available methods • Several well suited to data - poor assessments • Many more methods than presented here

– Interval analysis

– Fault trees

– Bayesian methods

– Fuzzy arithmetic

– Empirical distributions 

# • Easy - to - use software available for most methods (though education and training required for intelligent use!)

# Some Existing Probabilistic Tools **for Pesticides**

• Joint probability curves (ECOFRAM, 1999; Solomon et al. 2000. Crop Protection 19:649) • Probabilistic PRZM model (Warren - Hicks et al. 2002. ET&C 21:1570) • EPA’s Terrestrial Investigation Model (http://www.epa.gov/scipoly/sap/meetings/2004/033004 _mtg.htm) • LiquidPARAM (Moore et al. 2014. Rev EC&T 231:163), GranPARAM (Moore et al. 2010. IEAM 6:260) • See Application of Uncertainty Analysis to Ecological Risks of Pesticides , Warren - Hicks, W.J. and Hart, A.V. (eds.). SETAC Press, Pensacola, FL. • Many population models ( Railsback , S.F. and V. Grimm. 2011. Agent - Based and Individual - Based Modeling: A Practical Introduction. Princeton Univ Press, Princeton, NJ)

# A Simple Example: Risk to Aquatic **Invertebrate Prey**

• Aquatic exposure

– PRZM/EXAMS: 30 - year distribution of annual peak (acute ), 21 - day average or 60 - day average (chronic ) concentrations of pesticides in standard - sized ponds

– Modified PRZM/EXAMS

– SWAT, AGRO 

# • Effects : Species sensitivity distribution for aquatic invertebrates • Risk = Integration of exposure and effects curves

# Exposure Distribution

0 20 40 60 80 100

0 5 10 15 20 25 30**P r** **o b** **a b** **i l** **i t** **y**

**Concentration (mg/L**  **)** 0 0.005 0.01 0.015 0.02 0.025 0.3

# SSD for Aquatic Invertebrate Prey

0.0 0.2 0.4 0.6 0.8 1.0

0.0001 0.001 0.01 0.1 1 10 100 1000 10000**P r** **o p** **o r** **t i** **o n** **o f** **S p** **e c** **i e** **s**

**Concentration ( m**  **g/L)**

Log Gompertz Model 95% Confidence Limit Data

# Risk Curve

0 20 40 60 80 100

0 20 40 60 80 100**E x** **c e** **e d** **e n** **c e** **P r** **o b** **a b** **i l** **i t** **y (** **% )** 

**Percent of Species**  **Af fected**  **(%)**

# Monte Carlo Simulation **How?**

– replace each point estimate with a distribution

– repeatedly sample from each, tally answers in a histogram **Why?**

– simple to implement and explain

– summarizes entire distribution of exposure or risk **Why not?**

– requires a great deal of empirical information

– may need to invoke expert judgment in places

– difficult to quantify dependencies

# Monte Carlo Analysis

**Generate Output** 

**Risk F** **r e** **q u** **e n** **c y**  **Summarize Results**  **Dependencies Risk =**  **f (**  **x,y ,z**  **) Risk**  **Equation Analysis**  **Fine Tune**  **Input Variable y**  **Input Variable x** **F r** **e q** **u e** **n c** **y Input**  **PDFs Input Variable**  **z**

# Monte Carlo Output

0 20 40 60 80 100

0 50 100 150 200 250 300 350

**To tal Body** 

**B urden (mg**  **ai /kg**  **bw )** **C u** **m u** **l a** **t i** **v e** **P r** **o b** **a b** **i l** **i t** **y (** **% )** **.** 

## American widgeon gorge feeding on alfalfa fields on the day of application of pesticide X at 1 lb a.i ./A

# Propagating Variability and **Incertitude**

• Most PRAs combine variability and incertitude • NAS, EPA, DOE, etc. emphasize importance of tracking variability and incertitude separately • Major advantages:

– Allows assessor to be forthright about what is known and what is not

– Relative importance of variability and incertitude can be quantified

# Second **- order Monte Carlo**  **How?**

– let parameters of input distributions be distributions

– nest the Monte Carlo analyses **Why?**

– partitions variability and incertitude

– plots get narrower with better empirical information **Why not?**

– difficult to include some sources of uncertainty

– results are cumbersome to interpret and explain

– often difficult to separate incertitude and variability**Constants Random V**  **ariables We**  **ll Characterized**  **Having Incertitude**  

### Inputs

**D 1**  **2 3**  **4 5** **P (** **D )**  **B 1**  **- 5** **P (** **B )** 

**C 2**  **5 1**  **3 4** **P (** **C )**  **A**

**1 -**  **5 P** **( A** **)**



### Output **Va riability**  **ƒ (A**  **,B,C 1**  **,D 1**  **) ƒ**  **(A ,B,C**  **2 ,D**  **2 )**  **ƒ (A**  **,B,C 3**  **,D 3**  **) ƒ**  **(A ,B,C**  **4 ,D**  **4 )**  **ƒ (A**  **,B,C 5**  **,D 5**  **)** 

# Second **- order Monte Carlo** 

# Second **- order Monte Carlo Output** 

# Probability Bounds Analysis **How?**

– specify what you are sure about

– establish bounds on probability distributions

– specify dependencies (no assumption, independence, perfect, etc.) **Why?**

– accounts for incertitude about parameter values, distribution shape, and dependencies

– can bound on 1 st 

## order Monte Carlo results, bounds get narrower with better empirical information **Why not?**

– cannot handle second - order probabilities

– optimum bounds difficult to compute when variables repeated

# Probability Bounds Analysis **Output**

0 20 40 60 80 100

0 20 40 60 80 100 **% Decline in Fecundity** **E x** **c e** **e d** **e n** **c e** **P r** **o b** **a b** **i l** **i t** **y (** **% )**  Monte Carlo

LPB - Lower Probability Bound UPB - Upper Probability Bound Low - Intermediate Criterion Intermediate - High Criterion

# Decision Making With Uncertainty **Analysis**

• “Bright lines” lacking for the outputs of PRA • Probabilistic bright lines can assist communication and decision making • Bright lines will likely differ between receptor groups

‒ Indirect vs direct effects

‒ Low a nd high fecundity listed species

‒ Threatened vs endangered 0 20 40 60 80 100

0 20 40 60 80 100**E x** **c e** **e d** **e n** **c e** **P r** **o b** **a b** **i l** **i t** **y (** **% )** 

**% Species or Individuals**  **Affected**



## Low **r isk?**  **Intermediate risk? High risk?**  **Mitigation required?**

# IUCN Red List of Threatened **Species**

• Widely recognized for evaluating conservation status of plant and animal species • When uncertainty leads to a w ide range in results of species assessments, range of possible outcomes should be specified • Single category (e.g., critically endangered, vulnerable, least concern) must be chosen and basis for decision documented • Uncertainty may be represented as a plausible range around a best estimate 0 20 40 60 80 100 1 0.8 0.6 0.4 0.2 0 **Time (years) P** **r o** **b a** **b i** **l i** **t y**  **critical endangered**  **vulnerable P** **r o** **b a** **b i** **l i** **t y** **o f** **E x** **t i** **n c** **t i** **o n** 

# Assessing Risks of Pesticides to **Endangered Species**

• Use simple techniques (e.g., risk quotients compared to levels of concern) to identify risk scenarios of potential concern • Conduct sensitivity analysis to identify critical variables – guides data collection, indicates degree of confidence in QUA output • Use Monte Carlo analysis in higher tier assessments – but consider combining with a 2D method to determine degree of incertitude • Evaluate model performance and/or relevance and veracity of empirical data • Collect other lines of evidence (e.g., empirical data, field studies), conduct weight - of - evidence evaluation • List sources of uncertainty not included in analyses – discuss possible influence on risk estimates • Engage risk manager, trustees, industry, public, etc. throughout process

# Selected References

• Akçakaya , H.R. and Ferson , S. 2001. RAMAS ® Red List: Threatened Species Classifications under Uncertainty. Version 2.0. Applied Biomathematics, New York. • Akçakaya , H.R., Ferson , S., Burgman , M.A., Keith, D.A., Mace, G.M. and Todd, C.A. 2000. Making consistent IUCN classifications under uncertainty. Conservation Biology 14: 1001 - 1013. • Ferson , S., Moore, D.R.J., Van den Brink, P.J., Estes, T.L., Gallagher, K., O’Connor, R. and Verdonck , F. 2010. Bounding uncertainty analyses. In Application of Uncertainty Analysis to Ecological Risks of Pesticides, Warren - Hicks, W.J. and Hart, A.V. (eds.). SETAC Press, Pensacola, FL. pp. 89 - 122. • Moore, D.R.J., R.S. Teed, S.I. Rodney, R.P. Thompson and D.L. Fischer. 2010. Refined avian risk assessment for aldicarb in the United States. IEAM 6(1): 83 - 101. • Moore, D.R.J., Warren - Hicks, W.J., Qian , S., Fairbrother, A., Aldenberg , T., Barry, T., Luttik , R. and Ratte , H. - T. 2010. Uncertainty analysis using classical and Bayesian hierarchical models. In Application of Uncertainty Analysis to Ecological Risks of Pesticides, Warren Hicks, W.J. and Hart, A.V. (eds.). SETAC Press, Pensacola, FL. pp. 123 - 141. • Morgan, M.G. and M. Henrion . 1990. Uncertainty: A Guide to Dealing With Uncertainty in Quantitative Risk and Policy Analysis. Cambridge University Press, Cambridge, U.K. • Tucker, W.T., S. Ferson , A.M. Finkel and D. Slavin . 2008. Strategies for risk communication: Evolution, evidence and experience. Annals of the New York Academy of Sciences, Volume 1128.

# ESA Workshop

April 22, 2014 

# Opportunities for Improvements Summary

# Next Steps

An on - going public dialogue on endangered species assessment process and methods Identify which recommendations:
• can be adopted immediately
• merit further discussion
• schedule future meetings to move these forward

**25 26**  

## • Build on EPA FIFRA Risk Assessment

• Analyses should be conducted separately for direct and indirect effects
• Include taxon specific thresholds
• Include biological data and habitat requirements
• Develop focal species for crops and use biology to estimate risk to species
• Refine exposure estimates/upgrade models to include greater realism in the risk assessment 

# Suggestions for May Affect / No Effect Decision

# Examples of Further Steps in Developing Approach

• Adopt NAS recommendations regarding uncertainty analyses  Implement qualitative and quantitative approaches in future assessments  Communicate uncertainty
• Adopt a systematic W. of Evidence approach  Include a scoring system for strength of evidence
• Develop Additional Guidance  Probabilistic risk assessment  Interpretation of results of assessment  Population Models  Higher Tier RA tools / identified needs
**27**