U.S. EPA Office of Pesticide Programs

Public Workshop ~ April 2014 ~ Arlington, VA

Assessing Pesticide Risks

Quantitative and Weight-of-Evidence Approaches

Bruce K. Hope, Ph.D. 
2

 Committee on 

Ecological Risk 

Assessment Under 

FIFRA and ESA

– Report: April 2013

 CropLife Science 

Forum

– May 2013

 Agency efforts

– “Interim Approaches for 

National-Level Endangered 

Species...Assessments”

National Research Council (NRC)
Interim Memo: It’s a Start, but...

 Point of departure for exposure is not defined

– What will the EEC represent? Median, mean, 95%?

 Point to point comparisons (EEC to LD0.000001, etc.) 

are not “risk” estimates

– They are hazard or threshold assessments

 Step 1 & 2 hazard assessments produce “risk quotients” 

that are not easily transferable to Step 3 stochastic 

population models

– A common basis in probability (risk) is missing

3
Informing Decisions with Risk

 If I plan to make decision X (to register a pesticide)...

 What is the probability (p(Y)) that detrimental outcome Y 

will occur in the future? [where p(Y) is the risk estimate]

 What is my confidence in that estimate of p(Y)?

– Where confidence is affected by variability and incertitude

 Acceptability of p(Y)’s value is strictly a policy choice

– But knowledge of it’s size (large or very small) is an important 

component of informed decision making

4
5

Risk Characterization

Risk Estimate

Probability of 

eliciting a 

specified 

response in an 

organism.

CONCENTRATION

P

R

O

B

A

BILIT

Y

EXPOSURE (EEC) EXPOSURE-RESPONSE
Risk as Common Ground

Probability 

of Effect on 

Population

Exposure

ExposureResponse

Exposure 

Scenario

Population

Model(s)

EPA: Steps 1 & 2

Services: Step 3

Risk

estimate

(%)

6

Risk 

Characterization

CONCENTRATION

P

R

O

B

A

BILIT

Y
Beyond the Risk Estimate

 But what if a “risk estimate” isn’t enough?

– For decision makers, constituents, or stakeholders?

 Sadly, both hazard assessments (thresholds) and risk 

assessments (probabilities) are viewed with suspicion

– “Risking it away” (with numbers), cancer lottery, etc.

 Fortunately, there are methods for bringing more 

evidence – other than just a risk estimate - to bear on a 

pesticide registration decision

7
Weight-of-Evidence (WOE)

 Multiple lines-of-evidence (LOEs) are defined by the risk 

hypothesis (or question) – from Problem Formulation

 Weighting of each LOE is determined by how well it 

addresses that hypothesis (question)

– Weighting criteria, assigning weights, scaling weights

 Multiple LOEs (for the same hypothesis) converge to 

provide a WOE informed estimate of risk and of our 

confidence in that estimate

8
9

WOE Within EPA’s ERA Framework

Analysis

Plan

Measure of

Exposure

Measure of

Effect

Measure of

Characteristics

Line of Evidence

Exposure

Line of Evidence

Characteristics

Line of Evidence

Exposure/Response

Tools

Field (surveys)

Field (manipulation)

Laboratory Testing

Models

Legacy Data

Literature

Expert Elicitation

Graphic

stressor>exposure>response

Conceptual Model

Narrative

Risk

hypothesis

Exposure (x) to stressor (y) could, with 

probability (p), yield an adverse 

response (z) of magnitude (m).

Attribute

Entity

(receptor)

Assessment Endpoint

Problem Formulation

To Analysis & Risk 

Characterization 

Phases
10

Integrating Lines-of-Evidence

Line of Evidence

Exposure

Line of Evidence

Characteristics

Line of Evidence

Exposure/Response RE, W

Evidence Group 1

Line of Evidence

Exposure

Line of Evidence

Characteristics

Line of Evidence

Exposure/Response RE, W

Evidence Group n







WOE MATRIX
11

WOE Interpretation: An Example

Assessment Endpoint (entity, attribute): Salmon population, recruitment

Effect Endpoints: Premature egg hatching, juvenile mortality

Risk Hypotheses

RH 1, RH 2: Likelihood that effect endpoint will be realized due to DO concentrations in surface water.

RH 3, RH 4: Likelihood that effect endpoint will be realized due to Cu concentrations in surface water.

Evidence Group Weight (W)

1-2 2-3 3-4 4-5

R IS

K

E

S

T IM

A

T

E

> 75%

51-75%

25-50% 

5-24%  

< 5% 

Moderate to high 

confidence of a low 

to moderate risk 

Evidence 

Group #
Weight-of-Evidence Pros & Cons

 Pros

– Organize and utilize myriad types of ecological data

– Condense complex analyses for use by decision makers

– Can easily combine different tests / metrics

– Much more robust than a single LOE (e.g., a “risk” quotient)

 Cons

– Regulatory restrictions on use (CWA standards)

– Lack of established federal or state guidance

– Limited prior use in a variety of regulatory environments

– The enduring comfort of threshold-based decision making

– Budget and time constraints / additional expertise

12