Section 2.2.3:

Comparison of ToxCast data and Tier 1 ER assays for 

List 1 chemicals

Section 2.2.4:

Demonstration of a CompTox Approach to Evaluating 

ER Agonist Bioactivity for EDSP Universe Chemicals

Patience Browne

Exposure Assessment Coordination and Policy Division

Office of Science Coordination and Policy

Office of Chemical Safety and Pollution Prevention

Slide 1 of 17
Presentation Roadmap

Estrogen Receptor 

Bioactivity

Introduction

Androgen Receptor 

Bioactivity

Exposure

Toxicokinetics/Dosimetry

Integrated Bioactivity 

Exposure Ranking

Future Directions

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 2 of 17
Major Points of Section

 ER computational model is most mature and used 

as an example

• EDSP originally established to evaluate environmental 

chemicals with estrogen agonist activities

• Consistent with this, Tier 1 battery has more extensive 

coverage of estrogen agonist than other bioactivities 

or endocrine pathways

• ToxCast assays are similarly weighted towards ER 

activity

 EPA evaluated each proposed application of 

computational tools in a fit-for-purpose manner

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 3 of 17
Introduction

 Follow-up of the January 2013 SAP meeting “Prioritizing the 

Universe of Endocrine Disruptor Screening Program (EDSP) 

Chemicals Using Computational Toxicology Tools” SAP 

report commented: 

• The approach presented is appropriate for prioritizing the EDSP 

chemical inventory to identify chemicals that may initiate ERmediated events.

• Screening techniques can be conducted relatively quickly and 

efficiently and reduce unnecessary animal testing

 Further refinement to the approach based on SAP 

recommendations

• 18 (vs 8) assays integrated into a network model

• More extensive set of reference chemicals

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 4 of 17
Objectives: ER Pathway

 Before using the model of ER bioactivity in the 

EDSP framework, EPA is reviewing the 

scientific strengths and limitation for each 

goal outlined in the white paper

1. Prioritize chemicals for further EDSP screening 

and testing based on estimated bioactivity

2. Contribute to the weight of evidence evaluation 

of a chemical’s potential bioactivity

3. Substitute for specific endpoints in the EDSP Tier 

1 battery

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 5 of 17
Proposed Methodological Approach 

 Use computational tools to model ER 

bioactivity – focus on agonism

• 16 ToxCast assays relevant to agonist 

pathway

• Model AUC for R1 (agonist) and R2 

(antagonist) activities

 Prioritize chemicals in EDSP universe for 

ER agonist bioactivity

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 6 of 17
Scientific Confidence Needed

To Use [Any] New Assay

 Are reference chemicals that include a range 

of structures and potencies accurately 

detected?

• in vitro reference chemicals

• In vivo reference chemicals

 How well do new methods compare with 

current methods?

• ToxCast versus Tier 1 results

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 7 of 17
Scientific Confidence: ER Reference Chemicals

 Excellent performance 

of ER AUC scores 

against in vitro 

reference chemicals

 Excellent performance 

of ER AUC scores 

against in vivo 

reference chemicals

Table 2.9

# True Pos 28

# True Neg 12

# False Pos 1

# False Neg 1

PPV 0.97

NPV 0.92

BA 0.95

Sensitivity 0.97

Specificity 0.92

# True Pos 28

# True Neg 12

# False Pos 0

# False Neg 4

PPV 1.0

NPV 0.75

BA 0.94

Sensitivity 0.88

Specificity 1.0

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 8 of 17
Scientific Confidence: Current Methods

 How do HTS results compare with ERmediated response in validated Tier 1 assays?

Tier 1 Battery ER-specific endpoints

Tier 1 Screening 

Battery

Endocrine Pathway 

E + E - A + A -

HPG

Axis

HPT

Axis

in vitro

ER Binding ■ ■

ERα Transcriptional 

Activation ■

In vivo

Uterotrophic ■

Pubertal Female ■ ■ ■ ■

Fish Short-term 

Reproduction

(male & female)

■ ■ ■ ■ ■

NOTE: Full EDSP Tier 1 battery includes five in vitro and six in vivo assays that provide additional information on a chemical’s potential to interact 

with the androgen and thyroid, as well as alter steroidogenesis.

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 9 of 17
CODE ER AGONIST

(R1)

ER ANTAGONIST

(R2)

T1 ER

BINDING T1 ERTA T1 UT

17 0.04380 Equiv Neg Neg

23 0.04130 Neg 19.10% Neg

28 0.03240 Neg 18.40% Neg

22 0.0270 Neg 15.80% Neg

50 0.02210 Neg OSRI (P) Neg

8 0.02030 Equiv Neg Neg

9 0.01520 Equiv OSRI (P) Neg

40 0.009850 OSRI (N) Neg Neg

38 0.00950 Neg 24.70% OSRI (N)

11 0.006460 Neg Incl Neg

5 0.003570 OSRI (N) OSRI (N) OSRI (N)

5100 OSRI (N) Neg Neg

1200 Neg Incl Neg

2000 Neg Neg Neg

1000 OSRI (N) Neg Neg

1300 OSRI (N) Incl OSRI (N)

2900 Neg Neg Neg

2600 OSRI (N) OSRI (N) OSRI (N)

3100 Neg Neg Neg

320 0.000077 Neg Incl Neg

270 0.000133 Neg Neg Neg

3300 Neg Incl Neg

3400 Neg Neg Neg

3500 Neg Incl Neg

3600 Neg Neg Neg

1600 Neg OSRI (N) Neg

3700 Neg Neg Neg

3900 Neg Neg Neg

4100 OSRI (N) OSRI (P/N) OSRI (N)

4200 Equiv Neg Neg

4300 Neg OSRI (P/N) Neg

400 OSRI (N) Neg OSRI (N)

4400 OSRI (N) Neg OSRI (N)

Table 2.12 Slide 10 of 17
Scientific Confidence: Current Methods

 Excellent performance of ToxCast AUC scores 

against draft results of Tier 1 ER binding, ERTA, 

and uterotrophic bioassay 

• 100% concordance

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 11 of 17
HTP ER Agonist Bioactivity

 Chemical Space:

~10,000

1444

98

368

56

49

40

• EDSP Universe = ~10,000 chemicals

- ~7000 discrete chemicals

- 52 List 1 chemicals

- 107 List 2 chemicals

• ToxCast Universe = 

~1800 chemicals

- 1444 in EDSP Universe

- 49 List 1 chemicals

- 56 List 2 chemicals

• in vitro reference

chemicals = 40

- 26 in EDSP universe

• Peer-reviewed “GL-like” 

uterotrophic studies = 144

- 98 in ToxCast Universe

- 6 List 1 Chemicals

- 5 List 2 Chemicals

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 12 of 17
ER Agonist Bioactivity

Figure 2.16

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 13 of 17
ER Agonist Bioactivity

Figure 2.16

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 14 of 17
Appendix 3: Supplementary Data

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 15 of 17
Evaluating ER Model Performance 

 New method must be fit for purpose and ideally, perform as well 

or better than existing method

• Possibility of greater sensitivity in HTS due to redundancy of assays

– 18 assays detect multiple points in ER signaling pathway using multiple 

technologies

• ToxCast network model performance evaluated with a relatively large 

set (45) of reference chemicals 

– T1 ER binding: 23 (but 35% were not were not consistent with expected 

outcome)

– T1 ERTA: 12

– T1 UT: 7

• Caveat of using uterotrophic results as “true”

– 26% of chemicals with >1 guideline-like study had contradictory uterotrophic 

results (RADAR plots)

• Caveat of using Tier 1 ER endpoints as “true”

– Tier 1 designed as a battery, and current analysis takes ER endpoints out of the 

larger Tier 1 context

– in vitro Tier 1 ER interactions determined from 1 ER binding and 1 ERTA

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 16 of 17
Revisiting Objectives: ER Pathway

 Before using ER model in EDSP framework, EPA is 

reviewing the scientific strengths and limitation for 

each goal outlined in the white paper

1. Prioritize chemicals for further EDSP screening and 

testing based on estimated bioactivity

– List 1: 0 agonist, 0 antagonist; ~73% no bioactivity 

– List 2: 0 agonist, 0 antagonist; ~86% no bioactivity 

– Universe: 4.4% agonist, 0.6% antagonist; ~80% no bioactivity 

2. Contribute to the weight of evidence evaluation of a 

chemical’s potential bioactivity

3. Substitute for specific endpoints in the EDSP Tier 1 

battery

– Tier 1 ER binding

– Tier 1 ERTA

– Tier 1 Uterotrophic assay

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2-5, 2014 Slide 17 of 17