# Section 2.1: 

# Computational Tools & Models Richard Judson 

# National Center for Computational Toxicology Office of Research and Development

Slide 1 of 65

# Presentation Roadmap

Introduction Estrogen Receptor Bioactivity Androgen Receptor Bioactivity Exposure

Toxicokinetics/Dosimetry

Integrated Bioactivity Exposure Ranking

Future Directions FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 2 of 65

# In Vitro Estrogen Receptor Model Combines results from multiple ToxCast in vitro assays  Use multiple assays per pathway • Different technologies • Different points in pathway
 No assay is perfect • Assay Interference 

# • Noise
 Use model to integrate assays


#  Evaluate model against reference chemicals
 Methodology being applied to other pathways
FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 3 of 65

# Appendix Overview
 Appendix 1: ER AUC plots for reference chemicals • Shows plots of ER reference chemicals, assay and model data 

#  Appendix 2: ER AUC plots for all chemicals • Shows plots for all 1812 chemicals, assay and model data 

#  Appendix 3: Supplementary Data: ER, AR, and IBER • IBER data for all chemicals with ER and AR model data, HTTK data and exposure estimates

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 4 of 65

# Major theme: A ll Assays Have False Positives & Negatives Much of this “noise” is reproducible

- “assay interference”

- Result of interaction of chemical 

### with complex biology in the assay EDSP chemical universe is structurally diverse

- Solvents

- Surfactants

- Intentionally cytotoxic compounds

- Metals

- Inorganics

- Pesticides



### - Drugs Assays cluster by technology, suggesting technology- specific non - ER

bioactivity

Section 2.1

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 5 of 65

### Section 2.1, Figure 2.1, p. 21 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 6 of 65 

# ER Model

# Radioligand Biochemical Assays (Novascreen) Assay ID

Assay Name

Gene Target Organism

Tissue

A1 NVS_NR_bER ESR1 bovine uterus A2 NVS_NR_hER ESR1 human NA A3 NVS_NR_mERa Esr1 mouse NA 

### Common Features:

- Receptor binding / competitive binding

- Percent of positive control



### - Radioligand binding

- Scintillation counting readout

- 18 hour incubation

- Cell -free biochemical assay

- 96 -well format Section 2.1, Table 2.1, p.16

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 7 of 65

# Protein Complementation Assays (Odyssey Thera) Assay ID Assay Name

Gene Target

Assay Design Type

Biological Process Target

Detection Technology

Timepoint (Hours) Tissue

Cell Line Name

A4 OT_ER_ERaERa_0480 ESR1 protein fragment complementation assay protein stabilization Protein -fragment Complementation 8 kidney HEK293T A5 OT_ER_ERaERa_1440 ESR1 protein fragment complementation assay protein stabilization Protein -fragment Complementation 24 kidney HEK293T A6 OT_ER_ERaERb_0480 ESR1 protein fragment complementation assay protein stabilization Protein -fragment Complementation 8 kidney HEK293T A7 OT_ER_ERaERb_1440 ESR1 ESR2 protein fragment complementation assay protein stabilization Protein -fragment Complementation 24 kidney HEK293T A8 OT_ER_ERbERb_0480 ESR2 protein fragment complementation assay protein stabilization Protein -fragment Complementation 8 kidney HEK293T A9 OT_ER_ERbERb_1440 ESR2 protein fragment complementation assay protein stabilization Protein -fragment Complementation 24 kidney HEK293T A10 OT_ERa_EREGFP_0120 ESR1 fluorescent protein induction regulation of gene expression Microscopy 2 cervix HeLa A11 OT_ERa_EREGFP_0480 ESR1 fluorescent protein induction regulation of gene expression Microscopy 8 cervix HeLa

Common Features:

- Percent of positive control

- Fluorescence readout

- Scintillation counting readout

- Cell - based, immortalized cell line

- Human

- 384-well format 

### Section 2.1, Table 2.1, p.16

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 8 of 65

# RNA Transcription Activation Assays (Attagene) Assay ID Assay Name

Gene Target

A12 ATG_ERa_TRANS_up ESR1 A13 ATG_ERE_CIS_up ESR1 Common Features:

- Log2 fold induction (scaled x25)

- mRNA induction

- Sequencing readout, RT- PCR

- Capillary electrophoresis

- HepG2 liver immortalized cell line

- Human

- 24 hour incubation

- 24-well format 

### Section 2.1, Table 2.1, p.16

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 9 of 65

# Protein Transcription Activation Assays (Tox21) Assay ID Assay Name

Gene Target

Assay Design Type

Detection Technology

Detection Technology Subtype

Measurement Timepoint (Hours) Tissue

Cell Line Name

A14 Tox21_ERa_BLA_Agonist_ratio ESR1 beta lactamase induction GAL4 b - lactamase

reporter gene Fluorescence

intensity 24 kidney HEK293T A15 Tox21_ERa_LUC_BG1_Agonist ESR1 luciferase induction Luciferase - coupled

ATP quantitation Bioluminescence 48 ovary BG1 A17 Tox21_ERa_BLA_Antagonist_ratio ESR1 beta lactamase induction GAL4 b - lactamase

reporter gene Fluorescence

intensity 24 kidney HEK293T A18 Tox21_ERa_LUC_BG1_Antagonist ESR1 luciferase induction Luciferase - coupled

ATP quantitation Bioluminescence 48 ovary BG1

Common Features:

- Percent of positive control

- Immortalized cell line

- Human

- 1536 -well format

- Measure gene express8ion 

### Section 2.1, Table 2.1, p.16

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 10 of 65

# Proliferation Assay (ACEA) Assay ID

Assay Name

GeneTarget Normalized Data Type

Assay Design Type

Biological Process Target

Detection Technology

Detection Technology Subtype Measurement Timepoint (Hours) Organism Tissue Cell Format Cell Line Name Assay Footprint

A16 ACEA_T47D_80hr_Positive ESR1 percent activity real -time cell -growth kinetics cell

proliferation RT- CES Electrical

Sensor: Impedance 80 human breast cell line T47D 96 -well

plate 

### Section 2.1, Table 2.1, p.16

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 11 of 65

# Assay Bioactivity Patterns Track with Technology Drives need for model to separate ER from off-target bioactivity

Many assays are active
One or a few assays active


### Section 2.1, Figure 2.2, p. 24 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 12 of 65

### Section 2.1, Figure 2.1, p. 21 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 13 of 65 

# ER Model

# ER Computational Model

14 

### Penalty enforces physical assumption 

### that chemical will not hit many targets simultaneously

AUC Summarizes results A i 

### is the efficacy of the assay at a given concentration R j 

### is the “true” efficacy which is unobservable 

### F links receptors to assays

Solve a constrained least- squares problem 

### to minimize difference between the measured and predicted assay values Section 2.1, p. 21

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 14 of 65

# ER AUC Scales Monotonically With AC50 for a Pure Agonist Active in All Assays

AUC=0.1 Equivalent to AC50=100 uM

Section 2.1 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 15 of 65

# Example 1 – BPA – True Agonist

ER AUC=0.65

Binding assays active at lowest concentration 

### AUC “sign” feature will discount this Blue: agonist “receptor” Assays Receptors

Cytotoxicity Region: red line is median cytotox AC50 Section 2.1 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 16 of 65

# Example ER Curves

True Agonist True Antagonist

Negative - Broad Assay Interference Negative - Narrow Assay Interference Section 2.1, Figure 2.3, p. 26

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 17 of 65

# Selection of ER Reference Chemicals  Start with list from OECD TG 455
• “Test No. 455: Performance - Based Test Guideline for Stably Transfected Transactivation In Vitro Assays to Detect Estrogen Receptor Agonists” • http://www.oecd - ilibrary.org/environment/test - no -455 - performance based -test-guideline -for- stably-transfected -transactivation - in -vitro 

## assays -to - detect- estrogen - receptor- agonists_9789264185388 - en 

#  Primarily taken from literature surveys, input from NICEATM/ICCVAM, EPA  Criteria for selection was mainly consistency across laboratories in hit- call and relative potency  During cross- laboratory validation of TG 455 assay, there was not 100% consistency in calls 

#  Heavily weighted towards agonist positives Section 2.1, p. 18 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 18 of 65

# ER Reference Chemical List CASRN Chemical Name Agonist Potency Antagonist Potency 57 - 63 - 6 17alpha - Ethinyl estradiol Strong Inactive 50- 28- 2 17beta - Estradiol Strong NA 56- 53 - 1 Diethylstilbestrol (DES) Strong Inactive 84- 16- 2 meso - Hexestrol Strong NA 57 -91 -0 17alpha - Estradiol Moderate NA 140- 66-9 4-tert- Octylphenol Moderate NA 53 - 16- 7 Estrone Moderate NA 599- 64-4 4- Cumylphenol Weak NA 521 - 18- 6 5alpha - Dihydrotestosterone Weak Inactive 80-05 - 7 Bisphenol A Weak Inactive 77 -40- 7 Bisphenol B Weak NA 486- 66-8 Daidzein Weak NA 446- 72-0 Genistein Weak Inactive 143 - 50-0 Kepone Weak Inactive 789-02- 6 o,p' - DDT Weak NA 58- 18-4 17alpha - Methyltestosterone Very Weak NA 520- 36- 5 Apigenin Very Weak Inactive 85 - 68- 7 Butylbenzyl phthalate Very Weak Inactive 480-40-0 Chrysin Very Weak Inactive 115 - 32- 2 Dicofol Very Weak Inactive 117 -81 - 7 Diethylhexyl phthalate Very Weak Inactive 84- 74- 2 Di - n - butyl phthalate Very Weak Inactive 120-47 -8 Ethylparaben Very Weak NA 60168-88-9 Fenarimol Very Weak NA 520- 18- 3 Kaempferol Very Weak NA 72-43 - 5 Methoxychlor Very Weak NA 72- 55 -9 p,p’ - DDE Very Weak Inactive 104-40- 5 p - n - Nonylphenol Very Weak NA 1912- 24-9 Atrazine Inactive NA 50- 22- 6 Corticosterone Inactive NA 66-81 -9 Cycloheximide Inactive NA 13311 -84- 7 Flutamide Inactive NA 52-86-8 Haloperidol Inactive NA 52806- 53-8 Hydroxyflutamide Inactive NA 65277 -42- 1 Ketoconazole Inactive NA 330- 55 - 2 Linuron Inactive NA 57 - 30- 7 Phenobarbital Sodium Inactive NA 32809- 16-8 Procymidone Inactive NA 50- 55 - 5 Reserpine Inactive NA 52-01 - 7 Spironolactone Inactive NA 68392- 35-8 4- Hydroxytamoxifen (E/Z) NA Active 82640-04-8 Raloxifene NA Active 10540- 29- 1 Tamoxifen NA Active 54965 - 24- 1 Tamoxifen citrate NA Active 57 -83 -0 Progesterone NA Inactive 

## Note that many but not all chemicals are used for both agonist and antagonist references

Section 2.1, Table 2.2, p. 19

Slide 19 of 65

# ER Reference Chemical Performance

Section 2.1, Figure 2.4, p. 29

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 20 of 65

# Using the ER model
 Goal: Eliminate false positives, but not true / weak ones  Data contains “true” ER hits and various types of noise and assay interference • Noise: results could be different if we reran the assay tomorrow • Interference: results are reproducible, but do not arise from binding to the ER 

#  Factors to use in distinguishing ER from non - ER bioactivity • AUC • Number of technologies active (0,1,...) • Efficacy (Top/Emax) (Is the response>>noise threshold?) • Potency (Is the response at a low concentration?) • Z- score (?s the response well below cytotoxicity threshold?) • Chemical structure / class • Physico - chemical properties Section 2.1 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 21 of 65

# ER Summary for 1812 Chemicals

AUC Range 0 . 1 to 1 0 . 01 to 0 . 1 Agonist 97 196 Antagonist 21 109 R3 10 30 R6 96 199 R8/A 16 33 35 R9 54 56 A2 13 41 A3 23 24 A9 10 43 A 10 13 31 A 12 37 216 A 13 66 171 A 15 71 93 A 17 64 95 A 18 28 61 

### Higher intensity color indicates greater activity (AUC<0.1 set to zero) Section 2.1, Figure 2.5, p. 31
FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 22 of 65

# Most chemicals display a “burst” of potentially non selective bioactivity near cytotoxity concentration

1000 100 10 1 0.1 AC50 ( m M ) Number of Hits Burst Region Cytotoxicity Range 3 MAD Tested Concentration Range

21 18 15 12 9 6 3 0 - 3 - 6 - 9 Z Number of Hits Burst

Region Cytotoxicity Range 3 MAD 

### Concentration Space Z- Space

Bioactivity inferred 

### Section 2.1 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 23 of 65

# Example of Burst Bioactivity by Chemical

Cytotoxicity assays Section 2.1 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 24 of 65

# Burst Is Seen Across Technologies Cell line (HepG2)

Human Primary Cells Cell -free Biochemical

Bimodal Section 2.1
FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 25 of 65

# Schematic Explanation of the Burst

Oxidative Stress DNA Reactivity Protein Reactivity Mitochondrial stress ER stress Cell membrane disruption Specific apoptosis

... 

### Specific Non - specific

Section 2.1

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 26 of 65

# ER Example – 17beta Estradiol Section 2.1

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 27 of 65

# ER Example – 5alpha - Dihydrotestosterone

Section 2.1

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 28 of 65

# ER Example – Raloxifene

Section 2.1

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 29 of 65

# ER Example – Bisphenol A

Section 2.1

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 30 of 65

# ER Example – Branched Nonylphenol

Section 2.1

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 31 of 65

# ER Example – Linear Nonylphenol

Section 2.1

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 32 of 65

### Section 2.1

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 33 of 65



# ER Example – Kepone

### Section 2.1 

# ER Example – Butyl Benzyl Phthalate

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 34 of 65

### Section 2.1



# ER Example – DEHP

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 35 of 65

### Section 2.1



# ER Example – Tributyltin

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 36 of 65

### Section 2.1



# ER Example – Cycloheximide

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 37 of 65

### Section 2.1 

# ER Example – Dimethyl Hydrogen Phosphite

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 38 of 65

### Section 2.1



# ER Example – alpha - Cyclodextrin

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 39 of 65

# ER Antagonists

CASRN Name Use category AUC (Antagonist) 82640- 04- 8 Raloxifene hydrochloride Pharmaceutical 0.973 68392 - 35 - 8 4- Hydroxytamoxifen Pharmaceutical 0.961 129453 - 61 - 8 Fulvestrant Pharmaceutical 0.911 50-41 - 9 Clomiphene citrate Pharmaceutical 0.817 54965 - 24- 1 Tamoxifen citrate Pharmaceutical 0.776 10540- 29 - 1 Tamoxifen Pharmaceutical 0.695 84371 - 65 - 3 Mifepristone Pharmaceutical 0.456 13463 -41 - 7 Zinc pyrithione Anti -fungal/anti - bacterial/drug 0.335 26027 - 38 - 3 Polyoxyethylene(10)nonylphenyl ether Preservative 0.197 683 - 18 - 1 Dibutyltin dichloride Microbicide 0.179 989 - 38 - 8 Rhodamine 6G Dye 0.15 4342 - 36 - 3 Tributyltin benzoate Microbicide 0.146 2971 - 36 - 0 2,2 - Bis(4- hydroxyphenyl) - 1,1,1 -trichloroethane (HPTE) Degradate of Methoxychlor 0.143 56 - 95 - 1 Chlorhexidine diacetate Biocide 0.134 90357 - 06 - 5 Bicalutamide Pharmaceutical 0.129 66 - 81 - 9 Cycloheximide Fungicide 0.121 62 - 38 -4 Phenylmercuric acetate Fungicide 0.115 9004- 82 -4 Sodium lauryl polyoxyethylene ether sulfate Surfactant 0.107 25155 - 18 -4 Methylbenzethonium chloride Microbiocide 0.105 1034- 01 - 1 Octyl gallate Antioxidant 0.104 57 - 09 - 0 Hexadecyltrimethylammonium bromide Algaecide/antiseptic/anti - static agent 0.104



# Antagonism is often confounded by cytotoxicity These need to be examined further Section 2.1

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 40 of 65

# Pseudo- Receptor Diagnostics  Model separates chemicals into ER active, ER inactive, 

# and pseudo - receptor active (individual assays/technologies)  Aim: diagnose cause of pseudo - receptor activity 

#  Diagnostics: • Non - normal distribution of baseline noise / T • Cell - stress / cytotoxicity / Z 

# • Crosstalk / cross - reactivity • Pan - active compounds (PAINS) • Structure class trends 

# • Physico - chemical property trends 

#  Work in progress Section 2.1 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 41 of 65

# First Drivers of Pseudo- Receptor Activity:

Low T (non - normal noise) Low Z (cytotoxicity- related) PseudoReceptor N |T| |T| (Comparator) p(T) |Z| |Z| (Comparator) p(Z) R3 10 94 86 NS 9 8 . 8 NS R6 96 60 76 2 . 00 E - 09 3 . 8 7 . 3 2 . 10 E - 14 R8 33 101 110 NS 7 . 7 7 . 9 NS R9 54 98 97 NS - 0 . 57 2 . 8 2 . 80 E - 05 A2 13 91 88 NS 7 . 5 9 . 4 NS A3 23 81 80 NS 6 . 7 7 . 9 NS A9 10 47 101 1 . 40 E - 06 6 . 1 5 . 4 NS A 10 13 72 85 NS 6 . 8 6 . 3 NS A 12 37 53 81 2 . 10 E - 09 3 . 3 7 . 3 1 . 50 E - 06 A 13 66 33 71 9 . 10 E - 23 5 . 8 7 . 3 0 . 021 A 15 71 65 116 2 . 80 E - 08 4 . 5 6 . 2 0 . 022 A 17 64 75 86 0 . 037 1 . 4 3 . 6 0 . 019 A 18 28 95 101 NS 2 . 6 1 . 9 NS 

### Section 2.1, Table 2.3, p. 32 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 42 of 65

# T-distributions for T-significant Assays

Section 2.1 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 43 of 65

# Z-distributions for Z-significant Assays

Section 2.1 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 44 of 65

# Crosstalk / Cross- Reactivity
 Crosstalk – Bioactivity in one target of a multiplexed assay causes other readouts to fire • ATG, ACEA 

#  Cross - reactivity – A compound potent against one receptor is weakly potent against another • True active, but some other process is probably more important 

#  Can be difficult to distinguish Section 2.1, p. 34 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 45 of 65

# Quantify Cross- Reactivity
 Are pseudo - receptor actives more likely to also be active in many assay readouts than are ER - inactives?  Test for broad bioactivity in ATG CIS, ATG TRANS, NVS NR, Tox21 BLA agonist, Tox21 LUC 

# agonist, Tox21 BLA antagonist, Tox21 LUC antagonist Section 2.1, p. 34
FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 46 of 65

# Cross- Reactivity Results

Non - ER Assay Set Expected Pseudo Receptors Correlated

Pseudo- Receptor Reference

Mean PseudoReceptor mean p-value Attagene CIS A 13 , R 6 A 12 3 . 8 15 0 . 0023 Attagene CIS A 13 , R 6 A 18 3 . 8 18 0 . 0013 Attagene TRANS A 12 , R 6 A 12 1 . 5 6 . 7 0 . 00015 Attagene TRANS A 12 , R 6 A 13 1 . 5 3 . 4 0 . 00069 Attagene TRANS A 12 , R 6 A 18 1 . 5 5 . 9 0 . 0066 Attagene TRANS A 12 , R 6 R 6 1 . 5 2 . 7 0 . 00019 Novascreen NR A 1 , A 2 , A3 A 2 0 . 64 2 . 1 0 . 01 Novascreen NR A 1 , A 2 , A3 Agonist 0 . 64 1 . 8 0 . 00024 Novascreen NR A 1 , A 2 , A3 Antagonist 0 . 64 3 . 3 0 . 0015 Tox21 BLA Antagonist A 17, R 9 A 17 0 . 26 2 . 2 1 . 20 E - 07 Tox21 LUC Agonist A 15 , R 7 A 12 0 . 54 1 . 1 0 . 0078 Tox21 LUC Antagonist A 18, R 9 A 17 0 . 14 1 . 1 4 . 30 E - 06 Tox21 LUC Antagonist A 18, R 9 A 18 0 . 14 1 . 7 1 . 70 E - 05 Tox21 LUC Antagonist A 18, R 9 Agonist 0 . 14 0 . 41 0 . 0035 

### Example : Chemicals active in only A12 (ATG_ERa_TRANS) and no other ER assays have a mean of 15 hits in the ATG CIS battery, vs. an mean of 3.8 in chemicals inactive in all ER assays Interpretation : ER is just a bystander in some process causing all of these assay readouts to fire Section 2.1, Table 2.4, p. 35

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 47 of 65

# PAINs Pan -Assay Interference Compounds  Some compounds are observed to activate many assays 

#  Known problem in pharmaceutical development  Related to cell - stress issues?  Public structure - classification model is available  Test whether pseudo - receptor actives are more likely to be active in many ToxCast assays vs. 

# those inactive in all ER assays Section 2.1, p. 35 Baell and Holloway, J. Med. Chem. , 2010, 53 (7), pp 2719–2740 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 48 of 65

# PAINs Results

Receptor HI - Z reference

mean HI -Z pseudo receptor mean P (HI -Z) LO-Z reference mean LO-Z pseudo receptor mean P ( LO-Z) A10 0.026 0.0017 NS 0.018 0.025 NS A12 0.026 0.0039 NS 0.018 0.087 0.0013 A13 0.026 0.054 NS 0.018 0.026 0.0052 A15 0.026 0.001 NS 0.018 0.031 0.0043 A17 0.026 0.0048 NS 0.018 0.079 2.80E - 05 A18 0.026 0.045 NS 0.018 0.054 1.80E - 05 A2 0.026 0.0019 NS 0.018 0.046 0.032 A3 0.026 0.016 NS 0.018 0.051 0.0012 Agonist 0.026 0.072 1.80E - 08 0.018 0.076 3.80E - 25 Antagonist 0.026 0.18 6.10E - 05 0.018 0.055 4.50E - 05 R3 0.026 0.00067 NS 0.018 0.055 0.0041 R6 0.026 0.0047 NS 0.018 0.03 1.80E - 06 R8 0.026 0.0081 NS 0.018 0.029 0.014 

### Example : Chemicals that are model - agonists are active in an average of 7.2% of all non - ER ToxCast assays, while chemicals that are active in no ER assays are active in an average of 2.6% of all non - ER ToxCast assays, if one excludes all bioactivity for Z<3

Section 2.1, p. 35 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 49 of 65

# Chemical Structure Class  Are particular structure classes more likely to be 

# associated with pseudo - receptor activity? • All ToxCast chemicals are placed into structure classes based on hierarchical clustering 

#  Only significant findings: model - agonist and model 

# antagonist chemicals  ER agonist- associated classes: • genistein - like , paraben, alkyl - phenol, steroids targeting the 

# estrogen receptor and steroids targeting the androgen receptor 

#  ER antagonist- associated class: • Tamoxifen analogs Section 2.1, p. 36 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 50 of 65

# Physico-Chemical Properties  Are chemicals in particular ranges of physicochemical properties more likely to be pseudoreceptor active? 

#  Weak but statistically significant trends are seen, but need further refinement to be useful • MW, surface area, volume, pKa , polarizability, logP, 

# logS, logD • Overlap between property values for model actives (including reference chemicals) and 

# pseudo - receptor chemicals is significant Section 2.1, p. 36 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 51 of 65

#  Tox21, ~8000 chemicals in 4 assays;  FDA EDKB database of ~8000 chemicals from the literature;  METI database, ~2000 chemicals;  ChEMBL database, ~2000 chemicals. 

#  Training chemicals for specific QSAR models 

# Experimental / Literature Data for Comparison with ER In Vitro Model and QSAR Models EPA/NCCT, UMEA/Chem, FDA/NCTR, NIH/NCATS, Helmholtz/ISB



## 60,000 entries for ~15,000 chemicals

Section 2.1 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 52 of 65

# Comparison with ER Literature Data Chemical Set Receptor

N (Binding Positive)

N (Binding Negative)

N (Agonist Positive) N (Agonist Negative) N (Antagonist Positive) N (Antagonist Negative)



### Initial Agonist 55 0 25 0 1 0 Initial Antagonist 5 0 0 0 3 0 Initial None 11 3 2 0 2 0 Initial R6 4 1 3 0 0 0 Final Agonist 41 0 19 0 1 0 

### Final Antagonist 5 0 0 0 3 0 Use training data with at least 4 concordant calls Remove all Tox21 data before this comparison Some literature positive appear to be consistent false - positives

- Maneb and Mancozeb

Section 2.1, Table 2.5, p. 38 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 53 of 65

# Introduction to the EDSP21 Dashboard  Goal: To make ER and AR data easily available to all stakeholders • Assay- by- assays concentration - response plots 

# • Model scores – AUC agonist and antagonist • ER QSAR calls • Other relevant data 

#  First version is “beta” – intended to elicit comments, requests for further additions 

#  http://actor.epa.gov/edsp21

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 54 of 65

# http://actor.epa.gov/edsp21

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 55 of 65

# ToxCast + QSAR: CERAPP  Collaborative Estrogen Receptor Activity Prediction Project (CERAPP)  Goals: • Use ToxCast ER score (or other data) to build many QSAR models • Use consensus of models to prioritize chemicals for further testing 

#  Assumptions • ToxCast chemicals cover enough of chemical space to be a good “global” training set 

## • Consensus of many models will be better than any one individually 

#  Process • Curate chemical structures 

## • Curate literature data set • Build many models • Build consensus model • Evaluate models and consensus ER QSAR/ CERAPP, p. 38

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 56 of 65 DTU/food: Technical University of Denmark/ National Food Institute  EPA/NCCT: U.S. Environmental Protection Agency / National Center for Computational Toxicology  FDA/NCTR/DBB: U.S. Food and Drug Administration/ National Center for Toxicological Research/Division of Bioinformatics and Biostatistics  FDA/NCTR/DSB: U.S. Food and Drug Administration/ National Center for Toxicological Research/Division of Systems Biology  Helmholtz/ISB: Helmholtz Zentrum Muenchen /Institute of Structural Biology  ILS&EPA/NCCT: ILS Inc & EPA/NCCT  IRCSS: Istituto di Ricerche Farmacologiche “Mario Negri”  JRC_Ispra : Joint Research Centre of the European Commission, Ispra .  LockheedMartin&EPA : Lockheed Martin IS&GS/ High Performance Computing  NIH/NCATS: National Institutes of Health/ National Center for Advancing Translational Sciences  NIH/NCI: National Institutes of Health/ National Cancer Institute  RIFM: Research Institute for Fragrance Materials, Inc  UMEA/Chemistry: University of UMEA/ Chemistry department  UNC/MML: University of North Carolina/ Laboratory for Molecular Modeling  UniBA/ Pharma : University of Bari/ Department of Pharmacy  UNIMIB/ Michem : University of Milano - Bicocca / Milano Chemometrics and QSAR Research Group  UNISTRA/ Infochim : University of Strasbourg/ ChemoInformatique 

# CERAPP Participants:

ER QSAR/ CERAPP, p. 38 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 57 of 65INITIAL LIST OF CHEMICALS File parsing & first check Check for mixtures Check for salts & counter ions Normalization of specific chemotypes Manual inspection Removal of duplicates Treatment of tautomeric forms

CURATED DATASET Check for inorganics 

# Scheme of the Structure Curation Workflow Univ. North Carolina, Danish Technical University, EPA NCCT Implement in open -source KNIME Workflow

Fourches, Muratov, Tropsha. J Chem Inf Model, 2010, 29, 476 – 488 

### ER QSAR/ CERAPP, p. 38

FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 58 of 65

# Chemicals for Prediction: The Human Exposure Universe  EDSP Universe (10K)  Chemicals with known use (40K ) ( CPCat & ACToR ) 

#  Canadian Domestic Substances List (DSL) (23K)  EPA DSSTox – structures of EPA/FDA interest (15K)  ToxCast and Tox21 (In vitro ER data) (8K ) ~55k to ~32K unique set of structures
 Training set (ToxCast): 1,677 chemical structures  Prediction Set: 32,464 chemical structures ER QSAR/ CERAPP, p. 39 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 59 of 65

# ER QSAR Evaluation & Consensus

 Qualitative: • Binding: 22 models • Agonist: 11 models • Antagonist: 9 models 

#  Quantitative: • Binding: 3 models • Agonist: 3 models 

## • Antagonist: 2 models 

# Models received: Evaluation procedure:

 On the EPA training set (1677)  On the full evaluation set (~7k)  Evaluation set with multi - sources  Remove “Very Weak”  Remove single source  Remove chemicals outside the AD

Score functions & weights for consensus predictions 

### ER QSAR/ CERAPP, p. 39 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 60 of 65

# Concordance of 22 Qualitative ER Models

757 chemicals have >75% positive concordance

Active

Inactive

Prioritization Most models predict most chemicals as inactive

ER QSAR/ CERAPP, p. 39 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 61 of 65

## Total Database

Binders: 3961 Agonists: 2494 Antagonists: 2793 

# Consensus Evaluation Key point : As greater consistency is required from literature sources, QSAR consensus model performance 

## improves

ER QSAR/ CERAPP, p. 39 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 62 of 65

### • positive concordance < 0.6 => Potency class= Very weak

• 0.6=<positive concordance<0.75 => Potency class= Weak • 0.75=<positive concordance<0.9 => Potency class= Moderate • positive concordance>=0.9 => Potency class= Strong 

# Positive Concordance & Potency Level Key Points : 1. Fraction of ER QSAR models called positive is highly correlated with measured potency 2. If strong consensus is required from literature, consensus ER QSAR model performs well

ER QSAR/ CERAPP, Figure 2.8, p. 40 FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 63 of 65

# ER QSAR Summary
 Many ER QSAR and docking models built using ER 

# model result (AUC) as training data  Consensus of models gives high balanced accuracy for literature data that is internally consistent 

#  Open - source structure preparation process performed on all EDPS universe distinct chemicals (and others)  5 - 10% of chemical universe has predicted potential for 

# ER bioactivity  QSAR- model positives are candidates for follow- up in vitro testing ER QSAR/ CERAPP
FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 64 of 65

# ER Model Conclusions  1812 Chemicals run in 18 ToxCast ER assays  Model developed to quantify agonist or antagonist bioactivity  Assays and model perform well against reference 

# chemicals  A small number of chemicals have consistent ER bioactivity • 97 agonist • 21 antagonists • Most of these are in well - studied chemical classes 

# (phenols, steroids) Section 2.1
FIFRA SAP: Integrated Bioactivity Exposure Ranking Dec 2 - 5, 2014 Slide 65 of 65