Integrated Bioactivity and Exposure Ranking

Integrated Bioactivity and Exposure Ranking:

A Computational Approach for the 

Prioritization and Screening of Chemicals in 

the Endocrine Disruptor Screening Program

U.S. Environmental Protection Agency

Endocrine Disruptor Screening Program

Jointly developed by:

U.S. EPA Office of Chemical Safety and Pollution Prevention (OCSPP)

U.S. EPA Office of Research and Development (ORD)

U.S. EPA Office of Water (OW)

NIH National Toxicology Program Interagency Center for the Evaluation of 

Alternative Toxicological Methods (NICEATM)

FIFRA SAP December 2-5, 2014
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 2 of 111

Preface:

The U.S. Environmental Protection Agency (EPA) is continuing a series of scientific peer 

reviews focused on computational approaches for prioritization and screening of chemicals in 

the Endocrine Disruptor Screening Program (EDSP). This document describes innovative 

approaches for determining estrogen and androgen bioactivity based on a series of 

computational models integrating data from ToxCast high throughput screening (HTS) assays. 

The underlying science for each method is described, and EPA presents the scientific 

confidence in each method using computational models of estrogen and androgen bioactivity 

for:

1. Prioritizing chemicals for further screening and testing;

2. The utility of using HTS to contribute to weight of evidence based on Tier 1 battery data 

and other scientifically relevant information on a chemical’s potential endocrine 

bioactivity; and

3. The scientific support for the potential substitution of these HTS assays for some of the 

lower throughput assays that are currently part of EDSP screening.

This document also describes new science for an integrated bioactivity and exposure ranking 

(IBER) approach to rank chemicals and prioritize further EDSP consideration. The IBER 

approach is demonstrated using estrogen and androgen receptor pathway, with mention of 

parallel efforts underway to develop similar models for the androgen receptor pathway.

This document describes a series of generalizable chemical prioritization and screening 

methodologies based on endocrine bioactivity and exposure which could be applied and 

adapted to address other endocrine pathways and EDSP chemicals. The focus for review by 

the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) Scientific Advisory Panel 

(SAP) is on the concepts and methodologies, and not on results for specific chemicals that 

may be prioritized for EDSP screening. It is the intention of the EPA to complete the 

prioritization methodology and process after receiving recommendations from the SAP on the 

proposed computational approaches. None of the information on any individual chemical 

substance in this paper, EPA presentations at the SAP meeting, or the meeting summary are 

intended to support any conclusions other than to aid in the development of potential 

prioritization and screening methods. None of the examples presented should be taken as final 

decisions regarding potential bioactivity, exposure, hazards or risks of the substance, or that 

the EPA has or will make a determination that any use of the substance necessarily will pose a 

risk. Further, the results described herein are not designed to provide a scientific basis, by 

themselves, to allow the EPA to conclude that these substances have potential for endocrine 

effects, or are "endocrine disruptors". 

This document has been reviewed in accordance with EPA policy. Mention of trade names or 

commercial products do not constitute endorsement.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 3 of 111

Contents

Preface: ..................................................................................................................................... 2

List of Acronyms and Abbreviations .......................................................................................... 5

Section 1: Introduction............................................................................................................... 7

Section 2: Estrogen Pathway................................................................................................... 13

2.1: Computational Tools and Models .................................................................................. 13

2.1.1: In vitro data and model: ER..................................................................................... 13

2.2: Interpretation and application: ER ................................................................................. 40

2.2.1: Curated literature review of uterotrophic literature .................................................. 41

2.2.2: Comparison of ToxCast ER agonist AUC data and Uterotrophic studies................ 47

2.2.3: Comparison of ToxCast data and Tier 1 ER assays for List 1 chemicals................ 56

2.2.4: Demonstration of computational toxicological approach to evaluating ER agonist 

bioactivity of EDSP universe chemicals ............................................................................ 58

Section 3: Androgen Pathway ................................................................................................. 61

3.1: Computational Tools and Models: AR ........................................................................... 61

3.1.1: In vitro data and model: AR..................................................................................... 61

3.2: Interpretation and application: AR ................................................................................ 71

3.2.1: Curated literature review of Hershberger literature ................................................ 71

3.2.2: Comparison of ToxCast data and Hershberger studies ......................................... 73

3.2.3: Tier 1 AR Data ....................................................................................................... 73

3.2.4: Demonstration of computational toxicological approach to evaluating AR antagonist 

bioactivity of EDSP universe chemicals ............................................................................ 74

Section 4: High-Throughput Exposure (HTE) ......................................................................... 77

4.1: Programmatic Objectives and Challenges for High Throughput Exposure Estimation . 77

4.2: High Throughput Exposure Modeling ........................................................................... 77

Section 5: Toxicokinetics / Dosimetry ..................................................................................... 87

Section 6: Integrated Bioactivity Exposure Ranking (IBER) for Prioritization.......................... 92

6.1: Basic Approach for IBER............................................................................................... 92

6.2: Modeling uncertainty and variability .............................................................................. 93

6.3: Interpretation and application: IBER.............................................................................. 94

Section 7: Future directions ..................................................................................................... 99

7.1: Computational tools, models and data .......................................................................... 99

7.2: Data Release and Transparency: The EDSP21 Dashboard. 129................................ 100

7.3: EDSP Future Directions .............................................................................................. 101

References ............................................................................................................................ 105

List of Appendices ................................................................................................................. 111
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 4 of 111

Appendix 1: ER AUC plots for reference chemicals ........................................................... 111

Appendix 2: ER AUC plots for all chemicals....................................................................... 111

Appendix 3: Supplementary Data: ER, AR, and IBER........................................................ 111

Appendix 4: NICEATM_UT_GL_DB_All_Data.xls .............................................................. 111

Appendix 5: Uterotrophic study radar plots ........................................................................ 111

Appendix 6: Examination of ER in vitro-in vivo discordance............................................... 111

Appendix 7: AR AUC plots for reference chemicals ........................................................... 111

Appendix 8: AR AUC plots for all chemicals....................................................................... 111
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 5 of 111

List of Acronyms and Abbreviations

ACToR: EPA’s Aggregated Computational Toxicology 

Repository

AIC: Akaike Information Criterion

AOP: Adverse Outcome Pathway

AR: Androgen receptor

AUC: Area Under the Curve

BASS: EPA’s Bioaccumulation and Aquatic System 

Simulator

BCF: BioConcentration Factor

BW: Body weight

CAS-RN: Chemical Abstracts Service Registry Number 

(chemical identification number)

CCL 3: the third Contaminant Candidate List

CDC: the United States Centers for Disease Control

CDR: EPA Chemical Data Reporting (formerly IUR)

CPCat: Chemical-Product Categorization database

CPCPdb: Consumer Product Chemical Profiles database

CPRI: Crop Protection Research Institute

Css: plasma Concentration after Steady-State is reached 

due to repeated exposures

CTCP: Clinical Toxicology of Commercial Products

DSSTox: Distributed Structure-Searchable Toxicity 

Database

EBC: Expected Biological Concentrations 

EDSP: the Endocrine Disrupter Screening Program

EDSTAC: Endocrine Disruptors Screening and Testing 

Advisory Committee

EEC: Expected Environmental Concentration

E-FAST2: Exposure and Fate Assessment Screening Tool 

v2.0

EPA: the United States Environmental Protection Agency

EPI Suite: EPA’s Estimation Program Interface Suite

ER: Estrogen receptor

ExpoCast: EPA’s Exposure foreCast prioritization 

research program

EXAMS: EPA’s Exposure Analysis Modeling System

FDA: the United States Food and Drug Administration

FFDCA: Federal Food, Drug, and Cosmetic Act

FIFRA: Federal Insecticide, Fungicide, and Rodenticide 

Act

fup: fraction of chemical that is unbound in plasma when in 

the presence of endogenous levels of plasma 

protein

HEDS: EPA’s Human Exposure Data System

HPDB: Household Product Database

HPV: High Production Volume

HT: High-throughput

HTE: High-throughput exposure

HTS: High-throughput screening

HTTK: High Throughput Toxicokinetics

HUC: Hydrologic Unit Code 

IBER: Integrated (RTK) Bioactivity Exposure Ranking

IEMS: Integrated Exposure Modeling System 

IUR: EPA’s Inventory Update Reporting and Chemical 

Data Reporting list (now CDR)

IVIVE: in vitro-in vivo extrapolation

KABAM: EPA’s Kow (based) Aquatic BioAccumulation 

Model

KOW: Octanol-Water partition coefficient (also called 

lipophilicity or hydrophobicity)

LOD: Limit of detection

MC: Monte Carlo simulation

MCMC: Markov Chain Monte Carlo simulation (for 

Bayesian analysis)

MSDS: Material Safety Data Sheet

MySQL An open-source relational database management 

system that can be interfaced via a structure 

query language or “SQL

NCGC: the NIH National Chemical Genomics Center

NHANES: National Health and Nutrition Examination 

Survey

NIH: the United States National Institutes of Health

NIOSH: the CDC National Institute for Occupational Safety 

and Health

NLM: the United States National Library of Medicine

NPDWR: National Primary Drinking Water Regulation

NRC: National Research Council

NTP: the NIH National Toxicology Program

OPPT: EPA Office of Pollution Prevention and Toxics

ORD: EPA Office of Research and Development 

OSHA: the United States Occupational Safety and Health 

Administration

PBTK: Physiologically-Based Toxicokinetics

PCA: Principal Components Analysis

PDP: the USDA Pesticide Data Program

PRZM: EPA’s Pesticide Root Zone Model

PWS: Public Water System

QSAR: Quantitative Structure Activity Relationship

RAIDAR: Risk Assessment IDentification, And Ranking 

model
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 6 of 111

RED: Rapid Equilibrium Dialysis

RTK: Reverse Toxicokinetics

SAP: Scientific Advisory Panel

SDWA: Safe Drinking Water Act

SEEM: Systematic Empirical Evaluation of Models 

framework

SMILES: Simplified Molecular Input Line Entry System 

TK: Toxicokinetic

ToxCast: EPA’s Toxicity foreCast prioritization research 

program

Tox21: Toxicology in the 21st Century – the 

NTP/NCGC/EPA/FDA consortium for chemical 

hazard HTS

TSCA: the Toxic Substance Control Act

USDA: the United States Department of Agriculture

USGS: the United States Geological Service

USEtox: United Nations Environment Program and 

Society for Environmental Toxicology and 

Chemistry toxicity model
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 7 of 111

Section 1: Introduction

EPA developed the EDSP in response to the statutory mandate in the Federal Food, Drug, and 

Cosmetic Act (FFDCA) to "develop a screening program...to determine whether certain 

substances may have an effect in humans that is similar to an effect produced by a naturally 

occurring estrogen, or such other endocrine effects as the Administrator may designate." As 

part of the EDSP, the statute also gives EPA the authority to "provide for the testing of any 

other substance that may have an effect that is cumulative to an effect of a pesticide chemical 

if the Administrator determines that a substantial population may be exposed to such a 

substance." Beyond testing and determining endocrine effects, FFDCA also states that: "In the 

case of any substance that is found...to have an endocrine effect...the Administrator shall, as 

appropriate, take action under such statutory authority as is available to the Administrator...to 

ensure the protection of public health." In addition to FFDCA, the Safe Drinking Water Act 

(SDWA) gives EPA authority to provide for testing "of any other substances that may be found 

in sources of drinking water if the Administrator determines that a substantial population may 

be exposed to such substance."

To begin meeting these statutory mandates, in 1996 the EPA chartered a Federal Advisory 

Committee, the Endocrine Disruptor Screening and Testing Advisory Committee (EDSTAC), to 

address endocrine disruption. The EPA largely adopted the EDSTAC recommendations, and 

in an August 1998 Federal Register notice, established the EDSP. In its final report, EDSTAC 

made several key recommendations that EPA’s implementation of its endocrine screening 

program should:

 Address both potential human and ecological effects from chemical exposures

 Examine effects of these chemicals on estrogen, androgen and thyroid hormone-related 

processes

 Include pesticide and non-pesticide chemicals, contaminants, and (after evaluating single 

chemicals) mixtures

 Develop a two-tiered screening and testing strategy, now known as the Endocrine Disruptor 

Screening Program

The application of HTS tools has been an integral component of the EDSP from its beginning. 

In 1998, the EDSTAC recommended that “data resulting from HTPS [HTS] will be combined 

with exposure-related information, and with any other effects-related information that is 

available, for each chemical for the purpose of setting priorities for [Tier 1 screening] T1S.” 

However, in 1999 a joint review by the EPA Science Advisory Board (SAB) and the Federal 

Insecticide, Fungicide, and Rodenticide Act (FIFRA) Scientific Advisory Panel (SAP) concluded 

that “EPA failed to sufficiently demonstrate the utility of their HTPS [HTS] system for the 

purpose of chemical sorting and priority setting of estrogen, androgen and thyroid active 

chemicals.” The SAB/SAP did acknowledge that this was a work in progress and that, in 

general, the HTS approach had merit but required further development prior to implementation.

Subsequently, EPA embarked on an alternate course for EDSP priority setting. To identify the 

first chemicals to undergo screening in the EDSP (known as List 1), public comments and the 

recommendations from the joint SAP/SAB were considered. Based on these comments and 

recommendations, 50 and 100 chemicals were selected, drawn from pesticide active 

ingredients and pesticide inert ingredients that are also high production volume chemicals 

(HPV). The number of chemicals (50 to 100) was thought to be a reasonable undertaking to 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 8 of 111

allow the nascent EDSP to gain experience with the test systems before proceeding through 

the larger universe of chemicals. The first list of chemicals was based solely on exposure 

potential. To identify the specific chemicals that would comprise List 1, EPA applied the 

EDSTAC exposure recommendations, focusing on four pathways of exposure to pesticide 

active ingredients:

1. Consumption of food containing pesticide residues (food pathway);

2. Consumption of drinking water containing pesticide residues (water pathway);

3. Residential use of pesticide products (residential use pathway); and/or

4. Occupational contact with pesticide–treated surfaces (occupational exposure pathway).

For HPV pesticide inert ingredients, EPA focused on:

1. Human biological samples (human biological monitoring pathway);

2. Animal tissues that have human food uses (ecological biological pathway; e.g., fish 

tissues);

3. Drinking water (drinking water pathway); and/or

4. Indoor air (indoor air pathway).

Based on these exposure considerations and public review, the EPA issued test orders for a 

list of 67 pesticide chemicals (58 active ingredients and 9 HPV inert ingredients) for EDSP 

screening in 2009, referred to as List 1.

In developing the second list of chemicals for EDSP screening (List 2), EPA focused on 

continuing to address pesticide chemicals and beginning to address drinking water 

contaminants. Pesticide chemicals on List 2 represent pesticides that were scheduled for 

Registration Review during fiscal years 2007 and 2008. EPA identified drinking water 

contaminants for EDSP List 2 from chemicals that were regulated under the national primary 

drinking water regulation (NPDWR) (40 CFR part 141) under the SDWA or were unregulated 

contaminants that are listed on the third Contaminant Candidate List (CCL 3; 

http://water.epa.gov/scitech/drinkingwater/dws/ccl/ccl3.cfm) Based on these considerations, 

the final EDSP List 2 consists of 107 chemicals (41 pesticide active ingredients and 66 SDWA 

chemicals) was published, though test orders have not yet been issued for these chemicals

In November 2012, EPA published the document entitled “Endocrine Disruptor Screening 

Program Universe of Chemicals and General Validation Principles” identifying a universe of 

approximately 10,000 chemicals for EDSP prioritization and screening (US EPA 2012), based 

on the statutory authorities of the FFDCA and SDWA. This historical approach to listing 

chemicals based primarily on exposure considerations has only addressed a small fraction of 

this universe for screening (Figure 1.1). Even for the 174 chemicals on Lists 1 and 2, EPA 

has been careful to note that the public should not presume EPA currently suspects that such 

chemicals interfere with the endocrine systems of humans or other species. 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 9 of 111

Figure 1.1: Of the approximately 10,000 EDSP chemicals, including pesticide chemicals

(covered under the Federal Insecticide, Fungicide, and Rodenticide Act; FIFRA) and chemicals 

found in sources of drinking water (covered under the Safe Drinking Water Act, SDWA), only 

67 List 1 and 107 List 2 chemicals have been identified for screening and testing to date.

Based on the current pace of the Tier 1 screening assays it could take decades to screen all 

10,000 chemicals in the EDSP Universe. Therefore, EPA’s EDSP is actively pursuing the 

application of computational toxicology and exposure methods to create a more efficient and 

robust screening program. Incorporating innovative computational toxicological tools allow the 

agency to integrate bioactivity and exposure to prioritize and screen chemicals, and is 

consistent with the approach for originally recommended by EDSTAC in 1998. These recent 

advances in computational toxicology have brought EPA to an “evolutionary turning point” for 

EDSP prioritization, screening and testing. In addition to rapidly screening thousands of 

chemicals and overcoming throughput limitations of traditional chemical toxicity testing, EPA 

expects computational methods will eventually allow EPA to move forward with replacing some 

EDSP assays that rely on animals with non-animal alternatives.

In order to use high throughput methods to prioritize and screen chemicals, EPA must 

demonstrate that these methods are scientifically sound and can effectively detect endocrine 

bioactivity via receptor-mediated interactions and broader endocrine pathway effects. In FY 

2012, EPA began a multi-year transition for the EDSP to validate and incorporate

computational toxicology methods and high-throughput screens. The current FIFRA SAP 

review is a continuation of a series of reviews focused on the application of these high 

throughput tools for prioritizing and screening the universe of EDSP chemicals. Summaries of

SAP reviews, including the current review and a prospective future review, are listed in Table 

1.1.

EDSP Chemical 

Universe

10,000 chemicals

(FIFRA & SDWA)

EDSP List 2

107 Chemicals

EDSP List 1

67 Chemicals
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 10 of 111

Table 1.1: A summary of the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) 

Scientific Advisory Panel Reviews addressing computational tools for use in the Endocrine 

Disruptor Screening Program (EDSP). ER = estrogen receptor, ERES = ER Expert System, 

QSAR = Quantitative Structure Activity Relationship, ExpoCast = Exposure Forecaster, SEEM 

= Systematic Empirical Evaluation of Models, HTTK = High Throughput ToxicoKinetic, RTK = 

Reverse ToxicoKinetic, IBER = Integrated Bioactivity Exposure Ranking, AR = Androgen 

Receptor, E= Estrogen, A = Androgen, T = Thyroid, CompTox = Computational Toxicology.

FIFRA Scientific Advisory Panel Reviews 

(External Independent Scientific Peer Review)

Topics For Review

1. Scientific Issues Associated with Prioritizing the 

Universe of EDSP Chemicals Using Computational 

Toxicology Tools – January 2013

8 ER Assays

Physical-Chemical Properties

ERES QSAR

2. Scientific Issues Associated with New High Throughput 

Methods to Estimate Chemical Exposure – July 2014

ExpoCast

SEEM Framework

HTTK/RTK

3. Integrated Endocrine Bioactivity and Exposure-Based 

Prioritization and Screening for the Endocrine 

Disruptor Screening Program- December 2014

18 ER Assays + QSAR

9 AR Assays + QSAR

IBER List 2 & Universe (E&A)

Scientific evaluation for partial 

Tier 1 battery substitution

FUTURE: Use of Computational Toxicology for predicting 

endocrine bioactivity in an Adverse Outcome Pathway 

framework 

EAT Pathways

Steroidogenesis

CompTox predictions of 

neuroendocrine-intact in vivo

response

Refined Exposure Models

In January 2013, the SAP reviewed an EPA white paper describing eight HTS estrogen 

receptor (ER) agonist assays and the ER Expert System quantitative structure-activity 

relationship (QSAR) model as potential tools to be used for prioritizing EDSP chemicals. The 

SAP found steps in the prioritization scheme organized and clearly described. Panel members 

encouraged expanding the use of HTS and highlighted the need to consider exposure in the 

process. The SAP also noted that the androgen pathway may be similar to the ER pathway 

but should be focused on antagonism, and endocrine effects in the thyroid pathway involve 

multiple modes of action that are not receptor based.

In July 2014, EPA brought scientific issues associated with new high throughput methods to 

estimate chemical exposure to the SAP for review. ExpoCast is an EPA Office of Research 

and Development (ORD) initiative to develop the necessary approaches and tools for rapidly 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 11 of 111

prioritizing and screening thousands of chemicals based on the potential for human and 

ecological exposure. This focus of ExpoCast is distinct from many existing traditional, lower 

throughput exposure tools that require considerable data to either generate screening level 

assessments on a per chemical basis or full regulatory risk assessment. ExpoCast efforts 

have focused on empirically assessing the uncertainty in high throughput exposure (HTE)

predictions by evaluation and calibration with available monitoring data (e.g., biomonitoring or 

water concentration). By using rigorous statistical techniques to reconcile HTE estimates 

based on limited available data, the uncertainty in the HTE predictions is quantified. EPA 

refers to this framework as the Systematic Empirical Evaluation of Models (SEEM).

In addition to predictions of exposure, the July 2014 SAP reviewed high throughput 

toxicokinetic (HTTK) methods for extrapolating in vitro doses to in vivo concentration (in vitroto-in vivo extrapolation or IVIVE) for chemicals that have been run in a battery of highthroughput endocrine screening assays (e.g., ToxCast). HTTK provides a bridge between 

bioactivity measured in the high-throughput screening assays and exposure by either 

predicting tissue concentrations from an administered dose (i.e., what has been called forward 

toxicokinetics) or inferring administered doses that would be needed to cause tissue bioactive 

concentrations in vivo (i.e., reverse toxicokinetics or RTK). The importance of applying 

HTTK/RTK to high throughput exposure estimates to better discriminate between chemicals for 

prioritization is demonstrated (Figure 1.2). 

Figure 1.2: Bioactive concentrations (top figure) as 50% of the maximal active concentrations 

(AC50; μM) for 500 chemicals run in a variety of ToxCast assays converted to oral equivalent 

doses (mg/kg/day; bottom figure) using high throughput toxicokinetics (HTTK).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 12 of 111

The current FIFRA SAP review focuses on additional HTS bioactivity tools for the estrogen and 

androgen pathways, their use in chemical prioritization, and the initial evaluation of their 

potential to substitute for specific endpoints in lower throughput EDSP Tier 1 screening

assays. In addition, this SAP will review an approach that utilizes both bioactivity and 

exposure to prioritize chemicals for further consideration. In the Integrated BioactivityExposure Ranking (IBER), RTK can be used to estimate the daily administered dose (mg/kg 

BW/day) necessary to produce steady-state in vivo blood concentrations equivalent to 

concentrations showing biological activity in the in vitro HTS assays (μM). The putative 

bioactive administered doses can then be directly compared with predicted exposures (mg/kg 

BW/day) as portrayed in Figure 1.3.

Figure 1.3: A graphical representation of the Integrated Bioactivity-Exposure Ranking (IBER) 

approach indicating no priority chemicals (those that either have no potential bioactivity or no 

potential exposure), low (chemicals for which bioactive concentrations are considerably higher 

than potential exposures), medium, and high priority (chemicals for which bioactivity and 

exposure concentrations are close or overlapping).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 13 of 111

Section 2: Estrogen Pathway 

2.1: Computational Tools and Models

In this section, we describe an approach to combine data from multiple in vitro assays 

measuring estrogen receptor activity to make a weight of evidence prediction concerning “true”

activity. Note that “true” ER activity is determined in this analysis by consistent findings of 

activity across multiple laboratories and assays, and is exemplified by reference chemicals. 

Therefore, a key aspect of the analysis given below is the use of a reasonably large set of wellstudied reference chemicals. The need for this approach, using multiple in vitro assays and a 

mathematical model, arose from our study of a large data set of in vitro assays and chemicals 

[i.e., the ToxCast program (Judson et al. 2010)]. An important outcome of that analysis was 

the pervasive nature of technology-specific false positive results (relative to the expected 

target of the assay, as determined using well-studied reference chemicals) for most or all 

assays, that was largely associated with cytotoxicity and a variety of chemical and cell-type 

specific modes of cell stress. By combining data from multiple assays, and using a model that 

detects these technology-specific false positives, we can arrive at a more confident 

assessment of the in vitro estrogenicity of a chemical.

2.1.1: In vitro data and model: ER

In order to measure the effect of chemical substances on signaling pathways, a variety of in 

vitro assays have been widely used in drug development and toxicity testing programs. These 

assays range from biochemical assays using purified protein to more complex cellular assays 

that can respond to chemical perturbations in multiple ways. This observation is fundamental 

to the concept of “assay interference” (Auld et al. 2008, Baell and Holloway 2010, Thorne et al.

2010, Bruns and Watson 2012), whereby assays designed to measure binding to a protein or 

perturbations of a given pathway may produce false signals when the target protein is 

denatured or when other pathways in the system are altered. The standard approach to deal 

with assay interference issues is to deploy “orthogonal” assays (Miller et al. 2010, Thorne et al.

2010) that help distinguish activity towards the intended target or pathway from non-specific 

activities. In addition to assay interference issues, every assay has inherent limitations such 

as dynamic range or levels of background noise. Using a suite of assays to detect pathway 

perturbations overcomes limitations of a single assay and improves the ability to identify 

potential non-specific effects.

We have investigated several questions concerning pathway activity and assay interference 

using data from a collection of 18 in vitro assays that probe the ER pathway in a mammalian 

system. These assays are a combination of biochemical and cell-based in vitro assays that 

probe mechanistic perturbations of the ER pathway at multiple sites: receptor binding, receptor 

dimerization, chromatin binding of the mature transcription factor, gene transcription and 

changes in ER-induced cell growth kinetics. This battery of in vitro assays was used to screen 

a library of 1812 chemicals. Included in the chemical library were reference ER agonists and 

antagonists, as well as a large number of commercial chemicals with known estrogen-like 

activity (the reference chemicals). The library of 1812 chemicals also has a significant overlap 

with the EDSP Universe of chemicals. A subset of the chemicals would be expected to elicit 

activity in some, but not all assays, given the diversity of cell types and technologies included 

in the in vitro battery. The expected patterns of assay activity include: no assays activated, all 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 14 of 111

agonist assays activated, all antagonist assays activated, specific subsets of assays across 

technologies activated, technology-specific assay activation. To navigate this complexity, we 

developed mathematical and statistical models to infer whether chemicals that activate specific 

patterns of the in vitro assays were more likely to be ER agonists or antagonists, or were more 

likely to be causing assay activity through specific types of assay interference. Additionally, we 

combined the in vitro assay data with information derived from chemical structure and, where 

available, compared the results to data from publicly available sources.

Overall approach: The input data for the model includes chemical structures and 

concentration-response data for 18 ER-related in vitro assays, plus data for many, non-ER in 

vitro assay endpoints (ranging from 186-821 assays, depending on the chemical). The model 

produces scores for agonist activity, antagonist activity and for “false” activity in each assay 

technology. These scores can then be compared with reference chemicals, structure classes, 

and other properties.

Assays and Concentration-Response Analysis: The data used were generated by the EPA 

ToxCast program (Judson, Houck et al. 2010). The data set comprises concentrationresponse data on 1812 chemicals, with each chemical tested in 18 ER pathway assays. The

ER pathway assays include 3 cell-free biochemical radioligand ER binding assays 

[Novascreen (Knudsen et al. 2011, Sipes et al. 2013)]; a set of 3 protein complementation 

assays that measure formation of ER dimers and test for activity against ER-alpha and ERbeta, (each measured at 2 separate times for a total of 6 assay readouts) [Odyssey Thera]; 2 

assays measuring interaction of the mature transcription factor with DNA [Odyssey Thera]; 2 

reporter gene assays measuring RNA transcript levels [Attagene: (Martin et al. 2010)]; 2 

assays measuring reporter protein level readouts [Tox21: (Huang et al. 2014)]; an ER-sensitive 

cell proliferation assay [ACEA: (Rotroff et al. 2013)]; and 2 transactivation antagonist assays 

[Tox21: (Huang et al. 2014)]. The 1812 chemicals used in this model were selected because 

they had data for the full set of 18 ER assays. The assay sources refer to the company or 

laboratory where the assays were performed. The 18 assays used are listed in Table 2.1. 

The assay IDs correspond to Figure 2.1.

The chemicals were run in concentration-response format in all assays except for the cell-free 

binding assays (NVS). These assays were initially run at a single concentration (25 μM), and if 

significant activity (3 mean absolute deviation (MAD above) above the median or 30% activity) 

was seen, the chemical was then run in concentration-response mode. All of the 

concentration-response data were analyzed using the ToxCast data analysis pipeline, which 

automates the processes of baseline correction, normalization, curve-fitting, hit-calling and 

detection of a variety of potential confounders, which are annotated as “caution flags”. This 

pipeline, along with all of the raw and processed data, and annotations is publicly available 

[http://epa.gov/ncct/toxcast/data.html].

All in vitro assays except the Attagene assays were normalized to 17-estradiol in the agonist 

group and 4-hydroxy-tamoxifen in the antagonist group. Attagene data were normalized as a 

fold-change over the solvent (DMSO) control. The concentration-response data for each in 

vitro assay were fit to 3 models: a constant model (no activity), a Hill model, and a “Gain-Loss” 

model, which is a combination of two Hill models, one increasing in signal, and the other 

decreasing at higher concentration. The Gain-Loss model helps account for observed loss of 

activity (likely due to cytotoxicity) at high concentrations for certain chemical-assay pairs. Loss 

of signal assays (e.g., antagonist assays) were converted to gain of signal to make all assay 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 15 of 111

processing consistent. From the Hill model, we derived an AC50, Hill-slope and maximum 

activity (T or Top value). For the Gain-Loss model, there are two AC50 values and two Hill 

slopes (for the gain and loss sections, respectively), but a single Top value. In addition to the 

AC50 values, the model returns the AC10 (concentration at which the activity is 10% of 

maximum) and the ACC (concentration at which the activity reaches the significance cutoff 

threshold). To call an assay-chemical pair a “hit”, at least one concentration has to reach this 

significance threshold, which is typically 10 MAD around the baseline. The per-assay MAD is 

calculated using the distribution of the lowest 2 concentration points for all chemicals run in the 

assay, around the median of those points. The assay baseline is set as the median. To allow 

computational synthesis of the different in vitro assays, a set of synthetic concentrationresponse activities was generated for each chemical-assay pair at standardized concentrations 

using the experimentally-derived AC50, Hill-slope for the gain of signal (but not loss) and Top 

parameters. All AC50 values were in μM, and the synthetic concentrations were a 2-fold 

dilution series of 14 concentrations from 0.01-100 μM. For curves fit with a Gain-Loss model, 

only the gain portion of the curve was used to generate the synthetic concentration-response 

activities.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 16 of 111

Table 2.1: Summary of the in vitro assays used with their ID mapping to the model in Figure 2.1.NVS=Novascreen; 

OT=Odyssey Thera; ATG=Attagene; Tox21=assays run by the NIH Chemical Genomics Center as part of the Federal Tox21 

program. Assay ID

Assay NameSourceGene TargetNormalized Data TypeAssay Design TypeBiological Process TargetDetection TechnologyDetection Technology SubtypeMeasurement Time pointOrganismTissueCell FormatCell Line Name

A1 NVS_NR_bER NVS ESR1

percent 

activity

radioligand 

binding

receptor 

binding

Lysate-based 

radioligand 

detection

Scintillation 

counting 18 bovine uterus

tissuebased cellfree NA

A2 NVS_NR_hER NVS ESR1

percent 

activity

radioligand 

binding

receptor 

binding

Lysate-based 

radioligand 

detection

Scintillation 

counting 18 human NA cell-free NA

A3 NVS_NR_mERa NVS Esr1

percent 

activity

radioligand 

binding

receptor 

binding

Lysate-based 

radioligand 

detection

Scintillation 

counting 18 mouse NA cell-free NA

A4 OT_ER_ERaERa_0480 OT ESR1

percent 

activity

protein fragment 

complementation 

assay

protein 

stabilization

Protein-fragment 

Complementation

Fluorescence 

intensity 8 human kidney cell line HEK293T

A5 OT_ER_ERaERa_1440 OT ESR1

percent 

activity

protein fragment 

complementation 

assay

protein 

stabilization

Protein-fragment 

Complementation

Fluorescence 

intensity 24 human kidney cell line HEK293T

A6 OT_ER_ERaERb_0480 OT ESR1

percent 

activity

protein fragment 

complementation 

assay

protein 

stabilization

Protein-fragment 

Complementation

Fluorescence 

intensity 8 human kidney cell line HEK293T

A7 OT_ER_ERaERb_1440 OT

ESR1 

ESR2

percent 

activity

protein fragment 

complementation 

assay

protein 

stabilization

Protein-fragment 

Complementation

Fluorescence 

intensity 24 human kidney cell line HEK293T

A8 OT_ER_ERbERb_0480 OT ESR2

percent 

activity

protein fragment 

complementation 

assay

protein 

stabilization

Protein-fragment 

Complementation

Fluorescence 

intensity 8 human kidney cell line HEK293T
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 17 of 111 Assay ID Assay Name Source Gene Target Normalized Data Type Assay Design Type Biological Process Target Detection Technology Detection Technology Subtype Measurement Time point Organism Tissue Cell Format Cell Line Name

A9 OT_ER_ERbERb_1440 OT ESR2

percent 

activity

protein fragment 

complementation 

assay

protein 

stabilization

Protein-fragment 

Complementation

Fluorescence 

intensity 24 human kidney cell line HEK293T

A10 OT_ERa_EREGFP_0120 OT ESR1

percent 

activity

fluorescent 

protein induction

regulation of 

gene 

expression Microscopy

Optical 

microscopy: 

Fluorescence 

microscopy 2 human cervix cell line HeLa

A11 OT_ERa_EREGFP_0480 OT ESR1

percent 

activity

fluorescent 

protein induction

regulation of 

gene 

expression Microscopy

Optical 

microscopy: 

Fluorescence 

microscopy 8 human cervix cell line HeLa

A12 ATG_ERa_TRANS_up ATG ESR1

log2 fold 

induction mRNA induction

regulation of 

transcription 

factor activity

RT-PCR and 

Capillary 

electrophoresis

Fluorescence 

intensity 24 human liver cell line HepG2

A13 ATG_ERE_CIS_up ATG ESR1

log2 fold 

induction mRNA induction

regulation of 

transcription 

factor activity

RT-PCR and 

Capillary 

electrophoresis

Fluorescence 

intensity 24 human liver cell line HepG2

A14 Tox21_ERa_BLA_Agonist_ratio Tox21 ESR1

percent 

activity

beta lactamase 

induction

regulation of 

gene 

expression

GAL4 b-lactamase 

reporter gene

Fluorescence 

intensity 24 human kidney cell line HEK293T

A15 Tox21_ERa_LUC_BG1_Agonist Tox21 ESR1

percent 

activity

luciferase 

induction

regulation of 

gene 

expression

Luciferase-coupled 

ATP quantitation Bioluminescence 48 human ovary cell line BG1

A16 ACEA_T47D_80hr_Positive ACEA ESR1

percent 

activity

real-time cellgrowth kinetics cell proliferation RT-CESElectrical Sensor: 

Impedance 80 human breast cell line T47D

A17 Tox21_ERa_BLA_Antagonist_ratio Tox21 ESR1

percent 

activity

beta lactamase 

induction

regulation of 

gene 

expression

GAL4 b-lactamase 

reporter gene

Fluorescence 

intensity 24 human kidney cell line HEK293T

A18 Tox21_ERa_LUC_BG1_Antagonist Tox21 ESR1

percent 

activity

luciferase 

induction

regulation of 

gene 

expression

Luciferase-coupled 

ATP quantitation Bioluminescence 48 human ovary cell line BG1
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 18 of 111

Accounting for cell-stress and cytotoxicity-related assay interference: For many 

chemicals, we observe that there are a large number of hits (positive assay responses) for ER 

and non-ER assays in the concentration range approaching where cytotoxicity is observed. 

(Cytotoxicity is measured using a collection of 35 assays in the ToxCast battery that detect 

cytotoxicity or other measures of cell loss. These are run in several cell lines and primary cell 

types.) We call this set of cell-stress or cytotoxicity-related hits the “burst”. The following 

scheme is used to filter out these non-selective, cell-stress or cytotoxicity-related assay hits. 

For chemicals with two or more positive responses in cytotoxicity assays, we calculate the 

mean logAC50(cytotoxicity) and the median absolute deviation (MAD) of the 

logAC50(cytotoxicity) hits. Next, we calculate the median of the MAD of the 

logAC50(cytotoxicity) distributions across all chemicals to define the global cytotoxicity MAD. A 

new value (the Z-score) is then assigned to each assay hit:

 

globalcytotoxici tyMAD

chemical assay median chemical cytotox Z chemical assaylogAC50( , ) logAC50( , )

( , )





Equation (2.1)

If fewer than 2 cytotoxicity assays are hit, the median cytotoxicity concentration is arbitrarily set 

to 1000 μM, which simply sets all Z-values for assay hits to a value >3. A hit with a large value 

of Z occurs at concentrations significantly below where cytotoxicity is occurring. This hit is 

both unlikely to be caused by cell-stress or cytotoxicity-related processes, and is more likely to 

be able to cause toxicity through a target-selective mechanism. The global cytotoxicity MAD

from the dataset used in this analysis is 0.26 log units.

Reference chemicals: A set of 45 positive and negative reference chemicals were used to 

evaluate the performance of the model. These chemicals have been used previously to 

validate ER in vitro assays and were taken from the OECD Test Guideline 457 (TG457) BG1 

Luc Estrogen Receptor Transactivation test method for identifying estrogen receptor agonists 

and antagonists (OECD 2012). The reference chemicals, their expected potencies, and their 

predicted estrogen agonist and antagonist probabilities (defined below) are given in Table 2.2. 

Note that some chemicals serve as references for both agonist and antagonist modes.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 19 of 111

Table 2.2: Reference Chemicals. [Strong = AC50 <0.0001 μM, moderate = AC50 < 0.1 μM, 

weak = AC50 < 1 μM, very weak = all other activities, inactive = no detected activity].

CASRN Chemical Name Agonist Potency Antagonist Potency

57-63-6 17alpha-Ethinyl estradiol Strong Inactive

50-28-2 17beta-Estradiol Strong NA

56-53-1 Diethylstilbestrol (DES) Strong Inactive

84-16-2 meso-Hexestrol Strong NA

57-91-0 17alpha-Estradiol Moderate NA

140-66-9 4-tert-Octylphenol Moderate NA

53-16-7 Estrone Moderate NA

599-64-4 4-Cumylphenol Weak NA

521-18-6 5alpha-Dihydrotestosterone Weak Inactive

80-05-7 Bisphenol A Weak Inactive

77-40-7 Bisphenol B Weak NA

486-66-8 Daidzein Weak NA

446-72-0 Genistein Weak Inactive

143-50-0 Kepone Weak Inactive

789-02-6 o,p'-DDT Weak NA

58-18-4 17alpha-Methyltestosterone Very Weak NA

520-36-5 Apigenin Very Weak Inactive

85-68-7 Butylbenzyl phthalate Very Weak Inactive

480-40-0 Chrysin Very Weak Inactive

115-32-2 Dicofol Very Weak Inactive

117-81-7 Diethylhexyl phthalate Very Weak Inactive

84-74-2 Di-n-butyl phthalate Very Weak Inactive

120-47-8 Ethylparaben Very Weak NA

60168-88-9 Fenarimol Very Weak NA

520-18-3 Kaempferol Very Weak NA

72-43-5 Methoxychlor Very Weak NA

72-55-9 p,p’-DDE Very Weak Inactive

104-40-5 p-n-Nonylphenol Very Weak NA

1912-24-9 Atrazine Inactive NA

50-22-6 Corticosterone Inactive NA

66-81-9 Cycloheximide Inactive NA

13311-84-7 Flutamide Inactive NA

52-86-8 Haloperidol Inactive NA

52806-53-8 Hydroxyflutamide Inactive NA

65277-42-1 Ketoconazole Inactive NA

330-55-2 Linuron Inactive NA

57-30-7 Phenobarbital Sodium Inactive NA

32809-16-8 Procymidone Inactive NA

50-55-5 Reserpine Inactive NA

52-01-7 Spironolactone Inactive NA

68392-35-8 4-Hydroxytamoxifen (E/Z) NA Active

82640-04-8 Raloxifene NA Active

10540-29-1 Tamoxifen NA Active

54965-24-1 Tamoxifen citrate NA Active

57-83-0 Progesterone NA Inactive
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 20 of 111

Structure of the Network: Figure 2.1 is a graphical representation of the network used to 

evaluate the integrated in vitro assay responses. The model was based on the series of 

molecular events that typically occur in a nuclear receptor-mediated response. The process 

starts with the interaction of a chemical with a nuclear ER (Receptor node R1). For example, 

an ER agonist will cause the receptors to dimerize (node N1), translocate to the nucleus and 

recruit co-factors to form the complete, active transcription factor complex (TF) (node N2). 

This TF then binds to the chromatin DNA (node N3), initiates transcription of mRNA (node N4) 

and subsequent translation to protein (node N5). For ER agonist activity, one downstream 

effect can be cell proliferation (node N6). Each of these processes (with the exception of cofactor recruitment) was measured in the current collection of 18 in vitro assays represented as 

white stars. Table 2.1 provides the assay ID (A1-A18) to match the assay on Figure 2.1 and a 

brief assay description. The ER pathway is shown in two modes, agonist (blue) and antagonist 

(red). The model assumes that a chemical that interacts with the ER will bind in one or both of 

the agonist or antagonist conformations and that this will trigger activity in the appropriate 

pathway. Note that the model will allow for the prediction of mixed agonist/antagonist activity. 

In addition to ER-mediated effects, every in vitro assay is subject to processes that can lead to 

non-specific activity, independent of the ER pathway node that it is supposed to measure. The 

assay interference pathways were modeled as alternate “pseudo-receptors” (gray arrow 

nodes). The details of the process connecting the pseudo-receptors to the assays were 

simplified to a single connection because, in general, we do not know the intermediate details 

of these processes or even the identity of the pseudo-receptors. Note that the pseudoreceptors may not be what a biologist would define as a receptor, but are surrogates for 

generic processes such as cell-stress or cytotoxicity that can lead to non-ER mediated assay 

activity. It is possible to describe many possible alternative pathways, but in general, the data 

are not sufficient to distinguish between alternate models. Pseudo-receptors are assigned to 

each group of assays (technology group) and to each assay individually. Only a single 

example of an assay-specific pseudo-receptor is shown in Figure 2.1, but all assays (with the 

exception of A16/R8 where the assay and the receptor are identical) have their corresponding 

pseudo-receptor.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 21 of 111

Figure 2.1: Graphical representation of the network used in the in vitro analysis of the ER 

pathway across assays and technology platforms. Colored arrow nodes represent “receptors” 

with which a chemical can directly interact. Colored circles represent intermediate biological 

processes that are not directly observable. White stars represent the assays that measure 

activity at the biological nodes. Arrows represent transfer of information. Gray arrow nodes 

are the pseudo-receptors. Each assay (with the exception of A16) has an assay-specific 

pseudo-receptor, but only a single example is explicitly shown, for assay A1. 

Mathematical Model: The computational model assumes that the value (the efficacy, A) 

returned by an assay at a given concentration is a linear sum of the contributions from the 

receptors that it measures, i.e., it is a simple linear additive model:

Ai = ∑FijRj

j

Equation (2.2)

where the elements of the F matrix are 1 if there is a connection between a receptor and an 

assay, and zero otherwise. This holds for direct connections, where a receptor is directly 

linked to an assay in Figure 2.1, and for indirect connections, where a receptor is linked 

through one or more internal nodes, designated by the circles in Figure 2.1. Therefore, the 

model assumes lossless transmission of signals from the receptor through the internal nodes 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 22 of 111

to the assays. The goal is then to find a set of Rj values that minimize the difference between 

the predicted assay values (Ai

pred) and the measured ones (Ai

meas) for each chemical and 

concentration. For each chemical-concentration pair, a constrained least-squares minimization 

approach is used where the function being minimized is:

ε

2 = ∑(Ai

pred − Ai

meas)

2

i

+ penalty(R)

Equation (2.3)

where Ai

pred must satisfy the constraints:

Ai

pred ∈ [0,1]

Equation (2.4)

It is possible to assign weights to the assays in the sum of Equation 2.3, but in practice, this 

did not change the results in any qualitative way, and introduced a number of extra free 

parameters into the model. The term, penalty(R), penalizes solutions that predict that many 

receptors are being simultaneously activated by the chemical. It is given by

penalty(R) = α

(SR2)

(SR2 + SR0

2

)

Equation (2.5)

where SR is the sum of R values at that concentration, SR0 is a threshold value and α is a 

small number and was varied between 0 and 1. This penalty term helps stabilize the solutions 

and enforces a reasonable physical assumption about chemical promiscuity, i.e., that it is 

unlikely that most chemicals will strongly and specifically interact with many dissimilar 

molecular targets. Values used here are α=0.05 and SR0=0.5. Because there may not be a 

unique solution that minimizes the error term in Equation 2.3, the solution may depend on the 

initial conditions selected. In this analysis, we start from the low concentration end where the 

expected activity is zero, which is then used as the initial condition for all receptors in the 

model. For subsequent concentrations, we use the final values for the previous concentration 

as the initial values.

The model results in a response value (between 0 and 1) for each receptor at each 

concentration. These results are summarized as area under the curve (AUC), which is the 

integral across the concentration range:

AUCj =

1

Nconc ∑ sign(slope) × Rj

(conci)

Nconc

i=1

Equation (2.6)

The factor sign(slope) is included to handle cases where one of the assays or sets of assays is 

active at significantly lower concentrations than the remaining assays. The corresponding 

receptor curve will then rise and subsequently fall, and this AUC needs to be discounted. 

Finally, AUC values are scaled so that AUC(R1)=1 for 17 estradiol. The values of AUC can 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 23 of 111

be related to the potency (AC50), and are useful in interpreting different magnitudes. 

Throughout the analysis, we will use the following terminology for AUC values: AUC(R1) and 

AUC(R2) or AUC(agonist)/AUC(antagonist) will be the activity in the agonist and antagonist 

modes, respectively. AUC(Ri;i>2) will be activity in one of the other pseudo-receptors 3-9. 

AUC(Ai) will be the AUC value for one of the single-assay pseudo-receptors.

In summary, the network model starts with a vector of 26 values for each chemical, 

representing the activity of the chemical against each of the 9 receptors and technology-wise 

pseudo-receptors and 17 assay-specific pseudo-receptors. The model propagates this 

information through the network to create a predicted activity value for each of the assays. 

These predicted values can be compared with the measured values and the receptor vector 

altered to minimize the squared error between the predicted and measured assay activity. 

This procedure is then carried out for each of the 14 synthetic concentrations (0.01-100 μM) to 

provide a concentration-response profile for the receptor and assay-noise node responses.

Evidence for assay-technology-specific “false” activity. Two-dimensional hierarchical 

clustering was performed on the relative potencies of the 18 ER-related in vitro assays (Figure 

2.2). Strong clustering by technology / pseudo-receptor was observed. The activity for 

specific subsets of chemicals was technology-specific suggesting potential narrow assay 

interference. Other chemicals showed activity across a broad range of ER-related assays.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 24 of 111

Figure 2.2: Two-way hierarchical clustering of chemical activity across the 18 in vitro assays 

used to identify ER activity. The heatmap shows –log10(AC50) values for all assays and all 

chemicals with at least one assay hit. Darker red indicates more potent activity (lower AC50), 

while white represents inactive chemical-assay pairs. Note that the assays cluster by 

technology / pseudo-receptor. The “A” and “R” values refer to the assay and 

receptors/pseudo-receptors from Figure 2.1 and Table 2.1. AUC plots for all reference 

chemicals are illustrated in Appendix 1. 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 25 of 111

Interpreting the Results of the Model: One issue that arises with this approach is how to 

interpret these receptor AUC values. For a pure ER agonist (no activity in any of the other side 

pathways), the R1 vs. concentration curve closely resembles the concentration-response 

profile for any of the assays, with activity going from 0 to 1 with an AC50-like value close to that 

observed in the assays. For mixed cases, the concentration-response curves may be more 

complex than a Hill curve and will have a maximum less than 1. Qualitatively, we interpret 

these R values as concentration-specific probabilities that the chemical is interacting with the 

corresponding (pseudo) receptor.

The network model displayed four common types of activity, which can be represented by 

model chemicals (Figure 2.3). Bisphenol A (BPA) shows a clear concentration-response in 

agonist activity (right-hand panel, blue curve). However, there is activity in the R3 and R6 

pseudo-receptors (binding and transactivation assays) that rise at low concentrations and then 

drop. The corresponding in vitro assays show activity at lower concentrations than the others, 

but at high concentrations, substantial evidence points toward agonist activity. Note that BPA 

has activity in one of the antagonist assays (A17, gold), consistent with Selective Estrogen 

Response Modulator (SERM) activity at high concentrations. However, the results of the 

network model do not reflect this weak antagonist behavior. Dimethyl hydrogen phosphite 

shows activity in several of the Odyssey Thera (OT) assays (A4-A11), both for receptor 

dimerization and DNA binding. The resulting model prediction is mixed, with potential activity 

in the agonist and antagonist modes. Alpha-cyclodextrin shows strong activity only in the 

binding assays resulting in a strong assay interference signal in the R3 pseudo-receptor. This 

is almost certainly because this chemical interferes with the radioligand assay, potentially due 

to binding of the radioligand. This molecule is known to bind hydrophobic molecules such as 

fatty acids and cholesterol (Christian et al. 1997). Finally, raloxifene hydrochloride shows 

unambiguous antagonist activity. The most frequent case (not shown here) is one in which 

there is no activity in any assay.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 26 of 111
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 27 of 111
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 28 of 111

Figure 2.3: Results of the model for four example chemicals. For each chemical, the left-hand 

panel shows the synthetic concentration-response data for the 18 assays, colored by assay 

group as defined in the legend. The right-hand panel shows the corresponding magnitude of 

the receptor responses. The agonist receptor (R1) is designated by blue, the antagonist 

receptor (R2) by red and the other pseudo-receptors are colored as indicated in the legend. 

Concentrations are in μM. For chemicals with the “burst” defined (2 or more cytotoxicity hits), 

the burst center is indicated by a vertical red line, and the burst region (starting 3 burst MAD 

below the burst center) is indicated by the gray shaded region. If an assay curve has a 

“caution flag”, the curve is overlaid with *s (e.g. Raloxifene). ER AUC plost for all chemicals 

are provided in Appendix 2. 

Model Results for Reference Chemicals: The AUC values for the ER reference chemicals 

are shown in Figure 2.4. For the positive agonist chemicals, all but two have non-zero AUC 

values, these being diethylhexyl phthalate (DEHP) and dicofol. DEHP is inactive in all assays, 

but dicofol is active in 3 of the 6 dimerization assays near the top tested concentration, which 

results in a positive AUC(R4). These are very weak agonists, so are potentially only active at 

concentrations above where the chemicals have been tested for most or all assays. The 

AUC(R1) values for the other positive chemicals are ordered approximately with the expected 

potency category. None of the negative agonist reference chemicals showed non-zero 

AUC(R1) values. However, three negative agonist reference chemicals had a significant 

pseudo-receptor scores: spironolactone in R8, ketoconazole in R9 and cycloheximide in R3. 

Spironolactone targets the glucocorticoid receptor. It is known that the T47D cell line used in 

the proliferation assay (A16, R8) expresses the glucocorticoid and progesterone receptors, 

and that glucocorticoids and progestins can drive proliferation, just like estrogens (Cappelletti 

et al. 1995; Wan and Nordeen 2002). The activity in ketoconazole is in the two antagonist 

assays, in the cytotoxicity region. It is well-known that these loss-of-signal assays can be 

confounded by cytotoxicity, so this is the likely explanation for this false-positive call. 

Cycloheximide, a DNA cross-linking agent, activates two of the three cell-free binding assays, 

so there may be some type of chemical reactivity driving protein denaturation that is leading to 

this false activity. 

All four of the positive antagonist reference chemicals are positive and strong. With the 

exception of Di-n-butyl phthalate (DNBP), all of the antagonist negative reference chemicals 

are negative. Most of the negative antagonist reference chemicals are positive references for 

the agonist mode, and they appropriately show up with a high AUC(R1). Di-n-butyl phthalate 

is listed as a very weak positive agonist and a negative antagonist, while we see it called as 

very weak in both modes, but with this activity showing only in the cytotoxicity region (see the 

inset in the figure). Therefore, the activity in the antagonist assays which drive the antagonist

AUC may be driven by false-positive cytotoxicity loss of signal. In general, we see that the 

positive reference chemicals, with the exception of some that are very weak, are identified as 

having the appropriate activity (agonist, antagonist). Negative reference chemicals either have 

no activity in any assays or are active in one of the pseudo-receptor channels with scores 

higher than for the agonist or antagonist receptors. The AUC agonist (R1) and AUC

antagonist (R2) values for the reference chemicals and plots of the assay and receptor 

concentration-response profiles, are provided as Appendices 3 and 2, respectively).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 29 of 111
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 30 of 111

Figure 2.4: Summary plots of agonist and antagonist activity of reference chemicals and 

breakdown of agonist score across potency category. The top panel displays plots of agonist 

(top) and antagonist (bottom) activity for reference chemicals. Chemicals that are intended to 

be positive are indicated by a green circle, while those intended to be inactive are red. For the 

agonist, the expected potency range is also indicated. For AUC values ≤0.01, the value was 

set to 0.01. For chemicals with one or more pseudo-receptor AUC values greater than the 

expected one, the value is indicated by an X, and the receptor name is indicated. The receptor 

plot for Di-n-butyl phthalate is shown in the inset and the corresponding agonist and antagonist 

AUC points are boxed. The bottom panel shows a box and whiskers plot of the 

AUC(R1/agonist) values for chemicals in the reference activity ranges.

Activity Classifications of the Commercial and Environmental Chemicals: Our ultimate 

goal is to determine whether a chemical can interact with ER based on data which is subject to 

noise and a variety of assay interference types. Figure 2.5 shows the activity against each of 

the receptors and pseudo-receptors, including those accounting for a single assay. From this, 

one can see that there are several pseudo-receptors (e.g. R6, R9) for which there are many 

chemicals with an AUC value≥0.1. In total, 621 out of 1812 chemicals (36%) had at least one 

AUC≥0.1. Only a small number of chemicals (27) have more than one AUC≥0.1. As 

mentioned above, an AUC of 0.1 corresponds approximately to an AC50 of 100 M, which is 

the upper limit of testing for most of the assays.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 31 of 111

Figure 2.5: Heatmap of the activity of the 621 chemicals with an AUC value ≥0.1 in at least 

one receptor. The AUC matrix was filtered to show only values ≥0.1.

We explore several potential causes of pseudo-receptor activity. The first potential cause has 

to do with assay noise. All assays have a background level of noise, and one must define a 

threshold to decide between a hit and a non-hit. To be called a hit in the data analysis 

pipeline, the maximum efficacy must be above a defined statistical criteria, which is typically 10 

MAD above baseline. Given this relatively high threshold (10 MAD, essentially 10 standard 

deviations), one would expect few noise-related false positives if the noise was approximately 

normally distributed. In particular, for full agonists in an assay with only normally distributed 

noise, one should see T values of ~100% for the actives, and T values for inactive chemicals 

should be small (<<10 MAD). We compared the T distributions between the strong agonists 

and the chemicals in each of the pseudo-receptor groups using a t-test. Table 2.3 lists the 

mean T for chemicals active in the pseudo-receptors and in the agonists, and a t-test p-value. 

(Note that for the antagonist assays and R9, the comparator group was the antagonist 

chemicals, AUC(R2) ≥0.1. We see that the actives (AUC≥0.1) in R6, A9, A12, A13, A15 and 

A17 have significantly lower T values than the comparator groups, but are still above the 

statistical threshold, implying that at least some of this pseudo-receptor activity is associated 

with non-normally-distributed assay noise. We also tested to see if the pseudo-receptors had 

AC50 values above that of the comparator (i.e. were they significantly less potent on average). 

In all cases, the mean AC50 for the pseudo-receptor active chemicals was shifted to higher 

values, but the shift was not statistically significant. This implies that the low T values for these 

chemicals are not mostly due to curves with upper asymptotes not resolved because their AC50
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 32 of 111

values were close to the upper concentration tested. As part of the assay analysis pipeline, we 

apply a series of “caution flags” to assay-chemical data sets with specified properties, for 

instance being active with T just above the cutoff, or being inactive with T just below this cutoff. 

To see if some of the pseudo-receptor activity was associated with chemical-assay pairs with 

more-than average warnings, a test was performed for each assay, comparing the number of 

flags with chemicals called as assay-specific pseudo-receptor-active vs. those called as 

agonists. These distributions were compared with a Kolmogorov-Smirnov (KS) test. Assayspecific chemicals for assays A13 and A15 had significantly more caution flags than did those 

for chemicals called as agonists.

A second potential cause of off-target assay activity is cell-stress or cytotoxicity related activity. 

With in vitro assays, one often observes false activity at concentrations near cytotoxicity. We 

term this the cytotoxicity “burst”, although it has also been referred to as the “stress explosion”. 

The Z-score (see methods) is used to quantify how close an assay AC50 is to the cytotoxicity 

region. Qualitatively, Z<3 can be associated with cytotoxicity, and Z≥3 not. For most assays, 

we observe a bimodal distribution of Z-scores with a minimum at ~3, an example of which is 

shown in Figure 2.6 for ATG_ERa_TRANS_up (assay A12). We then compare the distribution 

of Z-scores for each pseudo-receptor with the same distribution for the chemicals classified as 

agonists or antagonists, using a one-sided t-test. From Table 2.3, one can see that many of 

the off-target activities are more likely to be in the burst region, namely R6, R9, A12, A13, A15 

and A17. However, the mean for these chemical sets are not necessarily <3. One specific set 

of pseudo-receptors are A17, A18 and R9, which are the antagonist assays, separately or 

together. These have the lowest mean Z-scores for the off-target chemicals, meaning that hit 

AC50s are concentrated in the cytotoxicity region. This is expected because these are loss-ofsignal antagonist assays which are often confounded by cytotoxicity. Note that for A18, the 

shift in Z-score is not statistically significant, but this is because the comparators (positive 

antagonists) also have low Z-scores on average.

Table 2.3: Statistics for T/top and Z for the pseudo-receptor active chemicals for pseudoreceptors with 10 or more chemicals. The reference or comparator chemicals for A17, A8 and 

R9 are the chemicals with AUC(R2)≥0.1, while all others are the chemicals with AUC(R1)≥0.1. 

NS=not significant, p>0.05.

PseudoReceptor N |T| |T| 

(Agonist) p(T) |Z| |Z| (Agonist) p(Z)

R3 10 94 86 NS 9 8.8 NS

R6 96 60 76 2.00E-09 3.8 7.3 2.10E-14

R8 33 101 110 NS 7.7 7.9 NS

R9 54 98 97 NS -0.57 2.8 2.80E-05

A2 13 91 88 NS 7.5 9.4 NS

A3 23 81 80 NS 6.7 7.9 NS

A9 10 47 101 1.40E-06 6.1 5.4 NS

A10 13 72 85 NS 6.8 6.3 NS

A12 37 53 81 2.10E-09 3.3 7.3 1.50E-06

A13 66 33 71 9.10E-23 5.8 7.3 0.021

A15 71 65 116 2.80E-08 4.5 6.2 0.022

A17 64 75 86 0.037 1.4 3.6 0.019

A18 28 95 101 NS 2.6 1.9 NS
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 33 of 111

Figure 2.6: Z-score plot for the assay ATG_ERa_TRANS_up showing a histogram of Z values 

(the distance between the median cytotoxicity concentration and a chemical’s AC50 in this 

assay, in units of global cytotoxicity MAD) for all chemicals active in this assay. One can see a 

bimodal distribution with one peak at zero (marked with a heavy red line) and another with a 

peak near 6. The light red lines are used to distinguish the well-separated low-Z from high-Z 

regions used for certain calculations. We hypothesize that chemicals active in the low-Z region 

are less likely to be estrogenic than those active in the high-Z region.

Next, chemicals were removed that had pseudo-receptor activity confounded by either low 

efficacy (low T) or activity in the cytotoxicity region (low Z). For each chemical, the median T 

and Z were calculated for the corresponding assays, and a “specificity score” was produced 

using a smooth sigmoid function centered at T=50% and Z=3, with width(T)=10% and 

width(Z)=0.2. Chemicals were excluded at this step if specificity(T) or specificity(Z) was <0.5. 

Figure 2.7 depicts the down-selection of chemicals with the set of filters just described. This is 

because we see a number of model agonists, including some of the very weak reference 

chemicals, where most or all of the activity is in the cytotoxicity region.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 34 of 111

Figure 2.7: Down-selection of chemicals through the steps described in the text. “Active” are 

chemicals with at least one AUC value≥0.1; “High-T” are those with T-specificity score≥0.5; 

“High Z” are those with Z-specificity score ≥0.5. The diagnostic tests listed are described 

below, and are used to explore additional causes of pseudo-receptor activity.

Two other potential cause of false-positive signals in an assay are crosstalk and crossreactivity. Crosstalk is defined here to mean that one readout in a multiplexed assay will be 

activated inappropriately if another target in the assay is activated. Cross-reactivity occurs 

when a chemical that is potent in one receptor (e.g. AR) is weakly potent in others.

The Attagene assay is highly multiplexed, with 52 readouts in a single well in the CIS assay 

and 30 readouts in the TRANS assay. Here we tested to see if there were more actives in 

non-ER assay endpoints for chemicals classified as Attagene pseudo-receptor actives (A12, 

A13, R6) than for chemicals classified as ER inactive. A KS test was performed comparing 

these distributions, separately for the multiplexed CIS and TRANS assay-readout sets. In 

addition to testing the Attagene pseudo-receptor, all other pseudo-receptors were also tested. 

The statistically significant comparisons are shown in Table 2.4. A12 and R6 pseudo-receptor 

chemicals were significantly more active in non-ER TRANS readouts, indicating direct 

crosstalk. However, we also see that A12 chemicals are significantly more active in the nonER CIS readouts, while A13 chemicals are significantly more active in the non-ER TRANS 

readouts (A12 is the TRANS ER readout, while A13 is the CIS ER readout). This is evidence 

of general cross-reactivity. We also see that A18 (one of the cytotoxicity-sensitive antagonist 

assays) is significantly more active in non-ER readouts for both CIS and TRANS.

Crosstalk / cross-reactivity in the Novascreen cell-free radioligand nuclear receptor binding 

assays was tested in the same manner. These are not multiplexed assays, but all use the 

same technology. We would expect pan-assay activity from any chemicals that, for instance, 

caused protein denaturation. Pseudo-receptor active A2 chemicals had a significantly higher 

number of other Novascreen NR actives than did ER inactive chemicals (Table 2.4). Both 

model Agonist and Antagonist (R1 and R2) chemicals also had significantly more Novascreen 

NR actives than did ER-inactive chemicals, indicating possible cross-reactivity (defined as 

where a chemical active in one NR is also active in other related NRs, but at different 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 35 of 111

concentrations). Other non-Novascreen pseudo-receptors did not show any significant 

difference.

Tox21 assays use two different formats (beta-lactamase or BLA and luciferase or LUC), and 

there are multiple NR assays run in each technology, in both agonist and antagonist modes. 

The test for increased activity in the Tox21 non-ER assays showed the following. Chemicals 

classified as A17 (BLA antagonist), had significantly more non-ER BLA antagonist hits than did 

ER-inactive chemicals. This is most likely a reflection of cell stress or cytotoxicity that was not 

already filtered out with the Z filter. Likewise, the A18 (LUC antagonist) chemicals were also 

more active in non-ER LUC antagonist assays than were the ER-inactive chemicals, again 

likely reflecting so-far unaccounted for cytotoxicity. A17 and model-agonist chemicals were 

also significantly over-represented in non-ER LUC antagonist actives. Finally, A12 chemicals 

had more actives in non-ER LUC agonist assays than did the ER-inactive chemicals.

Table 2.4: Statistics for crosstalk / cross-reactivity comparisons. The “Non-ER assay Set” is 

the collection of assays for which non-ER assays were selected for comparison. The 

“Expected Pseudo-Receptors” would be expected to be over-represented in actives in the nonER assay set if there was in-technology crosstalk. The “Correlated Pseudo-Receptors” are 

those for which there was statistically significant association in the KS test. The “Reference 

Mean” is the mean of the number of actives in the non-ER assay set for chemicals classified 

as ER-inactive. The “Pseudo-Receptor Mean” is the mean number of actives in the non-ER 

assays set for chemical classified as the “Correlated Pseudo-Receptor”. The p-value is the 

one-sided KS-test p-value. Only associations with p<0.01 are reported. Rows where the 

“expected” and “correlated” pseudo-receptors match are bolded.

Non-ER Assay Set

Expected 

PseudoReceptors

Correlated 

PseudoReceptor

Reference 

Mean

Pseudo 

Receptor 

mean

p-value

Attagene CIS A13, R6 A12 3.8 15 0.0023

Attagene CIS A13, R6 A18 3.8 18 0.0013

Attagene TRANS A12, R6 A12 1.5 6.7 0.00015

Attagene TRANS A12, R6 A13 1.5 3.4 0.00069

Attagene TRANS A12, R6 A18 1.5 5.9 0.0066

Attagene TRANS A12, R6 R6 1.5 2.7 0.00019

Novascreen NR A1, A2, A3 A2 0.64 2.1 0.01

Novascreen NR A1, A2, A3 Agonist 0.64 1.8 0.00024

Novascreen NR A1, A2, A3 Antagonist 0.64 3.3 0.0015

Tox21 BLA Antagonist A17, R9 A17 0.26 2.2 1.20E-07

Tox21 LUC Agonist A15, R7 A12 0.54 1.1 0.0078

Tox21 LUC Antagonist A18, R9 A17 0.14 1.1 4.30E-06

Tox21 LUC Antagonist A18, R9 A18 0.14 1.7 1.70E-05

Tox21 LUC Antagonist A18, R9 Agonist 0.14 0.41 0.0035

Another potential explanation of the pseudo-receptor actives is that they are so-called PAINS 

(Pan-assay interference compounds (Baell and Holloway 2010)). To test this, we compared 

the activity of the compounds in each of the receptor and pseudo-receptor groups with the ER-
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 36 of 111

inactive set, looking across the large set of ToxCast assays in which these compounds were 

tested. This number of assays ranged from 187 to 821, after the 18 ER assays were excluded. 

The distribution of per-chemical hits with Z≥3 for receptor or pseudo-receptor group was tested 

against the distribution for the inactive group using a one-sided KS-test (High-Z in Figure 2.7). 

We also performed the same test for chemicals in the Low-Z group from Figure 2.7. We see 

significant activity in the broad group of non-ER assays in several pseudo-receptor chemical 

groups among those chemicals showing activity only at low Z (i.e. in the cytotoxicity region). 

These sets are Agonist, Antagonist, R3, R6, A3, A12, A13, A15, A17 and A18. This is a 

reflection that, in the cytotoxicity region, many assays are active, likely due to non-receptormediated processes. At high Z, only the Agonist and Antagonist chemicals are more active 

than ER-inactive chemicals in the broader set of assays. This reflects the fact that many of the 

true ER actives (especially drugs and hormones) appear to show significant cross-reactivity 

with other receptors, although at higher concentrations than where ER is activated (Agonist 

reference and receptor means: 0.026 and 0.072, p=1.8E-8; Antagonist reference and receptor 

means: 0.026 and 0.18, p=6.1E-5).

All chemicals in the tested library were assigned a structure class, based on structure-similarity 

clustering. We tested for a significant association between being in a given structure class and 

a chemical being assigned to one of the receptors or pseudo-receptors after Z and T filtering. 

This analysis used a chi-squared test. The only significant associations were with modelagonist chemicals, AUC(R1)≥0.1 and structure classes genistein-like, paraben, alkyl-phenol, 

steroids targeting the estrogen receptor and steroids targeting the androgen receptor. The 

only structure class significantly associated with model-antagonists was tamoxifen analogs. 

We further examined the broad steroid class of compounds for cross-reactivity. Within the 238 

agonist, antagonist and pseudo-receptor actives of Figure 2.7, there are 10 steroids that target 

other nuclear receptors: AR / androgen receptor (Dehydroepiandrosterone, Bicalutamide, 

5alpha-Dihydrotestosterone, Cyproterone acetate, Testosterone propionate, 4-Androstene3,17-dione), NR3C1/ glucocorticoid receptor (Melengestrol acetate, Triamcinolone, 

Prednisone, Mifepristone, Pregnenolone carbonitrile) and PGR / progesterone receptor 

(17alpha-Hydroxyprogesterone, Norgestrel). Assay A16, using an ER-responsive proliferation 

readout in T47D cells, is also responsive to chemicals targeting the progesterone receptor. All 

four PGR ligands in the chemical set (Hydroxyprogesterone caproate, 17alphaHydroxyprogesterone, Norgestrel, Norethindrone) activate this assay with sub-micro-molar 

AC50s.

We next looked for significant differences in physico-chemical properties between the ERinactive chemicals and chemicals assigned to the pseudo-receptor groups after Z and T 

filtering. A t-test was used. Calculated properties were taken from QikProp 

(http://www.schrodinger.com/QikProp/), ACD Labs (http://www.acdlabs.com/) and MOE 

(http://www.chemcomp.com/MOE-Molecular_Operating_Environment.htm). Electronic 

properties that showed statistically significant differences between pseudo-receptor and 

inactive chemical sets include: charge state at physiological pH (agonist and A12 chemicals 

are significantly more likely to be charged); diploe moment (agonist and A10 chemical have 

lower dipole moments on average); electron affinity (A17 and A18 chemicals have higher 

values on average); ionization potential (A12, agonist and antagonist chemicals have lower 

values on average); polarizability (A18, agonist and antagonists are more polarizable on 

average); and acid pKa (agonist and antagonist chemicals have larger pKa values on average). 

Size-related properties are molecular weight (A18, agonist and antagonist chemicals are larger 

on average); solvent-accessible surface area (SASA) (A17, A18, agonist and antagonist 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 37 of 111

chemicals have larger SASA on average); and molecular volume (A17, A18, agonist and 

antagonist chemicals have larger volume on average). Partition coefficients include octonalwater partition coefficient, logP (A18, agonist and antagonist chemicals have larger logP 

values on average, so are more lipophilic); charge-corrected logP (logD) (agonist and 

antagonist chemicals have larger values on average); and water solubility (logS) (A18, agonist 

and antagonist chemicals have smaller values on average, so are less water soluble and more 

lipophilic as above). Although we see statistical trends between agonists, antagonists and the 

pseudo-receptor actives, in general these effects are not large enough to explain the assayinterference effects in a chemical-by-chemical basis.

We have made a comparison of the results of our model with ER in vitro data from the 

literature. Literature data was mostly taken from the Endocrine Disruptor Knowledgebase 

(EDKB) (Ding et al. 2010; Shen et al. 2013) and ChEMBL (Gaulton et al. 2012). In addition, 

reference sets from published QSAR models were included. The compilation of this curated

ER literature database is in preparation. (See the ER QSAR section below). We selected a 

subset of chemicals from the literature database that had at least 4 reports in the literature, 

with a preponderance reporting the same call (active or inactive; 80% or more must be one 

class). Separate classifications were made for binding assays, agonist assays and antagonist 

assays. The reason for requiring consistency across multiple reports is that the literature 

database has the same types of disagreement between different laboratories and assays as 

we see in the current data set. Most of the chemicals in our data set have few or no 

associated reports in the literature. Table 2.5 summarizes the results of the comparison for 

receptors / pseudo-receptors with at least 5 chemicals with at least 4 reports each. 

Comparisons are made separately for binding, agonism and antagonism. For instance, for 

chemicals classified as agonists (AUC(R1)≥0.1) before Z and T filtering, 53 were reported in 

the literature to be positive for binding, 23 to be positive for agonist activity and 1 to be positive 

for antagonist activity. None of these chemicals was reported to be inactive in any of these 

modes. Note that chemicals that are true agonists or antagonists should be active in binding 

assays. Other rows in the table can be interpreted in the same way. Therefore, we see that 

there is good agreement with agonists and antagonists both before (Initial) and after (Final) Z 

and T filtering. However, there are a number of chemicals, prior to Z and T filtering that we 

have classified as inactive (None) or pseudo-receptor active in R6. The 4 chemicals with 

consistent literature positive calls classified as R6 are Endosulfan, Dieldrin, Endrin and 

Triamcinolone. The first three are polychloro-bicyclic compounds. These are chemically 

similar to Kepone, which is one of the positive agonist reference compounds, and which is 

correctly classified as an agonist. So for these related compounds, we see some, but limited 

ER activity. Triamcinolone is a steroid drug targeting NR3C1 (glucocorticoid receptor).

There are 11 compounds that we classify as negative for which there are multiple literature 

reports of being positive binders. These include Endosulfan 1, which just missed the cutoff for 

being called an agonist (AUC(R1)=0.09); Dipentyl phthalate, a weak antagonist 

(AUC(R2)=0.06); another 7 compounds very weakly active (>10 M) in one or more assays 

(Sodium 2-mercaptobenzothiolate, 4,4'-Dithiodimorpholine, Benz(a)anthracene, 2,4-Di-tertbutylphenol, Dicyclohexyl phthalate, 2-Mercaptobenzothiazole, 5-Fluorouracil), but with activity 

too weak to have any pseudo-receptor value above the 0.1 threshold. The final two chemicals 

are maneb and mancozeb. These were not active in any of the ER assays, but are among the 

most promiscuous chemicals in the ToxCast battery. Overall, this set of 11 chemicals is 

relatively promiscuous, especially in the low-Z region, indicating that some of this reported 

activity may be due to assay interference.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 38 of 111

Table 2.5: Results of comparison of the current model results with activity calls from the 

literature. A comparison for a chemical was made if at least 4 literature report for that chemical 

in one of the 3 modes (Binding, Agonist, Antagonist) were found, and if the calls were at least 

80% concordant (positive or negative). The numbers in the cells of the table are counts of 

chemicals. Initial and Final chemical sets refer to before and after T and Z filtering. Chemical Set Receptor N (Binding Positive) N (Binding Negative) N (Agonist Positive) N (Agonist Negative) N (Antagonist Positive)

N (Antagonist Negative)

Initial Agonist 55 0 25 0 1 0

Initial Antagonist 5 0 0 0 3 0

Initial None 11 3 2 0 2 0

Initial R6 4 1 3 0 0 0

Final Agonist 41 0 19 0 1 0

Final Antagonist 5 0 0 0 3 0

ER Model Summary: This section has demonstrated a method to combine data from multiple 

in vitro assays in the ER pathway in order to generate a consensus call of ER activity. The 

need for this approach is driven by the presence of false positive and false negative calls in all 

assays. An ongoing topic of research is trying to use chemical structure and properties to 

understand exactly why specific chemicals cause this (mainly) false positive activity.

ER QSAR Overview: For technical and cost-related reasons, it will be difficult to procure all 

EDSP Universe chemicals and process them through all of the relevant in vitro assays. 

Because of this, we are developing structure-based models (QSARs and molecular docking) to 

extrapolate from the current in vitro data set, and to guide the acquisition of further in vitro data 

in a targeted fashion. The results of the ER model described above provide a large, highquality training set to use for developing and testing QSAR models. We anticipate that we will 

use an iterative approach to evaluating the EDSP Universe for ER activity consisting of in vitro

screening on the ToxCast chemicals, running the model to determine actives in that set, using

the entire data set to build QSAR models, taking the predicted actives from the QSAR models 

and testing them in in vitro assays, refining the model with new data, and iterating. This 

section is included to briefly summarize our activities and to demonstrate progress in an area 

deemed important by the January 2013 SAP. All methods and results of this effort are being 

documented in several manuscripts that should be published in 2015.

Just as every in vitro assay has false positives and false negatives, so will every structurebased model. So, just as in the case of the ER in vitro assays, we have compiled predictions 

from a large suite of structure-based models, in an effort called CERAPP (Collaborative 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 39 of 111

Estrogen Receptor Activity Project). The CERAPP combines multiple structure-based 

computational models for prediction of ER activity, and uses the predicted results of these 

models in prioritization of chemicals for further testing. The CERAPP approach will (at 

minimum) address the following questions:

1. What chemicals are predicted to be ER-active in all or the majority of models? These 

will be prioritized for further study.

2. What chemicals are predicted to be ER-inactive in all or a majority of the models? 

These will be set aside relative to any follow-up.

3. What chemicals are outside of the domain of applicability of all or most models? These 

chemicals are candidates for generating additional in vitro data to fill in these 

uncharacterized regions of chemical space.

For CERAPP, the EPA provided the results of the ER model discussed in the previous section 

for 1677 chemicals with well-defined structure (a subset of the 1812 chemicals described 

above are mixtures), and collection of 32,464 chemical structures for which ER activity was to 

be predicted. Each group then either used the EPA data to train a new model, or used an 

existing model that had been trained with other data. A subset of the CERAPP partners then 

compiled data from the literature on approximately ~7,500 additional chemicals with limited ER 

data. This literature data set was then used to evaluate the predictive accuracy of the 

individual models. Finally, the models were combined to give a consensus prediction score 

across the large 32K chemical structure library.

A total of 17 research groups in the U.S. and Europe submitted models based on this training 

set or training data they had from other sources. Some groups submitted multiple models. 

These models used a variety of machine learning approaches, as well as molecular docking. 

Models were separately built for binding, agonist and antagonist activity.

The CERAPP goals are being carried out in multiple steps that can be categorized into 5 parts

including chemical structure curation, QSAR/docking modeling, experimental data preparation, 

model evaluation and consensus modeling. Each step was assigned to a workgroup 

comprised of representatives from a subset of the participating research groups. A common 

theme is that these workgroups would define and document best practices.

In order to evaluate models, we obtained a set of ~7000 compounds with ER activity from the 

literature, and built consensus models from that. One finding of this effort is that data from the 

literature for which the same chemical was tested in multiple labs with different assays, had a 

similar rate of discordance to what we see in the ToxCast ER assays. We evaluated the 

performance of each model and the model consensus against increasingly refined subsets of 

the literature data. By more refined, we mean subsets in which the chemicals were tested in 

increasing number of independent labs, and had consistent (positive or negative) results 

across the labs.

The final result of this effort is consensus scores for binding, agonism and antagonism, 

combining predictions from all of the relevant models. The combining method weighted 

models based on their prediction accuracy against the training and external test sets. One 

important observation about the individual models is that their overall balanced accuracy 

values (mean of sensitivity and specificity) do not vary greatly, but some models 

overemphasize sensitivity over specificity, and others go the opposite direction. The 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 40 of 111

consensus model is more balanced. Overall, the consensus predicts about 10% of the 

evaluated chemicals could interact with the estrogen receptor. This consensus model 

achieves ~90% balanced accuracy when predicting the most refined and consistent set of 

experimental data. A subset of these chemicals will be selected for inclusion in later rounds of 

in vitro testing, and will then be evaluated with the model described above. The performance 

of the final consensus model is illustrated in Figure 2.8.

Figure 2.8: Summary of ER QSAR predictivity against literature calls. Each curve shows the 

balanced accuracy of the consensus QSAR model against a subset of the literature having at 

least N consistent calls (sources) for each chemical (active vs. inactive). The x-axis is the 

fraction of QSAR models that call a chemical positive used to define “consensus-positive”.

2.2: Interpretation and application: ER

In this section, we present various analyses comparing ToxCast models of in vitro ER 

bioactivity with in vivo activity and evaluate the scientific confidence in using ToxCast data to:

1. Prioritize chemicals in the EDSP universe based on bioactivity;

2. Provide data that may contribute to future weight of evidence evaluations of a 

chemical’s potential to interact with the estrogen receptor; and

3. Substitute for specific assays and/or endpoints in the EDSP Tier 1 battery.

The EDSP is tasked with rapidly screening approximately 10,000 chemicals in the chemical 

universe defined by the FIFRA and SDWA statutes, and as previously mentioned, the 

chemicals identified for EDSP screening on List 1 and List 2 were selected based on potential 

exposure or pesticide registration review schedules, rather than indications of potential 

bioactivity. Conducting the EDSP Tier 1 screening battery requires a great deal of financial, 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 41 of 111

temporal, animal, and human resources. Since the first test orders were issued in 2009, 52 

chemicals have been screened in the battery and List 1 weight of evidence determinations are 

being finalized by EPA. Incorporating computational tools in the existing EDSP framework can 

literally transform a decades to centuries-long undertaking into a project that can likely be 

completed in years, while simultaneously increasing the likelihood that the most bioactive 

chemicals are prioritized.

This section illustrates the proposed prioritization approach for estrogen receptor agonist 

activity, as most environmental chemicals that interact with the estrogen receptor are expected 

to act as agonists (Safe et al. 2001). The prioritization approach uses EPA’s ToxCast high 

throughput in vitro screening assays, computational models, and semi-automated curation of 

peer-reviewed endocrine toxicology literature, and discusses scientific confidence in the 

results. This strategy would represent a pivotal paradigm shift in the EDSP approach 

consistent with recommendation of EPA’s strategic plan for evaluating the toxicity of chemicals

and the 2007 NAS report, specifically to: 1) provide broad coverage of chemicals examined; 2) 

reduce the cost and time of toxicity testing; 3) reduce animal use; and 4) develop a robust 

scientific basis for assessing health effects of environmental agents.

2.2.1: Curated literature review of uterotrophic literature

Curation process: The uterotrophic bioassay is a short-term in vivo screen for estrogenicity 

and is one of the 11 EDSP Tier 1 screening assays. The EPA EDSP uterotrophic assay,

OCSPP 890.1600, is performed on immature female rats or ovariectomized (OVX) adult 

female mice or rats (U.S. EPA, 2011), and is a measure of in vivo estrogen receptor agonism 

in female mammals without an intact hypothalamo-pituitary gonadal axis. To build confidence 

in the ability of computational toxicology methods to prioritize and screen chemicals for 

potential estrogenic activity, concordance between results from ToxCast in vitro ER assays

and the rodent uterotrophic bioassay was examined.

The National Toxicology Program Interagency Center for the Evaluation of Alternative 

Toxicological Methods (NICEATM) conducted a comprehensive literature search to identify 

uterotrophic studies for all 1812 chemicals run in the 18 high throughput ToxCast assays that 

are relevant to the ER pathway. Semi-automated literature searches were performed with the 

NCBI PubMatrix tool (http://pubmatrix.grc.nia.nih.gov/), using both chemical name and CASRN 

as the search terms. The modifier terms “uterotrophic assay”, “uterotrophic”, “uterotropic” (a 

common alternative) and “uterine weight” were used to identify articles of interest. Relevant 

articles were then downloaded for manual data curation. The search yielded more than 1000 

papers, of which 670 were deemed potentially relevant to the ToxCast chemical inventory. 

These manuscripts were downloaded and detailed study information (species, strain, study 

type, assay type, assay target, dose administration route, age at first dose, OVX status, age at 

OVX, dosing length/frequency, number of doses, number of animals, positive/negative 

controls, diet, time of necropsy, lowest effect level, response observed, maximum dose tested) 

were recorded for each chemical/study/protocol combination along with the reported bioactivity

for the dose range tested; “active” with a corresponding lowest effective level (LEL) or 

“inactive” with a corresponding highest dose tested (HDT). Two NICEATM scientists 

independently reviewed and extracted data from each report.

Compliance with study design requirements set forth in EPA OCSPP 890.1600 and OECD 440 

Test Guidelines was evaluated based on the information extracted from each report. A study 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 42 of 111

was considered to be “guideline-like” (GL) if all six of the following minimum criteria (MC) were 

met:

1. Animal model (any of the following) 

a. Immature rats, ovariectomized (OVX) adult rats, or OVX adult mice are 

acceptable (immature mice are not acceptable).

b. OVX animals: OVX should be performed between six and eight weeks of age 

(allowing at least 14 days post-surgery before dosing for rats and seven days 

post-surgery for mice)

c. Immature rats: dosing should begin between post-natal day (PND) 18 and PND 

21, and be completed by PND 25

2. Group size 

a. Each control group should have a minimum of three animals and each test group 

should have a minimum of five animals

3. Route of administration

a. Acceptable routes of administration: oral gavage (p.o.), subcutaneous (s.c.) 

injection, or intraperitoneal (i.p.) injection

4. Number of dose groups

a. Minimum of two dose levels plus positive control and vehicle control

5. Dosing interval

a. Dosing for a minimum of three consecutive days

6. Necropsy timing

a. Animal necropsy should be carried out 18-36 hours after the last dose

Data for an additional criterion (MC7) was recorded to indicate whether or not levels of 

phytoestrogen in the diet were reported, but this criterion was not used to exclude data due to 

the small number of studies reporting this information (<5% of the 670 papers reviewed). 

A score of 0 (no) or 1 (yes) was recorded for each of the minimum criteria (MC 1-6) based on 

whether the study/protocol fulfilled that particular requirement. These scores were added to 

yield a total score for each study, and only those studies meeting all six criteria were 

considered GL. It should be noted that compliance with the minimum criteria identified above 

is not equivalent to a thorough assessment of overall study quality (e.g., comprehensive 

systematic review).

The resulting database of uterotrophic outcomes (UTDB) currently contains data for 98 

chemicals in the ToxCast inventory, with information from 442 GL studies extracted from 92 

publications. This compiled information can serve as a valuable anchoring point for assessing 

the concordance of uterotrophic outcomes and in vitro/computational approaches. The 

“NICEATM_UTDB_GL.xlsx” provides a description of each study chemical combination

(Appendix 4). This file represents ongoing work and new information will be added as it 

becomes available.

Impact of study design on uterotrophic outcome: There are six basic study designs that 

can meet GL criteria, depending on species (rat or mouse), route of administration (oral or 

injection1), and use of OVX (rat or mouse) or immature (rat only) animals. A breakdown of 

 

1 While both subcutaneous and intraperitoneal routes of injection are acceptable as GL, 99% of the studies in the database 

used the subcutaneous route. Therefore, for this analysis, “injection” includes both subcutaneous and intraperitoneal studies.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 43 of 111

results by study design is provided in Table 2.6. Data from chemicals commonly used as 

positive controls (i.e., estradiols) were removed from this analysis due to the large number of 

results and inherent bias associated with their inclusion (i.e., negative results would indicate a 

failed “positive” control and would therefore not typically be reported), leaving 357 GL 

uterotrophic entries. The immature rat model was used for 73% (268/357) of the studies in the 

database, with 73% (196/267) of these using injection as the route of administration. The 

distribution of active and inactive outcomes was similar across rat models, with active 

outcomes being more prevalent overall (74% [228/310] of all rat outcomes were active). This 

is in contrast to uterotrophic results reported in mouse models, in which 64% (30/47) of all 

outcomes were inactive, with the OVX_mouse_oral design producing inactive outcomes in 

73% (16/22) of the studies. It should be kept in mind that the selection of chemicals tested in 

these studies is neither random nor uniformly distributed with respect to uterotrophic 

bioactivity, and the performance of a particular study design, especially those with a small 

number of studies (e.g., OVX_rat_injection), could be heavily influenced by a single publication 

from one laboratory testing multiple chemicals in that particular study design.

Table 2.6: Distribution of uterotrophic outcomes by study design.

Im_Rat 

Inj

Im_Rat 

Oral

OVX_Rat 

Inj

OVX_Rat 

Oral

OVX_Mouse 

Inj

OVX_Mouse 

Oral

# Active 142 52 29 5 11 6

# Inactive 55 19 3 5 14 16

%Active 0.72 0.73 0.91 0.50 0.44 0.27

%Inactive 0.28 0.27 0.09 0.50 0.56 0.73

%Total 0.55 0.20 0.09 0.03 0.07 0.06

Im -immature, OVX -ovariectomized, # Active -the number of experiments reporting 

substances as active, # Inactive -the number of experiments reporting substances as inactive. 

Data for positive controls is not included in this table.

Reproducibility of uterotrophic outcomes: The UTDB provides an opportunity to assess 

both the qualitative and quantitative reproducibility of the uterotrophic assay across many 

chemicals tested at many different laboratories. Of the 70 chemicals2in the database with at 

least two reported uterotrophic studies, 18 (26%) had at least one study with a discordant

outcome, resulting in a chemical being classified as both “active” and “inactive” for 

uterotrophic bioactivity, shown in Figure 2.9. A list of chemicals with discordant results is 

provided in Table 2.7, and a radar chart of each chemical in the table, reflecting study design 

and outcome, is provided in Appendix 5. Discordant outcomes could result from differences 

in overall study design and/or the range of doses tested in each study (e.g., the highest dose 

tested in an “inactive” study may have been below the dose that would produce a tissue 

concentration required for bioactivity).

Of the 18 chemicals listed in Table 2.7, 10 (56%, shaded) of these had discordant uterotrophic

outcomes that may be attributable to differences in study design. Results from the testing of 

butylparaben provide an example of how study design can impact uterotrophic outcomes, as 

shown in Figure 2.10. In the case of butylparaben, all active outcomes were reported in the 

 

2 Chemicals routinely used as positive controls, such as estradiols, were uniformly positive and excluded from this analysis.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 44 of 111

three study designs using s.c. injection as the route of administration (immature rat, OVX rat, 

OVX mouse), whereas inactive results were reported for both study designs that used oral 

dosing (immature rat, OVX mouse). 

The eight non-shaded chemicals in Table 2.7 had discordant outcomes reported for studies 

using the same basic study design. Uterotrophic outcomes were compared to determine if the 

HDT for “inactive” outcomes was below the LEL reported for “active” outcomes, in which case 

the results would actually support one another. LEL and HDT values from the study design 

producing the most discordant results (immature_rat_injection) are provided in Figure 2.11. 

For chemicals which had discordant outcomes reported in the same study design, it was 

common for HDT to be above LEL doses reported in other studies, although the difference in 

these values was normally less than one order of magnitude. Most studies in the UTDB 

typically used log-spaced dosing and four or fewer doses, resulting in poor resolution of lowest 

effect levels (generally defined as >10% increase in wet uterine weight, p<0.05), which could 

explain LELs and HDT reported at similar doses. However, reports of “inactive” results 

obtained at doses well above all reported LELs are difficult to reconcile. For example, see 

Figure 2.11, CASRN 80-05-7 (Bisphenol A), “active”by s.c. injection at 2 mg/kg/day in one 

study using the immature rat model, and “inactive” by s.c. injection at 1000 mg/kg/day in 

another study using the same model.

Figure 2.9: Results from uterotrophic studies for chemicals that have more than one GL 

study. Blue bars represent the number of “active” reports; red bars represent the number of 

inactive reports.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 45 of 111

Table 2.7: List of chemicals with discordant uterotrophic results in GL studies. Shaded 

chemicals had discordant uterotrophic outcomes (active / inactive) in study designs that 

differed significantly from one another. The remaining chemicals (non-shaded) had discordant 

results reported in assays with the same basic study design.

CASRN Name # Active # Inactive

80-05-7 Bisphenol A 37 6

446-72-0 Genistein 27 1

72-43-5 Methoxychlor 18 1

789-02-6 o,p'-DDT 15 1

94-26-8 Butylparaben 8 2

56-53-1 Diethylstilbestrol 8 1

104-40-5 4-n-Nonylphenol (linear, para) 5 4

140-66-9 4-(1,1,3,3-Tetramethylbutyl)phenol 3 1

120-47-8 Ethylparaben 1 3

119-61-9 Benzophenone 1 2

99-76-3 Methylparaben 1 2

56-55-3 Benz(a)anthracene 1 2

1806-26-4 4-Octylphenol 1 2

94-13-3 Propylparaben 1 2

52645-53-1 Permethrin 1 1

50-55-5 Reserpine 1 1

520-36-5 Apigenin 1 1

486-66-8 Daidzein 1 1
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 46 of 111

Figure 2.10: Example of differences in study design that may be associated with 

discordant uterotrophic outcomes. The number of active (black lines) and inactive (red 

lines) results is shown for butylparaben as a function of study design.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 47 of 111

Figure 2.11: LEL / HDT for chemicals with discordant results in the 

Immature_Rat_Injection study design. Markers reflect Lowest Effect Levels (LEL) for 

chemicals classified as “active” in the uterotrophic bioassay (blue markers), and highest dose 

tested (HDT) for those with “inactive” uterotrophic outcomes (red markers).

2.2.2: Comparison of ToxCast ER agonist AUC data and Uterotrophic studies

Chemicals used to assess performance of ER agonist model: OECD Guidance Document 

(Number 34) on the validation and international acceptance of new or updated test methods for 

hazard assessment indicates that the relevance of a new test method should be characterized 

against data generated with a list of reference chemicals in the original test method (OECD, 

2005). These reference data should cover the expected range of responses and the range of 

chemicals to which the assay will be applied (the domain of applicability).

Reference chemical lists previously developed by ICCVAM and OECD to assess the 

performance of ER agonist assays were based on the reported activity of these chemicals in in 

vitro assays, and therefore only provide a reference for expected in vitro activity. A small 

number of chemicals (n=7) were used for the OECD validation study of the uterotrophic assay, 

but a larger consensus list of reference chemicals has not been developed to assess the 

performance of alternatives to the uterotrophic assay. Consequently, a list of reference 

chemicals was developed de novo using studies from the uterotrophic database (UTDB) 

described in Section 2.2.1. Because the focus of the current evaluation is specific to ER 

agonist activity, chemicals with strong ER antagonist activity (tamoxifen, clomiphene citrate, 

and raloxifene hydrochloride) were excluded from consideration as reference chemicals. The 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 48 of 111

following criteria, which characterize the overall potential of chemicals in the UTDB to show 

bioactivity in the uterotrophic assay, were used to develop a list of reference chemicals:

Active – a chemical was classified as active if two or more independent GL studies classified 

the chemical as active in the uterotrophic bioassay, regardless of the number of inactive 

studies.

Inactive – a chemical was classified as inactive if two or more independent GL studies 

classified the chemical as inactive in the uterotrophic bioassay, and there were no other GL 

studies reporting the chemical as being active in the uterotrophic bioassay.

The resulting list of 42 reference chemicals for uterotrophic bioactivity (29 active, 13 inactive) is 

provided in Table 2.8. It should be noted that this list is dynamic and will be updated as more 

information is added to the database. 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 49 of 111

Table 2.8: 42 Reference Chemicals used for Uterotrophic Bioactivity.

CASRN Name GL

_Active

GL_Inact

ive Bioactivity L_LEL (mg/kg/day)

R1 

AUC

50-28-2 Estradiol 30 0 Active 0.00001 1.01

57-63-6 Ethinyl Estradiol 61 0 Active 0.0001 1.01

56-53-1 Diethylstilbestrol 10 1 Active 0.00025 1.01

72-33-3 Mestranol 3 0 Active 0.027 0.91

57-91-0 Alfatradiol 2 0 Active 0.4 1.01

50-27-1 Estriol 4 0 Active 0.67 0.96

789-02-6 o,p'-DDT 15 1 Active 1 0.49

446-72-0 Genistein 27 1 Active 1 0.77

80-05-7 Bisphenol A 37 6 Active 2 0.66

68-22-4 Norethindrone 2 0 Active 2 0.67

53-16-7 Estrone 8 0 Active 2 0.99

474-86-2 Equilin 2 0 Active 2 0.98

17924-92-4 Zearalenone 5 0 Active 2 0.89

1478-61-1 Bisphenol AF 4 0 Active 4 0.82

58-18-4 Methyltestosterone 3 0 Active 10 0.44

80-09-1 Bisphenol S 2 0 Active 20 0.32

77-40-7 Bisphenol B 2 0 Active 20 0.75

72-43-5 Methoxychlor 21 1 Active 20 0.38

599-64-4 p-Cumylphenol 2 0 Active 20 0.54

521-18-6 Dihydrotestosterone 3 0 Active 20 0.50

104-43-8 4-Dodecylphenol 3 0 Active 40 0.56

140-66-9

4-(1,1,3,3-

Tetramethylbutyl)phenol 3 1 Active 56 0.56

94-26-8 Butylparaben 8 2 Active 59 0.38

98-54-4 p-tert-Butylphenol 2 0 Active 100 0.22

131-56-6 2,4-Dihydroxybenzophenone 3 0 Active 100 0.40

80-46-6 4-(1,1-Dimethylpropyl)phenol 4 0 Active 200 0.40

5153-25-3

Benzoic acid, 4-hydroxy-, 2-

ethylhexyl ester 2 0 Active 200 0.51

131-55-5 Benzophenone-2 6 0 Active 200 0.56

556-67-2 Octamethylcyclotetrasiloxane 3 0 Active 250 0.00

99-96-7 4-Hydroxybenzoic acid 0 2 Inactive NA 0.00

87-86-5 Pentachlorophenol 0 2 Inactive NA 0.00

84-75-3 Dihexyl phthalate 0 2 Inactive NA 0.02

84-74-2 Dibutyl phthalate 0 2 Inactive NA 0.04

84-66-2 Diethyl phthalate 0 2 Inactive NA 0.00

84-61-7 Dicyclohexyl phthalate 0 2 Inactive NA 0.05

61-82-5 Amitrole 0 2 Inactive NA 0.00

520-18-3 Kaempferol 0 3 Inactive NA 0.34

51630-58-1 Fenvalerate 0 2 Inactive NA 0.00

1912-24-9 Atrazine 0 2 Inactive NA 0.00

1461-22-9 Tributylchlorostannane 0 2 Inactive NA 0.00

117-81-7 Bis(2-ethylhexyl)phthalate 0 2 Inactive NA 0.00

103-23-1 Bis(2-ethylhexyl)hexanedioate 0 2 Inactive NA 0.00

GL_Active =number of guideline-like (GL) studies reporting activity, GL_Inactive =number of 

GL studies reporting no activity at the highest dose tested, L_LEL = lowest of the lowest effect 

levels reported for the chemical. The last column is the ER agonist (R1) AUC value (see 

Section 2.1.1), with values for the two chemicals that show discordance between in vitro

activity and in vivo uterotrophic outcomes highlighted in red.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 50 of 111

The 42 chemicals listed in Table 2.8 (29 active, 13 inactive) were used to assess the 

performance of the of the ER Pathway model for predicting uterotrophic bioactivity. A clear 

separation of reference chemical bioactivity (active/inactive) is apparent when viewed as a 

function of model score, as shown in Figure 2.12. When using an ER agonist AUC value of 

>=0.1 as the lower limit for bioactivity, the model scores served as a good surrogate for the 

qualitative assessment of uterotrophic bioactivity, with a balanced accuracy of 95% (Table 

2.9). 

Figure 2.12: ER Agonist (R1) AUC and uterotrophic outcomes. Uterotrophic reference 

chemicals are plotted according to their ER model score (ER agonist AUC) and color coded 

based on their uterotrophic bioactivity; active (blue), inactive (red).

Table 2.9: ER (Agonist) AUC Performance using uterotrophic Reference Chemicals.

# True Pos 28

# True Neg 12

# False Pos 1

# False Neg 1

PPV 0.97

NPV 0.92

BA 0.95

Sensitivity 0.97

Specificity 0.92
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 51 of 111

Performance of ER Agonist AUC model scores (>=0.1 as active) when used to assess the 

potential uterotrophic bioactivity of reference chemicals. PPV -positive predictive value, NPV -

negative predictive value, BA -balanced accuracy.

The single false positive chemical was kampferol (ER agonist R1 AUC=0.33, inactive in

uterotrophic assay,). Kaempferol was classified as inactive in three uterotrophic assays; 1) an

Ovx mouse study using either oral or s.c. dosing for seven days at concentrations up to 1000 

mg/kg/day, and 2) an immature rat study using oral dosing for four days at concentrations up 

to 200 mg/kg/day. Kaempferol is weakly active in the ER Agonist (R1) Model (AUC=0.33). 

Weak activity of kaempferol has previously been reported in other in vitro systems (Zava, 

1997), supporting the ER agonist AUC R1 characterization of this chemical as a “weak ER 

agonist” in vitro. One explanation for chemicals showing activity in vitro but not in vivo could 

be the lack of bioavailability in the animal model due to a number of factors, including 

metabolic transformation and/or high clearance. Extensive metabolism of kaempferol by UDPglucuronosyltransferase has been reported in both humans and rats (Oliveira and Watson 

2000) and may account for the discrepancy between the in vitro and in vivo activities. In 

addition, the design of the uterotrophic studies with kaempferol may not have been optimal to 

detect weak ER bioactivity, as testing in mice is more likely to produce an inactive result (see 

Section 2.2.1), and oral dosing in rats would presumably result in lower systemic exposure 

levels (relative to s.c. injection) and higher clearance by the gut/intestinal UDPglucuronosyltransferases cited above.

The single false negative chemical was octamethylcyclotetrasiloxane (ER agonist R1 

AUC=0.00, active in uterotrophic assay). Octamethylcyclotetrasiloxane (D4; Figure 2.13) is 

used in the formulation of various hair care, skin care, and deodorant products, as well as 

topical pharmaceuticals and as a component in breast implants. Depending on the product 

type, the concentration of D4 in formulations can be up to 54%. D4 was classified as active in 

three uterotrophic assay protocols; 1) a study using two strains of immature rats (SD and F344) with oral dosing for four days reporting bioactivity at 250 mg/kg/day, and 2) an OVX 

mouse study using oral dosing for three days reporting bioactivity with a LEL of 250 mg/kg/day

(the highest LEL of any reference chemical). A report from the EU Scientific Committee on 

Consumer Safety (SCCS, 2010) cited additional uterotrophic studies with D4 which 

demonstrated that the uterotrophic effects of D4 were ablated by pre-treatment with ICI 

182,780, an estrogenic receptor (ER) antagonist. Another cited study demonstrated that 

ovariectomized αER KO mice showed no increases in uterine weights when treated with D4, 

adding further evidence that D4 (or a metabolite) has weak estrogenic activity mediated 

through estrogen receptor alpha. The lack of in vitro activity by D4 may be due to a number of 

factors, including: i) in vivo metabolic conversion to an active compound, which cannot be 

replicated by the in vitro systems, ii) uterotrophic bioactivity mediated via a non-ER 

mechanism, iii) weak in vivo activity (high LEL) that would require serum concentrations that 

are too high to be replicated by in vitro systems, or iv) inaccurate assumptions regarding the

quantities of test material used during the screening process (quantity / purity). This last 

possibility is raised due to the fact that D4 is highly volatile (vapor pressure 132 Pa at 25°C), 

and its identify and purity in the HTS stock solution have not yet been confirmed due to the 

structure’s incompatibility with the separation/detection systems currently being employed in 

the ToxCast chemical QC program.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 52 of 111

Figure 2.13: Structure of Octamethylcyclotetrasiloxane (D4), CASRN 556-67-2

A further quantitative evaluation was conducted to examine the relationship between ER 

agonist (R1) AUC and the LELs reported for the uterotrophic reference chemicals, Figure 

2.14. Based on these data, there appears to be a mathematical relationship between in vitro

AUC values and the level of in vivo potency as determined using the LEL, raising the 

possibility of both qualitative and quantitative utility for the ER agonist (R1) pathway model. 

Figure 2.14: Relationship between AUC and LEL for uterotrophic reference chemicals. 

LELs for each bioactive uterotrophic reference chemical (y-axis, log10 scale) are shown as a 

function of ER agonist (R1) AUC (y-axis, linear scale). The markers are colored by chemical, 

with all chemicals having a minimum of two reported LELs from independent studies.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 53 of 111

Analysis of additional (non-reference) chemicals in the UTDB: Chemicals in the UTDB 

without independent confirmation of their bioactivity in a GL study (i.e., those that were active 

or inactive in single study, or inactive in multiple studies with a single active report) were 

examined with respect to their uterotrophic bioactivity and ER agonist (R1) model score, 

Figure 2.15.

Figure 2.15: ER agonist (R1) AUC and uterotrophic bioactivity of non-reference 

chemicals. Non-reference chemicals are plotted according to their ER model score (ER 

agonist AUC) and color-coded based on their uterotrophic bioactivity. Chemicals with one 

report of uterotrophic bioactivity are indicated as “active” (blue) and those with one “active” 

study and one or more inactive uterotrophic outcomes are labeled “active/inactive” (green). 

Chemicals indicated as “inactive” (red) were reported to be inactive in a single uterotrophic 

study.

Of the 58 non-reference chemicals having ER agonist (R1) AUC values < 0.01 (“in vitro

inactive”), 9% (5/58) had a single report of bioactivity in a GL uterotrophic assay. Chemicals 

with potential “false negative” in vitro results are listed in Table 2.10, along with their AUC, 

LEL, and # of active/inactive citations. 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 54 of 111

Table 2.10: Chemicals with ER (agonist) AUC values <0.1 and one report of uterotrophic

bioactivity (potential false negatives)

CASRN Name AUC # Active # Inactive LEL 

(mg/kg/day)

77-06-5 Gibberellic acid 0.00 1 0 ~0.0007

50-55-5 Reserpine 0.00 1 1 3

3380-34-5 Triclosan 0.00 1 0 8

99-76-3 Methylparaben 0.008 1 1 55

52645-53-1 Permethrin 0.00 1 1 800

AUC = ER agonist (R1) AUC value, #Active = number of GL studies reporting uterotrophic 

activity, #Inactive = number of GL studies reporting no uterotrophic activity at the highest dose 

tested, LEL = lowest effect level reported for the chemical.

The translational relevance of in vitro assays is a function of their ability to measure the effects 

of chemicals, or their metabolites, at blood / tissue concentrations required for bioactivity in 

vivo. As mentioned previously, some of the possible explanations for discrepancies between 

chemicals being bioactive in the uterotrophic assay while demonstrating no activity in the 

ToxCast ER agonist assays are:

i. Metabolic conversion, which cannot be replicated by the in vitro systems, is required for 

bioactivity

ii. Weak in vivo activity may require serum concentrations outside the range assessable 

by in vitro systems,

iii. The uterotrophic response is mediated via a non-ER mechanism,

iv. Insufficient quantities/purity of test material used during the screening process and

v. For papers without external confirmation of the results, the findings may simply not be 

reproducible (the independent and orthogonal nature of the 16 ToxCast assays used to 

assess ER agonist (R1) activity greatly enhances the level of confidence in the 

reproducibility of the in vitro results).

Methylparaben (Routledgea, 1998) and permethrin (Go, 1999) have been reported as 

possessing weak in vitro ER activity at concentrations near or above 100 uM, the maximum 

concentration tested in all but one ToxCast assays, raising the possibility that the minimum 

concentration required for bioactivity is outside the testing range of the ER (agonist) AUC

model. It has also been reported that metabolites of permethrin are more ER-active than the 

parent compounds (Tange, 2014), which may further explain the observed difference in ER 

(agonist) AUC and in vivo bioactivity for this chemical.

The literature contains mixed information regarding the in vitro bioactivity of Triclosan, ranging 

from weak ER agonist activity (Gee, 2008) to a lack of ER agonist activity, but active as an ER 

antagonist (Ahn, 2008). These differences may be attributable to the metabolic capacity of the 

cell lines used, as metabolites of Triclosan (e.g., dichlorodibenzo‐p‐dioxin) may be more 

bioactive than the parent compound (Dann and Hontella 2011). The lack of bioactivity seen 

with Triclosan in the ToxCast assays is consistent with reports in the literature and may reflect 

the lack of metabolic capacity of the cell lines used. Rerserpine may also require metabolic 

transformation for bioactivity, with trimethoxybenzoic acid being one of the primary metabolites 

(Stitzel, 1976), and many phenolic acids are known to be weak ER-agonist.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 55 of 111

Both Triclosan and permethrin were found to be cytotoxic in the ToxCast assays at 

concentrations of ~10 μM, limiting the ability of the ER agonist (R1) model to assess the in 

vitro activity of these chemicals at concentrations that may be physiologically relevant. 

Another possibility is that the lack of in vitro activity at concentrations up to the limit of 

cytotoxicity may indicate that a less toxic, more ER-bioactive, metabolite is formed in vivo; 

permitting systemic exposure at concentrations that are bioactive without causing overt 

toxicity.

One study reported gibberellic acid (GA) as being highly potent in the uterotrophic assay, with 

activity similar to 17β-estradiol (Gawienowski, 1980). The results of this study are inconsistent 

with previous reports from the same author using an immature mouse model (therefore not 

considered GL) that demonstrated a lack of bioactivity at doses up to 100-fold higher and over 

a longer time frame (Gawienowski, 1977). In attempting to repeat the 1980 study, Bejar (1988) 

found that GA actually caused a decrease in uterine weight in an OVX rat model.

A more detailed description of the potential false negative chemicals is provided in Appendix 

6.

Of the 16 non-reference chemicals with ER agonist (R1) AUC values >=0.1, 75% (12/16) were 

identified as “active” in one uterotrophic study, while five chemicals were reported to be 

“inactive” for bioactivity in a single uterotrophic study report. The five chemicals associated 

with potential false positive ToxCast results are listed in Table 2.11.

Table 2.11: Chemicals with ER (agonist) AUC values >=0.1 and one report of being inactive 

for uterotrophic bioactivity (potential false positives).

CASRN Name AU

C

# 

Active

# 

Inactive

HDT 

(mg/kg/day) Model Route

1085-12-7 Heptylparaben 0.384 0 1 1000 OVX_Mouse Oral/Inj

77-09-8 Phenolphthalein 0.363 0 1 1000 OVX_Mouse Oral/Inj

500-38-9

Nordihydroguaiaretic 

acid 0.289 0 1 1000 OVX_Mouse Oral/Inj

85-68-7

Butyl benzyl 

phthalate 0.249 0 1 1000 Im_Rat Inj

135-19-3 Betanaphthol 0.122 0 1 1000 Im_Rat Inj

AUC = ER agonist (R1) AUC value, #Active = number of GL studies reporting uterotrophic 

activity, #Inactive = number of GL studies reporting no uterotrophic activity at the highest dose 

tested, HDT = highest dose tested.

The lack of concordance between the in vitro ER (agonist) AUC and uterotrophic outcomes for 

the chemicals listed in Table 2.11 is most likely related to the lack of bioavailability in vivo, 

possibly due to extensive metabolism/clearance of these compounds. The lack of in vivo

activity due to insufficient dose levels being used in the uterotrophic studies is unlikely, as the 

highest dose tested for each (1000 mg/kg/day) is the maximum recommended by both OECD 

and EPA.

Conclusions: A database of results from “guideline-like” (GL) uterotrophic studies was 

compiled to assist with the evaluation of alternative approaches to the in vivo screen for ERmediated bioactivity. Using this database, 43 chemicals having concordant outcomes in at 

least two independent studies were chosen as “reference” chemicals to assess the 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 56 of 111

performance of the ToxCast ER agonist (R1) AUC model. When using this list of reference 

chemicals (30 Active, 13 inactive), the ER agonist (R1) AUC model demonstrated a balanced 

accuracy of 95%, active predictive value of 97%, and inactive predictive value of 92%. 

Discordant results between the ER agonist (R1) AUC and uterotrophic outcomes were most 

likely due to lack of bioavailability of the chemical in vivo (“false positives”) or insufficient 

concentrations of the chemical, or its metabolite, in the ToxCast assays necessary to 

approximate blood/tissue concentrations that result in bioactivity (“false negatives”). There 

appears to be a strong relationship between in vitro AUC values and doses causing in vivo

bioactivity, raising the possibility of both qualitative and quantitative utility for the ER (agonist) 

pathway model.

These results indicate that the ToxCast ER AUC model may provide a rapid and effective 

approach for prioritizing and screening chemicals that may have bioactivity in the ER signaling 

pathway.

2.2.3: Comparison of ToxCast data and Tier 1 ER assays for List 1 chemicals

ToxCast data were compared to results of the EDSP Tier 1 battery assays for List 1 chemicals 

for which both types of data were available. The ToxCast R1 (agonist) and R2 (antagonist) 

scores were compared with results of the two in vitro and one in vivo Tier 1 assays that 

indicate the potential of a test chemical to interact with the estrogen receptor. It is worth 

acknowledging that the EDSP Tier 1 battery was designed to be more sensitive to ER agonist 

activity, as most environmental chemicals that interact with the estrogen receptor are expected 

to act as agonists. The EDSP Tier 1 guideline in vitro ER binding assay (OCSPP 890.1250) 

uses primarily ERα obtained from rat uterine cytosol and does not distinguish between 

estrogen agonist and antagonists activities. The guideline measures test chemical 

displacement of radioligand ([3H]17β-estradiol) from the ER across a range of concentrations 

in three independent runs of the assay Results of the assay are “positive” if test chemical 

displaces >50% of radioligand (and Log(IC50) is calculated), “equivocal” if test chemical 

displaces <50% but >25% of radioligand, and “negative” if test chemical displaces <25% of 

radioligand. The Tier 1 in vitro Estrogen Receptor Transcriptional Activation Assay (ERTA; 

OCSPP 890.1300) measures chemiluminescence in response to ERα-mediated increase (i.e.,

agonist activity) in luciferase gene expression. A test chemical is “positive” if the maximum 

response induced by the test chemical > 10% of the maximal positive control 17β-estradiol in 

at least two of three assay runs (i.e., RPCmax ≥10). If the test chemical fails to achieve at least 

10% of the response of the positive control, a negative response is recorded for the test 

chemical. The Tier 1 Uterotrophic Assay (OCSPP 890.1600; OECD No.440) is a short-term, in 

vivo assay designed to detect estrogen agonist activity indicated by an increase in uterine 

weight. Methodological details of the Tier 1 guideline assay are summarized in Section 2.2.1. 

Review of the List 1/ Tier 1 assay data described above indicated none of the 52 chemicals 

had consistent estrogen agonist activity in the ER binding, ERTA, or uterotrophic assays 

(Table 2.12). None of the 52 List 1 chemicals were positive for either ER binding or 

uterotrophic assays, and only 4 of 52 chemicals increased ER-mediated gene expression more 

than the 1/10 that of the positive control, indicating these chemicals have very low or no 

potential for interacting with the estrogen receptor (Table 2.12).

ToxCast result were 100% concordant with the Tier 1 in vitro ER binding, ERTA, and in vivo

uterotrophic assays, indicating List 1 chemicals had little to no estrogen receptor bioactivity. 

ToxCast estrogen agonist (R1) and antagonist (R2) AUC model scores available for 49 of the 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 57 of 111

52 EDSP List 1 chemicals were below 0.1 for both ER agonist and antagonist activity (Table 

2.12). Model scores for 11 chemicals were >0 and <0.1 for estrogen agonist (R1) and two 

chemicals had values >0 and <0.1 for estrogen antagonist (R2) activities. All of the activity for 

these weak compounds was detected at concentrations similar to those that resulted in 

cytotoxicity and may be explained by cell-stress or cytotoxicity-related false positive activity. 

Estrogen (R1) agonist bioactivity of very weak or negative reference chemicals were in a 

similar range (Table 2.12, Figure 2.16), though additional testing is required to confidently 

resolve “true negatives”.

Table 2.12: Estrogen activity of List 1 chemicals determined from ToxCast ER AUC model 

scores and EDSP Tier 1 estrogen receptor (ER) binding, estrogen receptor transactivation 

(ERTA), and uterotrophic (UT) assay results are indicated below. None of the 49 chemicals 

run in ToxCast had R1 (agonist) or R2 (antagonist) model scores greater than 0.1, considered 

the criterion for “positive” activity. Similarly, none of the chemicals had clear positive agonist 

activity in the Tier 1 ER in vitro assays (ER binding and ERTA) or in vivo (uterotrophic) assays. 

ToxCast results for three List 1 chemicals were not available (NA). Note: List 1 chemicals in 

the table below are coded because Tier 1 weight of evidence determinations for List 1 

chemicals are not final.

CODE ER AGONIST

(R1)

ER ANTAGONIST

(R2)

T1 ER

BINDING T1 ERTA T1 UT

17 0.0438 0 Equiv Neg Neg

23 0.0413 0 Neg 19.10% Neg

28 0.0324 0 Neg 18.40% Neg

22 0.027 0 Neg 15.80% Neg

50 0.0221 0 Neg OSRI (P) Neg

8 0.0203 0 Equiv Neg Neg

9 0.0152 0 Equiv OSRI (P) Neg

40 0.00985 0 OSRI (N) Neg Neg

38 0.0095 0 Neg 24.70% OSRI (N)

11 0.00646 0 Neg Incl Neg

5 0.00357 0 OSRI (N) OSRI (N) OSRI (N)

51 0 0 OSRI (N) Neg Neg

12 0 0 Neg Incl Neg

20 0 0 Neg Neg Neg

10 0 0 OSRI (N) Neg Neg

13 0 0 OSRI (N) Incl OSRI (N)

29 0 0 Neg Neg Neg

15 0 0 Neg Incl Neg

14 0 0 Neg Incl Neg

52 0 0 Neg Neg Neg

21 0 0 Neg Incl Neg

30 0 0 Neg Incl Neg

24 0 0 Equiv Incl Neg

25 0 0 Neg Incl Neg

26 0 0 OSRI (N) OSRI (N) OSRI (N)

31 0 0 Neg Neg Neg
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 58 of 111

CODE ER AGONIST

(R1)

ER ANTAGONIST

(R2)

T1 ER

BINDING T1 ERTA T1 UT

32 0 0.000077 Neg Incl Neg

27 0 0.000133 Neg Neg Neg

33 0 0 Neg Incl Neg

34 0 0 Neg Neg Neg

35 0 0 Neg Incl Neg

36 0 0 Neg Neg Neg

16 0 0 Neg OSRI (N) Neg

37 0 0 Neg Neg Neg

39 0 0 Neg Neg Neg

41 0 0 OSRI (N) OSRI (P/N) OSRI (N)

42 0 0 Equiv Neg Neg

43 0 0 Neg OSRI (P/N) Neg

4 0 0 OSRI (N) Neg OSRI (N)

44 0 0 OSRI (N) Neg OSRI (N)

45 0 0 Neg OSRI (N) Neg

46 0 0 Neg Neg Neg

47 0 0 Neg Neg Neg

19 0 0 Incl Neg Neg

17 0 0 Neg Neg Neg

7 0 0 Neg Neg Neg

48 0 0 Neg Incl Neg

49 0 0 Neg Neg Neg

6 0 0 Neg Neg OSRI (N)

3 NA NA Neg Incl Neg

2 NA NA Neg Neg Neg

1 NA NA Neg Neg Neg

2.2.4: Demonstration of computational toxicological approach to evaluating ER agonist 

bioactivity of EDSP universe chemicals

Before new computational toxicological tools are used in real-world applications, the 

application of new methods should be clearly defined and the utility should be adequately 

determined. For many test systems, this is often achieved by comparing the performance of 

the new method for selected chemicals with established activities (i.e., reference chemicals),

and comparing results of new methods with existing methods in current practice.

A variety of pesticides and environmental chemicals act as estrogen agonists and/or androgen 

antagonists (Kojima et al. 2004, Safe et al. 2001). For this reason, ToxCast R1 (estrogen 

agonist) AUC scores are examined to demonstrate the proposed approach for integrating

computational tools into the existing EDSP screening and testing framework. ER agonist (R1)

scores indicate none of the 49 (of 52) List 1 chemicals had values > 0.1, considered the 

threshold for positive bioactivity based on model scores for reference chemicals with 

recognized estrogen bioactivity (see Sections 2.2.1 and 2.2.2 for discussion; Figure 2.8). ER 

agonist scores for 56 (of 107) List 2 chemicals were in a similar range, indicating low to no ER 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 59 of 111

agonist activity (Figure 2.8). However, when R1 AUC scores are examined for the remainder 

of the ~1800 chemicals from the EDSP universe there are chemicals with estrogen agonist 

bioactivity (R1 > 0.1), many with scores similar to potent reference chemicals (Figure 2.8). 

Those chemicals with potential estrogen agonist bioactivity, considered in concert with 

estimated exposure levels (Section 8), are higher priority candidates for additional endocrine 

screening and testing.

Figure 2.16: Relative estrogen bioactivity of the List 1, List 2, and EDSP Universe chemicals. 

Estrogen receptor agonist (R1) bioactivity is plotted for the EDSP List 1 (blue), List 2 (orange), 

and 73 (4.4%) EDSP Universe chemicals (grey) with R1 scores > 0.1. The grey solid line 

depicts the agonist (R1) AUC cut-off used to identify chemicals for positive estrogen bioactivity. 

Positive (green) and negative (black) reference chemicals are included for context. Note: 

None of the List 1 or List 2 chemicals and only about 5% of the remaining chemicals tested in 

ToxCast have agonist (R1) AUC scores > 0.1. ER AUC agonist (R1) scores for Reference 

Chemicals, List 1, List 2, and other chemicals in the EDSP Universe are included in Appendix 

3. 

Analyses presented in Section 2 are meant to provide scientific support for the utility of 

ToxCast models for predicting estrogen receptor bioactivity. ToxCast high throughput assays 

measure multiple endpoints in the ER signaling pathway using different technologies and the 

network models integrate assay results into an estimate of ER agonist or antagonist bioactivity 

strength. ER agonist (R1) and antagonist (R2) AUC scores compare very well with reported 

bioactivity of reference chemicals across a range of structures and potencies (Section 2.1.1). 

The agonist (R1) AUC model also accurately predicted in vivo ER agonist activity for a 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 60 of 111

relatively large set (~150) of chemicals (Section 2.2.2). Together, these analyses provide a 

high degree of scientific confidence in the ability of ToxCast ER models to predict bioactivity 

and demonstrate the utility of using these computational tools to prioritize chemicals in the 

EDSP Universe.

In addition to a high degree of concordance with in vitro reference chemicals and in vivo peerreviewed uterotrophic studies, the agonist (R1) and antagonist (R2) AUC models agreed with 

ER-relevant Tier 1 battery endpoints (i.e., ER binding, ERTA, and uterotrophic results; Section 

2.2.3). In all cases, Tier 1 results and agonist (R1) and antagonist (R2) AUC values suggested 

very low or no endocrine receptor interactions of the 49 chemicals examined (Table 2.12). 

Given the redundancy of coverage in the 18 ToxCast ER assay, it is unlikely that running a 

single guideline ER binding and/or ERTA assay would provide additional or contradictory data. 

Additionally, the EDSP guideline uterotrophic study is a short-term in vivo assay with a single 

endpoint and may actually be less sensitive than in vitro assays (Akahori et al., 2008), 

particularly given the redundancy in the ToxCast ER assay suite (Section 2.1). The 

performance of the ToxCast models, concordance with other data streams, and agreement 

with Tier 1 assay results suggests ToxCast data may be used to substitute for the ER binding, 

ERTA, and uterotrophic endpoints in the Tier 1 battery. Analyses in this section provide a high 

degree of scientific confidence in the computational tools presented.

As previously stated, most ER bioactive chemicals are expected to have largely agonist, rather 

than antagonist, activities, and for this reason, far more data from literature studies are 

available for the potential ER in vitro and in vivo agonist response. Results of the ER agonist 

(R1) model indicate only about 5% of the 1812 chemicals run have potential ER agonist 

bioactivity. The ToxCast model is also capable of detecting estrogen antagonist reference 

chemicals with a high degree of accuracy (see Section 2.2.1), suggesting the ToxCast ER 

antagonist (R2) AUC model also performs very well. Only about 1% of the chemicals tested 

had ER antagonist bioactivity and it is worth noting that none of the List 1 or List 2 chemicals 

have threshold level ER bioactivity of either type.

Using computational tools to screen the EDSP universe would allow for rapid identification of 

chemicals with the greatest potential for endocrine bioactivity and would allow EPA to move 

away from lists of a few chemicals with relatively low or no potential endocrine activity. The 

use of computational methods to screen for estrogen receptor activity takes advantage of both 

existing and innovative, emerging technologies to implement a scientifically robust and 

comprehensive chemical prioritization process for EDSP. The prioritization of chemicals in the 

EDSP universe would greatly reduce the time, cost, and animal use required to identify 

chemicals with the greatest associated endocrine activity and would be consistent with EPA’s 

strategic plan for evaluating chemical toxicity (US EPA, 2009). EPA aims to use these tools for 

1) prioritizing chemicals in the EDSP universe, 2) potentially contributing to future weight of 

evidence determinations of a chemical’s potential to interact with the estrogen receptor, and 3)

substituting for specific endpoints in the EDSP Tier 1 screening battery.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 61 of 111

Section 3: Androgen Pathway

3.1: Computational Tools and Models: AR

In this section, we describe an approach to combine data from multiple in vitro assays 

measuring androgen receptor activity to make a prediction concerning “true” activity, in much 

the same manner demonstrated for the estrogen receptor in the previous section. Again, a key 

aspect of the analysis is the use of well-studied reference chemicals.

We probe the mammalian androgen receptor (AR) signaling pathway activity using a collection 

of nine biochemical and cell-based in vitro assays that measure mechanistic perturbations of 

the AR pathway at multiple sites: receptor binding, receptor dimerization, chromatin binding of 

the mature transcription factor, and gene transcription. This battery of in vitro assays was 

used to screen a library of 1855 chemicals. Included in the chemical library were reference AR 

agonists and antagonists, as well as selective androgen receptor modulators (SARMs). For 

the AR, we followed a process parallel to that for the ER. Chemicals were run in ToxCast f AR

in vitro assays, and a model was built to generate agonist and antagonist scores. Similar to 

the ER pathway analysis, expected patterns of assay activity include: no assays activated, all 

agonist assays activated, all antagonist assays activated, specific subsets of assays across 

technologies activated, technology-specific assay activation. To navigate this complexity, we 

developed mathematical and statistical models to infer whether chemicals that activate specific 

patterns of the in vitro assays were more likely to be AR agonists or antagonists, or signals 

were more likely to be due to specific types of assay interference. By combining data from 

multiple assays, and using a model that detects technology-specific false positives, we arrived 

at an assessment of the in vitro androgenicity of a chemical. Just as with ER, QSAR models 

are being constructed using the AR model results.

3.1.1: In vitro data and model: AR

Assays and Chemicals: The data used in model construction were generated by the EPA 

ToxCast program (Judson et al. 2010). The data set comprises concentration-response data 

on 1855 chemicals, with each chemical tested in up to nine AR pathway assays. (This is 

larger than the 1812 for the ER model because more chemicals had data from the complete 

set of AR assays.) These include 2 cell-free biochemical radioligand AR binding assays 

[Novascreen: (Knudsen et al. 2011, Sipes et al. 2013)]; a set of 2 cofactor recruitment assays 

that measure protein complementation between AR and SRC1 [Odyssey Thera]; 1 

transactivation assay measuring RNA transcript levels [Attagene: (Martin et al. 2010)]; 2 

transactivation assays measuring reporter protein level readouts [Tox21]; and 2 transactivation 

antagonist assays [Tox21]. The 1855 chemicals used in this model were selected because 

they had data for the full set of nine AR assays. The assay sources refer to the company or 

laboratory where the assays were performed. The nine assays used are listed in Table 3.1. 

The assay IDs correspond to Figure 3.1 below. Chemicals were run in concentration 

response format and analyzed via the ToxCast data analysis pipeline in the same way as the 

ER pathway assays (See Section 2.1.1 and http://epa.gov/ncct/toxcast/data.html).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 62 of 111

Table 3.1: Summary of the in vitro AR assays used with their ID mapping to the model in Figure 3.1. NVS=Novascreen; 

OT=Odyssey Thera; ATG=Attagene; Tox21=assays run by the NIH Chemical Genomics Center as part of the Federal Tox21 

program. Assay ID

Assay NameSourceGeneTargetNormalized Data TypeAssay Design TypeBiological Process TargetDetection TechnologyDetection Technology SubtypeMeasurement TimepointOrganismTissueCell FormatCell Line NameAssay Footprint

A1 NVS_NR_hAR NVS AR

percent 

activity

radioligand 

binding

receptor 

binding

Lysate-based 

radiodetection

Scintillation 

counting 20 human NA cell-free NA

96-

well

A2 NVS_NR_rAR NVS AR

percent 

activity

radioligand 

binding

receptor 

binding

Filter-based 

radiodetection

Scintillation 

counting 18 rat

prostat

e

tissuebased

cell-free NA

96-

well

A3 OT_AR_ARSRC1_0480 OT AR

percent 

activity

protein 

fragment 

complementati

on assay

protein 

stabilization

Proteinfragment 

Complementati

on

Fluorescence 

intensity 8 human kidney cell line

HEK2

93T

384-

well

A4 OT_AR_ARSRC1_0960 OT AR

percent 

activity

protein 

fragment 

complementati

on assay

protein 

stabilization

Proteinfragment 

Complementati

on

Fluorescence 

intensity 16 human kidney cell line

HEK2

93T

384-

well

A5 ATG_AR_TRANS_up ATG AR

log2 fold 

induction

mRNA 

induction

regulation of 

transcription 

factor 

activity

RT-PCR and 

Capillary 

electrophoresis

Fluorescence 

intensity 24 human liver cell line

HepG

2

24-

well

A6 Tox21_AR_BLA_Agonist_ratio Tox21 AR

percent 

activity

beta lactamase 

induction

regulation of 

gene 

expression

GAL4 blactamase 

reporter gene

Fluorescence 

intensity 24 human kidney cell line

HEK2

93T

1536-

well

A7 Tox21_AR_LUC_MDAKB2_Agonist Tox21 AR

percent 

activity

luciferase 

induction

regulation of 

gene 

expression

Luciferasecoupled ATP 

quantitation

Bioluminesce

nce 24 human breast cell line

MDAkb21536-

well

A8 Tox21_AR_BLA_Antagonist_ratio Tox21 AR

percent 

activity

beta lactamase 

induction

regulation of 

gene 

expression

GAL4 blactamase 

reporter gene

Fluorescence 

intensity 24 human kidney cell line

HEK2

93T

1536-

well

A9 Tox21_AR_LUC_MDAKB2_Antagonist Tox21 AR

percent 

activity

luciferase 

induction

regulation of 

gene 

expression

Luciferasecoupled ATP 

quantitation

Bioluminesce

nce 24 human breast cell line

MDAkb21536-

well
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 63 of 111

Reference chemicals: A set of 23 positive and negative reference chemicals was used to 

evaluate the performance of the model. These reference chemicals were identified using the 

ICCVAM final report on the evaluation of in vitro test methods for detecting potential endocrine 

disruptors (ICCVAM 2003) and the OECD draft validation report on the AR-mediated Stably 

Transfected Transcriptional Activation (AR-STTA) Assay (OECD 2010). The following list of

reference chemicals were those that had consistent in vitro results across both reports, and

were also in the ToxCast library. The reference chemicals and their predicted AR agonist and 

antagonist activities are given in Table 3.2. Chemicals designated as SARM (Selective 

Androgen Receptor Modulator) are designated to have both agonist and antagonist activity.

Table 3.2: Androgen Receptor Reference Chemicals. SARM=Selective Androgen Receptor 

Modulator.

CASRN Chemical Name AR Activity

2392-39-4 Dexamethasone Agonist

63-05-8 4-Androstenedione Agonist

521-18-6 5a-Dihydrotestosterone Agonist

58-18-4 Methyl testosterone Agonist

57-85-2 Testosterone propionate Agonist

13311-84-7 Flutamide Antagonist

140-66-9 4-tert-Octylphenol Antagonist

32809-16-8 Procymidone Antagonist

80-05-7 Bisphenol A Antagonist

50471-44-8 Vinclozolin Antagonist

72-55-9 p,p'-DDE Antagonist

52806-53-8 Hydroxyflutamide Antagonist

56-53-1 Diethylstilbestrol Antagonist

84-74-2 Di-n-butyl-phthalate Inactive

117-81-7 Diethylhexyl phthalate Inactive

1912-24-9 Atrazine Inactive

427-51-0 Cyproterone acetate SARM

50-28-2 17-β-Estradiol SARM

53-16-7 Estrone SARM

330-55-2 Linuron SARM

52-01-7 Spironolactone SARM

57-83-0 Progesterone SARM

206-44-0 Fluoranthene SARM
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 64 of 111

Structure of the Network: Figure 3.1 is a graphical representation of the network used to 

evaluate the integrated in vitro assay responses. The model was based on the series of 

molecular events that typically occur in a receptor-mediated response. The process starts with 

the interaction of a chemical with a nuclear AR (Receptor node R1). For example, an AR 

agonist will cause the receptors to dimerize (node N1), translocate to the nucleus and recruit 

co-factors to form the complete active transcription factor complex (TF) (node N2). This TF 

then binds to the chromatin DNA (node N3), initiates transcription of mRNA (node N4) and 

subsequent translation to protein (node N5). Each of these processes (with the exception of 

dimerization and DNA binding) was measured in the current collection of 9 in vitro assays 

represented as white stars. Table 3.1 provides the assay ID (A1-A9) to match the assay on 

Figure 3.1 and a brief assay description. The AR pathway is shown in two modes – agonist 

(blue) and antagonist (red). The model assumes that a chemical that interacts with the AR will 

bind in one or both of the agonist or antagonist conformations and that this will trigger activity 

in the appropriate pathway. Note that the model will allow for the prediction of mixed 

agonist/antagonist activity, or identification of SARMs (Selective Androgen Receptor 

Modulators).

As was explained in the case of the ER pathway model (Section 2.1.1), every in vitro assay is 

subject to processes that can lead to non-specific activity, independent of the AR pathway 

node that it is supposed to measure. The assay interference pathways were modeled as 

alternate “pseudo-receptors” (gray arrow nodes). Pseudo-receptors are assigned to each 

group of assays (technology group) and to each assay individually. Only a single example of 

an assay-specific pseudo-receptor is shown in Figure 3.1, but all assays (with the exception of 

A5/R5 where the assay and the pseudo-receptor are identical) have their corresponding 

pseudo-receptor.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 65 of 111

Figure 3.1: Graphical representation of the network used in the in vitro analysis of the AR 

pathway across assays and technology platforms. Colored arrow nodes represent “receptors” 

with which a chemical can directly interact. Colored circles represent intermediate biological 

processes that are not directly observable. White stars represent the assays that measure 

activity at the biological nodes. Arrows represent transfer of information. Gray arrow nodes 

are the pseudo-receptors. Each assay (with the exception of A5) has an assay-specific 

pseudo-receptor, but only a single example is explicitly shown, for assay A1.

Model Results for Reference Chemicals: The results for the AR reference chemicals in the 

agonist and antagonist modes are shown in Figure 3.2. The 5 agonist reference chemicals 

are active as expected, as are all 8 antagonist reference chemicals. There were two weak 

false-positives in the agonist mode [p,p’-DDE, AR agonist (R1) AUC =0.04 and 

4−(1,1,3,3−tetramethylbutyl)phenol, AR agonist (R1) AUC =0.02]. The SARM fluoranthene 

had a zero AUC in both AR agonist (R1) and AR antagonist (R2) mode, and was active in the 

two Odyssey Thera co-factor recruitment assays. Linuron, another SARM, was active in AR 

antagonist (R2) mode (AUC=0.04), but not in AR agonist (R1) mode. Linuron was active in the 

two cell-free binding assays and in one of the antagonist transactivation assays. There were 

no false-positives in the antagonist mode.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 66 of 111

Note that the model was not specifically designed to detect mixed, SARM-like activity. In 

particular, the AUC calculation tends to down-weight one receptor if it is overwhelmed by 

another at higher concentrations, which is what we see for most of these chemicals (See 

Figure 3.3). However, one can see clear mixed activity predicted for all of the SARMs except 

for linuron and fluoranthene.

Figure 3.2: Summary plots of agonist (top) and antagonist (bottom) activity of reference 

chemicals. Chemicals that are intended to be positive are indicated by a green circle, while 

those intended to be inactive are red. For AUC values ≤0.01, the value was set to 0.01. For 

chemicals with one or more pseudo-receptor AUC values greater than the expected one, the 

value is indicated by an X, and the receptor name is indicated.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 67 of 111

Figure 3.3: Model plots for SARM reference chemicals. The AR agonist receptor (R1) is 

designated by blue, the AR antagonist receptor (R2) by red and the other pseudo-receptors 

are colored as indicated in the legend. Concentrations are in μM. For chemicals with the 

“cytotoxicity burst” defined (2 or more cytotoxicity hits), the burst center is indicated by a 

vertical red line, and the burst region (starting 3 burst MAD below the burst center) is indicated 

by the gray shaded region. The vertical green line designates the predicted AC50

corresponding to the AR agonist (R1) or AR antagonist (R2) model with the highest AUC 

score. Mixed agonist/antagonist activity is predicted for all SARM reference compounds 

except linuron and fluoranthene.

Summary of Data for Overall Chemical Set: Of the 1855 chemicals tested in the AR pathway 

assays, 1378 were essentially inactive in all assays. As with ER, the chemical hits tend to 

cluster with the receptors / assay technologies, indicating that some fraction of the assay 

activity is false positives due to technology-specific assay interference. There were 20 

chemicals with agonist (R1) AUC≥0.1, and 93 chemicals with antagonist (R2) AUC≥0.1. A 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 68 of 111

total of 477 chemicals had at least one AUC value greater than 0.1, illustrated in Figure 3.4. 

Note that there are a large number of chemicals potent against A8 and R7. This is largely 

driven by the cytotoxicity-driven false positives in the antagonist loss-of-signal assays, which is 

more prevalent in assay A8.

Figure 3.4: Two-way hierarchical clustering of chemical activity across the 9 in vitro assays 

used to identify AR activity. The heatmap shows –log10(AC50) values for all assays and all 

chemicals with at least one assay hit. A total of 477 out of 1855 chemicals was active in one 

or more of the assays. Darker red indicates more potent activity (lower AC50), while white 

represents inactive chemical-assay pairs. Note that the assays cluster by technology / 

pseudo-receptor. The “A” and “R” values refer to the assay and receptors/pseudo-receptors 

from Figure 3.1 and Table 3.1. AR AUC plots for reference chemicals are included at 

Appendix 7.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 69 of 111

Overall, there were 20/1855 chemicals with AR agonist (R1) AUC≥0.1 and 93 chemicals with 

AR antagonist (R2) AUC≥0.1. Table 3.3 lists the 65 chemicals that are specific for one of the 

receptors, and have high Z and high Emax values. This list excludes the majority of the highAUC AR antagonists (R2), primarily because the activity is confined to the cell-stress / 

cytotoxicity region. There are a large number of these receptor-specific chemicals that only hit 

one or the other of the cell-free binding assays. One hypothesis that we are pursuing is that 

these chemicals react with the proteins or otherwise cause denaturation, which will cause 

displacement of the radioligand, leading to a binding-like signal. AR AUC agonist (R1) and 

antagonist (R2) scores are included, along with AUC plots (Appendices 3 and 8, 

respectively).

Table 3.3: AR chemicals that are specific for one of the receptors (AUC≥0.1) and have high Z 

and Emax specificity, so are therefore less likely to be confounded by cytotoxicity. However 

the chemicals specific for A8, A9 and R7 (green highlighting in the “Specific Receptor” column) 

typically only have activity in the loss-of signal antagonist assays. CASRN Name AUC Agonist (R1)

AUCAntagonist (R2)Specific Receptor

10161-33-8 17beta-Trenbolone 1.09 0 Agonist

58-18-4 17-Methyltestosterone 1.08 0 Agonist

521-18-6 5alpha-Dihydrotestosterone 1.06 0 Agonist

68-22-4 Norethindrone 1.02 0 Agonist

57-85-2 Testosterone propionate 1.01 0 Agonist

797-63-7 Norgestrel 0.97 0 Agonist

63-05-8 4-Androstene-3,17-dione 0.83 0 Agonist

50-28-2 17beta-Estradiol 0.79 0.06 Agonist

84371-65-3 Mifepristone 0.04 0.75 Antagonist

68-96-2 17alpha-Hydroxyprogesterone 0.69 0 Agonist

57-91-0 17alpha-Estradiol 0.61 0.05 Agonist

864283-48-7 GSK232420A 0.57 0.05 Agonist

57-83-0 Progesterone 0.56 0.08 Agonist

52806-53-8 Hydroxyflutamide 0 0.55 Antagonist

90357-06-5 Bicalutamide 0 0.35 Antagonist

52-01-7 Spironolactone 0.05 0.29 Antagonist

474-86-2 Equilin 0.02 0.28 Antagonist

53-03-2 Prednisone 0.27 0 Agonist

427-51-0 Cyproterone acetate 0.26 0.07 Agonist

6459-94-5 C.I. Acid Red 114 0 0.06 A2

196808-45-4 Farglitazar 0 0.06 A1

131-56-6 2,4-Dihydroxybenzophenone 0 0.06 A2

87-01-4 7-(Dimethylamino)-4-methylcoumarin 0.06 0 A2

126833-17-8 Fenhexamid 0 0.05 A1

137-26-8 Thiram 0 0.05 R7
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 70 of 111 CASRN Name AUC Agonist (R1) AUC Antagonist (R2) Specific Receptor

10540-29-1 Tamoxifen 0.02 0.05 A1

96489-71-3 Pyridaben 0 0.05 R7

126-72-7 Tris(2,3-dibromopropyl) phosphate 0 0.02 A8

119168-77-3 Tebufenpyrad 0 0 R7

83-79-4 Rotenone 0 0 R7

989-38-8 Rhodamine 6G 0 0 R7

175013-18-0 Pyraclostrobin 0 0 R7

122453-73-0 Chlorfenapyr 0 0 R7

3811-73-2 Sodium (2-pyridylthio)-N-oxide 0 0 R7

56-38-2 Parathion 0 0 A1

599-79-1 Sulfasalazine 0 0 A1

298-00-0 Methyl parathion 0 0 A1

78491-02-8 Diazolidinyl urea 0 0 A4

131-17-9 Diallyl phthalate 0 0 A2

81777-89-1 Clomazone 0 0 A1

10016-20-3 alpha-Cyclodextrin 0 0 A1

2433-14-9 4-Cyclohexylcyclohexanol 0 0 A1

149062-75-9 PharmaGSID_47259 0 0 A1

2870-32-8 C.I. Direct Yellow 12 0 0 A2

6153-64-6 Oxytetracycline dihydrate 0 0 A2

654055-01-3 Morin hydrate 0 0 A2

106-50-3 1,4-Benzenediamine 0 0 A2

28631-66-5 C.I. Acid Blue 22 0 0 A2

2610-11-9 C.I. Direct Red 81 disodium salt 0 0 A2

2467-02-9 Bisphenol F 0 0 A2

90-43-7 2-Phenylphenol 0 0 A2

68555-86-2 Acid Orange 156 0 0 A2

1118-46-3 Butyltin trichloride 0 0 A2

112809-51-5 Letrozole 0 0 A2

75-12-7 Formamide 0 0 A3

119-53-9 Benzoin 0 0 A4

32388-55-9 1-Cedr-8-en-9-ylethanone 0 0 A4

98730-04-2 Benoxacor 0 0 A8

13463-41-7 Zinc pyrithione 0 0 R7

143390-89-0 Kresoxim-methyl 0 0 R7

101463-69-8 Flufenoxuron 0 0 R7

51-28-5 2,4-Dinitrophenol 0 0 R7

220860-50-4 CP-457920 0 0 R7

81-88-9 Rhodamine B 0 0 R7

80-07-9 4,4'-Dichlorodiphenyl sulfone 0 0 R7
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 71 of 111

3.2: Interpretation and application: AR

In this section, we present various analyses and approaches for comparing ToxCast models of 

in vitro AR bioactivity with in vivo activity and evaluate the scientific confidence in using 

ToxCast data to: 1) prioritize chemicals based on bioactivity in the EDSP universe; 2)

contribute to a weight of evidence evaluation of a chemical’s potential to interact with the 

androgen receptor; and 3) substitute for specific endpoints in the EDSP Tier 1 battery. As 

discussed in earlier sections, the primary objective of EPA's EDSP is to rapidly screen 

chemicals in the EDSP universe with increased efficiency in order to protect human and 

environmental health. EPA is developing a science-based, strategic approach for EDSP 

chemical prioritization similar to the method described for the estrogen pathway. We present 

EPA’s proposed approach for implementation and application of computational methods in the 

existing EDSP screening and testing framework to rapidly prioritize chemicals for potential 

androgen receptor-mediated bioactivity. The results of AR agonist (R1) and antagonist (R2) 

model scores for in vitro reference chemicals were presented in the previous section to 

determine model performance (Section 3.2). Here we describe an effort currently underway to 

compare computational model scores with results of the Hershberger assay to determine 

model performance for predicting in vivo bioactivity. Though not as mature as analyses for the 

ER pathway, this section outlines the proposed approach for using computational tools to 

predict AR pathway bioactivity, describes the status of the current effort, and presents the 

current scientific confidence associated with incorporating computational tools for each of the 

three steps above.

3.2.1: Curated literature review of Hershberger literature

To build confidence in the ability of computational toxicology methods to prioritize and screen 

chemicals for potential androgen activity, agreement between ToxCast in vitro results and 

rodent Hershberger bioassay is being examined. The traditional short-term, in vivo screen for 

androgen activity, the Hershberger assay, was established in the 1930s and has been a 

standardized and widely used assay since the 1970s (Dorfman 1969). The Hershberger assay 

uses male rats castrated during the peripubertal stage, removing potential neuroendocrine 

influences, and determines the effect of exposures to potential androgen agonists and 

antagonists on five accessory sex organ weights: the ventral prostate, seminal vesicle (plus 

fluids and coagulating glands), levator and ani-bulbocavernosus (LABC) muscle, paired 

Cowper’s glands, and the glans penis. Run in the agonist mode, weights of accessory sex 

organs of animals exposed to two or more concentrations of test chemical are compared to 

castrated controls. Run in the antagonist mode, accessory sex organ weights of exposed to 

test chemical coadmistered with testosterone propionate, a potent androgen agonist, are 

compared to controls administered only testosterone proprionate. The Hershberger assay was 

validated by the Organization for Economic Cooperation and Development and the OECD Test 

Guideline 441 was used to develop the harmonized EPA EDSP Tier 1 Hershberger assay 

(OCSPP Guideline 890.1400).

To leverage information available from existing Hershberger assays published in peerreviewed scientific literature, EPA contracted a systematic, curated literature review. A semiautomated literature search for Hershberger studies was performed for ~1800 chemicals run in 

ToxCast androgen assays using chemical name and CASRN as search terms, with 

“Hershberger assay”, “Hershberger”, and “androgen dependent accessory sex tissue weight” 

as modifiers in the PubMed database (http://www.ncbi.nlm.nih.gov/pubmed). Relevant articles 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 72 of 111

were compiled and methodological details of each study were manually transcribed for each 

chemical/study/protocol combination. Studies were reviewed to determine methodological 

consistency with EPA OCSPP 890.1400 and OECD 441 Hershberger test guidelines. Each 

publication was independently reviewed by at least two scientists and evaluated for 

consistency with the following seven minimum criteria:

1. Animal model

a. Studies must use male rats castrated between PND 42 and PND 53, with >7 days 

allowed for post-surgery recovery before the onset of chemical exposure. Immature 

rats are not acceptable due to the lack of demonstrated sensitivity and rats should 

be Sprague-Dawley or Wistar strains, due to the later development of Fisher 344 

animals.

2. Sample size

a. Each test group and all control groups should have a minimum of 4 animals for 

positive results and 6 animals for negative results.

3. Test chemical administration

a. Test chemicals must be administered by oral gavage or subcutaneous injection.

4. Dose levels

a. The agonist portion of the experiment must include > 2 test chemical dose levels, a 

positive control group administered 0.2 to 0.4 mg/kg/d of testosterone priopionate 

(TP) by subcutaneous injection, and a negative (vehicle) control. 

b. The antagonist portion of the study must include > 3 doses of test chemical + TP, a 

positive control coadministered TP and 3 mg/kg/d of the androgen receptor 

antagonist flutamide, and negative (TP only) control. In the agonist portion of the 

study, all 5 accessory sex tissues (AST) weights should be increased in the TP 

(positive) control relative to the vehicle (negative) control. Similarly, in the 

antagonist portion, all 5 AST weights in the TP + flutamide (positive) control should 

be less than the TP (negative) control animals.

5. Duration of exposure

a. Dosing should begin between PND 49 and PND 60 and extend for a minimum of 10 

days.

6. Tissues examined

a. Absolute weights of 5 (AST) [ventral prostate (VP), seminal vesicles (SV, plus fluids

and coagulating glands), levator ani-bulbocavernosus (LABC) muscle, paired 

Cowper’s glands, and the glans penis] are examined in all animals. 

b. Tissue weights are compared to the appropriate negative control, the vehicle 

controls (agonist) and TP only control (antagonist).

7. Sample collection

a. Necropsy of study animals must be conducted within 18 to 36 hours after the last 

dose

Minimum criteria of agonist and antagonist portions of assay were considered independently. 

A score of 0 or 1 was recorded for each of the minimum criteria (MC 1-7), based on whether 

the study/protocol fulfilled the particular requirement. Each study was reviewed independently 

by two scientists, and the relevant information recorded and scored for the seven MC. Any 

papers that had differing MC evaluations between the two reviewers were flagged, and 

discussed among reviewers and with EPA until a consensus was reached. Parsing the study 

quality requirements among the different minimum criteria and scoring them independently 

affords the opportunity to relax or tighten the acceptance threshold. For example, many 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 73 of 111

studies included chemicals run at a single dose and to include (particularly positive response) 

data in the analyses, MC4 could be discounted. The expectation was chemicals tested in 

Hershberger assays with scores of 7 for both agonist and antagonist portions of the test were 

“guideline like” studies and would yield the same results for a test chemical run in the EPA 

EDSP Tier 1 guideline study.

3.2.2: Comparison of ToxCast data and Hershberger studies

The curation of data from studies identified in the systematic literature search is currently 

underway. Initial summaries supplied by contractor indicate about 40 chemicals with 

Hershberger assays meeting at least six of seven minimum criteria. These studies are being 

secondarily reviewed by EPA scientists to assure agency agreement, data quality, and 

methodological consistency with guideline studies. Though the number of Hershberger studies 

are likely to be more limited than high quality uterotrophic assays, results will be used to: 1) 

provide an in vivo comparison for AR bioactivity predicted for EDSP by ToxCast models; 2) 

determine the degree of scientific confidence in using ToxCast AR model predictions to 

contribute to the weight of evidence process; and 3) potentially substitute for specific endpoints 

in the EDSP Tier 1 battery, similar to the approach described for the ER pathway in Section 

2.2.2

3.2.3: Tier 1 AR Data

ToxCast data will be compared to results of the EDSP Tier 1 battery assays for List 1 

chemicals for which both types of data are available. The agonist (R1) and antagonist (R2) 

scores will be compared with results of the in vitro AR binding assay and in vivo Hershberger 

Tier 1 assays that indicate potential test chemical androgen receptor bioactivity. The EDSP 

Tier 1 guideline in vitro AR binding assay (OCSPP 890.1150) uses AR obtained from rat 

prostate cytosol and does not distinguish between androgen agonist and antagonists activities. 

The assay measures test chemical displacement of radioligand ([3H]R1881) from the AR 

across a range of concentrations in three independent runs. Results are reported as “positive” 

if test chemical displaces >50% of radioligand (and Log(IC50) is calculated), “equivocal” if test 

chemical displaces <50% but >25% of radioligand, and “negative” if test chemical displaces 

<25% of radioligand. Tier 1 in vivo Hershberger Assay (OCSPP 890.1400; OECD No.441) is 

designed to detect androgen agonist and antagonist activities indicated by any change in five 

accessory sex tissue weights. Tier 1 Hershberger assay methodology is summarized in 

Section 3.2.1.

Most environmental chemicals that interact with the androgen receptor are expected to act as 

antagonists or SARMs (Kojima et al. 2004, Safe et al. 2001), however, in vitro antagonist 

bioactivity (i.e., reduction in signal) may be difficult to discriminate from cell-stress or 

cytotoxicity. To be confident in the scientific performance of the AR computational models of 

bioactivity, several additional steps must be completed before AR model outputs can be 

compared with Tier 1 AR endpoints to determine if computational models of AR bioactivity are

fit for the purpose of contributing to a weight of evidence evaluation and/or substituting for 

specific Tier 1 AR endpoints. First, reference chemical potency must be established to ensure 

a range of chemicals are included and the AR models reliably detects weak positive 

bioactivities. In addition, AR model scores must be compared with Hershberger assay results 

to determine the degree of concordance between high throughput in vitro bioactivity and in vivo

response of chemicals for which both types of data are available.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 74 of 111

3.2.4: Demonstration of computational toxicological approach to evaluating AR 

antagonist bioactivity of EDSP universe chemicals

Analyses presented in Section 3 are intended to provide scientific support for using

computational tools to predict androgen receptor bioactivity. The ToxCast network model is 

built from results of nine high throughput assays measuring multiple endpoints in the AR 

signaling pathway using different technologies. AR agonist (R1) and antagonist (R2) AUC 

scores compare very well with reported bioactivity of reference chemicals across a range of 

structures (85% concordance; Section 3.1.1), providing a high degree of scientific confidence 

in the ability of ToxCast AR models. Though analyses are not as developed for the AR 

pathway as those described for the ER pathway in Section 2, we have demonstrated robust 

science supporting the use of computational tools for prioritizing chemicals for potential AR 

bioactivity. Potency has not yet been defined for the reference chemicals used in this analysis, 

many of which are pharmaceuticals designed to target the androgen receptor and are 

therefore expected to be strongly bioactive. To assure AR models detect weakly bioactive 

environmental compounds as well as strong pharmaceutical chemicals, the reference set must 

perform consistently in in vitro AR assays and include a wide range of potencies. Though 

additional reference chemicals may be added to address the lower limit of bioactivity detected 

in the AR models, the ability of the AR AUC models to detect the current set of AR reference 

chemicals with a high degree of accuracy demonstrates the utility of computational tools for 

prioritization. Chemicals in the EDSP universe with AR agonist (R1) or antagonist (R2) scores 

> 0.05 are higher priority candidates for additional endocrine screening and testing.

AR antagonist (R2) scores indicate eight of 49 List 1 chemicals (including one reference 

SARM) had values AR antagonist (R2) AUC scores > 0.05, considered the threshold for 

positive antagonist bioactivity (Figure 3.5). AR antagonist (R2) AUC scores for six of 56 List 2 

chemicals with ToxCast data were positive and included one reference antagonist (Figure 

3.5). It should be noted that only about 7.8% of chemical (114) of the ~1450 chemicals in the 

EDSP universe for which there are HTS data were active as androgen antagonists (R2 AUC >

0.05), though some of these has AR antagonist scores similar to or exceeding the most potent 

reference antagonist actives (Figure 3.5). Those chemicals with potential androgen

antagonist bioactivity, considered in concert with estimated exposure levels (Section 6), are 

higher priority candidates for additional endocrine screening and testing.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 75 of 111

Figure 3.5: AR agonist (R1) AUC scores for EDSP List 1 (blue) and List 2 (orange) chemicals 

are shown in the figure above. Chemicals with androgen antagonist bioactivity (i.e., R2 AUC 

scores > 0.0.5) in the EDSP Universe are shown above (grey). Reference antagonists (red) 

and selective androgen receptor modulators (SARMs; yellow) are included. Note: About 88% 

of chemicals had no detectable AR antagonist bioactivity (R2 = 0). AR AUC antagonist (R2) 

scores are included for reference chemicals, List 1, List 2, and other chemicals in the EDSP 

Universe (Appendix 3).

EPA is proposing use of computational tools for: 1) prioritizing chemicals for further screening 

and testing; 2) contributing to weight of evidence based on Tier 1 battery data and other 

scientifically relevant information on a chemical’s potential endocrine bioactivity; and 3)

potential substitution of these HTS assays for some of the lower throughput assays that are 

currently part of EDSP screening. Prior to use, scientific confidence for each step must be 

demonstrated in a fit-for-purpose manner. The AR pathway analyses are currently underway 

and at this time, we propose that adequate confidence has been demonstrated only for the first 

goal (prioritization). Additional resolution of performance of the AR models and potential utility 

for contributing to the weight of evidence determination of AR bioactivity will likely be improved 

by adding additional reference chemicals. In addition, ongoing comparisons of models scores 

with Tier 1 assay endpoints and in vivo Hershberger results gathered from literature studies

will determine the scientific confidence in substituting high throughput bioactivity for Tier 1 AR 

binding and Hershberger assays. AR antagonist bioactivity for List 1 and List 2 chemicals 

appears to be relatively limited, thought there are over 100 chemicals in the EDSP universe 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 76 of 111

with antagonist (R2) AUC scores greater than the 0.05 threshold value for potential antagonist 

bioactivity. Though using computational estimates of AR bioactivity to contribute to weight of 

evidence determinations or substitute for specific Tier 1 battery endpoints is premature at this 

time, given the AR AUC model performance for reference chemicals, AR bioactivity predicted 

by the ToxCast may support prioritizing chemicals with positive model scores for additional 

screening and testing.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 77 of 111

Section 4: High-Throughput Exposure (HTE)

4.1: Programmatic Objectives and Challenges for High Throughput Exposure 

Estimation

The challenge to implementing recommendations to the EDSP to incorporate exposure 

information for screening and priority setting (below) has been: 1) the lack of HTE models to 

handle the thousands of chemicals in the EDSP Universe; and 2) the lack of exposure/use 

data readily available for most of the EDSP Universe of chemicals.

 Recommendation to the EDSP Program by the EDSTAC in August 1998 “Data 

resulting from HTPS [HTS] will be combined with exposure-related information... for the 

purpose of setting priorities for T1S [Tier 1 Screening]” and 

 January 2013 FIFRA SAP “A number of approaches were identified by the Panel to 

potentially improve comparisons and prioritization efforts. HTP [HTS] assays could 

effectively be used to categorize chemicals in a tiered manner...Exposure potential could 

be incorporated into the ranking and prioritize chemicals for further testing...”

Traditional low-throughput exposure models are designed to do one or a few chemicals at a 

time, usually for a single scenario (e.g., indoor residential exposure), and require an extensive 

set of input data. Using this traditional approach is not feasible for EDSP since it would take 

decades to collect the necessary data and model exposures for the 10,000+ chemicals in the 

EDSP universe.

While pesticides may have sufficient exposure/use data to use traditional exposure estimation 

methods, they are only small fraction of the EDSP Universe. The vast majority of EDSP 

chemicals have a sparse amount exposure/use data. So, one of the challenges to the EDSP 

is to design the best modeling tools to make useful exposure estimations using the available 

data that can be collected from Web-based databases and other data mining techniques.

The EDSP believes it can produce useful HTE estimations based on the available, albeit in 

many cases sparse, exposure/use data. In keeping with ‘fit for purpose,’ the EDSP exposure 

estimations need not be as precise as traditional low-throughput exposure models, but need 

only be sufficiently precise to screen and prioritize EDSP chemicals. This material was 

covered in detail in the July 2014 FIFRA SAP review on exposure models. The complete white 

paper and all meeting materials can be found on the FIFRA SAP website 

(http://www.epa.gov/scipoly/sap/meetings/2014/072914meeting.html).

4.2: High Throughput Exposure Modeling 

In order to provide the necessary exposure science for the thousands of chemicals in the 

EDSP Universe, the EPA and National Academies have developed a long-range vision for 

exposure science in the 21st century, and a strategy for implementing this vision over the next 

twenty years (National Research Council 2007, National Research Council 2009, National 

Research Council 2011, National Research Council 2012). Within this vision, efficient 

exposure screening methods should be developed for rapid implementation, be based on 

basic chemical properties, and provide quantitative estimates for hundreds to thousands of 

chemicals (Mitchell et al. 2013, Wambaugh et al. 2013). The EPA’s ExpoCast initiative aims to 

provide these capabilities (Cohen-Hubal et al. 2010).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 78 of 111

The ExpoCast project has developed a method for generating rapid, exposure estimates using 

minimal information and estimates uncertainty using monitoring data. The methods described 

in Wambaugh et al. (Wambaugh et al. 2013) make use of the available models that are 

capable of making potential exposure predictions for thousands of chemicals in minutes. In 

order to achieve these efficiencies, the various available HTE models (Mitchell et al. 2013), 

make assumptions, particularly with respect to the granularity of space and time (Arnot et al.

2006, Rosenbaum et al. 2008). We attempt to compare exposure predictions to exposures 

inferred from monitoring data, and we do so systematically across as many chemicals (and 

chemistries) as possible to quantify the impact of these assumptions.

A key development within ExpoCast is the SEEM (Systematic Empirical Evaluation of Models) 

framework, which we have applied to exposure prioritization. In order to prioritize thousands of 

chemicals for the EDSP, we must use limited data and computational models based upon 

broad assumptions while also carefully characterizing the uncertainty in our approach. These 

important components are provided by the SEEM framework illustrated in 

Figure 4.1. SEEM is designed to integrate the predictions of multiple models and empirically 

evaluate model performance systematically, over as many chemicals as possible (Oreskes et 

al. 1994, Oreskes 1998). SEEM makes predictions of human exposure and weights 

contributing HTE model components appropriately, all with an appropriate characterization of 

uncertainty. The SEEM framework is intended to be sufficiently flexible to make use of all 

extant models capable of high-throughput exposure prediction (Mitchell et al. 2013), and 

incorporate new models as they become available. Within this framework, new and revised 

models can be evaluated for improved predictive ability, and ability to decrease uncertainty, 

while new data can be incorporated to better characterize model performance.

Ideally, “actual exposure” values would be used to evaluate the accuracy of the HTE 

predictions. Unfortunately, actual exposure events are both difficult to monitor and confounded 

by the inherent complexities of human behavior. However, as illustrated in Figure 4.2, we can 

obtain data that could be used to characterize potential sources (i.e., upstream) of exposure 

such as composition of consumer products, characterization of environmental releases, or 

monitoring data on media concentrations. We can also obtain exposure indicator data (i.e.,

downstream), particularly biomarkers of exposure (e.g., chemical concentrations in urine or 

plasma). Since the exposure event itself is difficult to observe, we develop models to either 

estimate exposure from upstream sources (forward modeling), or infer exposure from 

downstream sources (reverse modeling). Both forward and reverse modeling have 

drawbacks, and recent advances have been made to compare the results of both approaches 

to establish model effectiveness – that is, predictive ability of models for chemicals covered by 

biomonitoring data (Wambaugh et al. 2013).

One program that contributes to monitoring of environmental chemicals is the National Health 

and Nutrition Examination Survey (NHANES). NHANES is designed to assess the health and 

nutritional status of adults and children in the U.S. (http://www.cdc.goc/nchs/nhanes.htm). As 

part of NHANES, the CDC conducts ongoing assessments of collected human samples (e.g.,

blood, urine) in order to quantify the level of chemical compounds present in the average U.S. 

resident, and regularly presents findings from these studies in The National Report on Human 

Exposure to Environmental Chemicals (CDC 2011). NHANES covers a few hundred of the 

thousands of environmental chemicals and potential metabolites for which data are needed 

(Egeghy et al. 2012).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 79 of 111

Figure 4.1: Illustration of the SEEM framework applied to high-throughput exposure modeling. 

Exposures inferred from biomonitoring data (y-axis) are jointly regressed on predicted 

exposures for those chemicals using multiple HTE models (x-axis) to evaluate the predictive 

ability of the models. The regression weights allow evaluation of each exposure model (a 

weight of zero would indicate no predictive ability). The regression also serves as a calibration 

and estimation of uncertainty for extrapolating to other chemicals.

Given that human biomonitoring data are indirect estimates of “actual exposure”, the y-axis of 

the SEEM analysis in 

Figure 4.1 must be estimated using TK data associated with the chemicals of interest. The 

exposure inference was conducted under the assumption of steady-state conditions (Mage et 

al. 2004). This is a significant assumption when trying to relate human tissue and fluid sample 

chemical composition metrics to real world environmental exposures across various media 

(e.g., air, water, soil), which may be sporadic and/or seasonal, over the course of a lifetime. At 

this time, however, these assumptions are necessary for chemicals where there are no data to 

do otherwise (Egeghy et al. 2012, Mitchell et al. 2013) and these assumptions are a key 

contributor to the overall uncertainty estimated in Wambaugh et al. (Wambaugh et al. 2013). 

The benefit of these assumptions is that the evaluation and calibration of exposure model 

predictions for thousands of environmental chemicals takes a few days at most. As HTTK data 

becomes available (Section 5), it may eventually be possible to characterize the confidence in 

these inferences with respect to the drivers of variability (e.g., exposure frequency, duration, 

and chemical metabolism/excretion half-life). As finer grained chemical use information 

becomes available, it might be used jointly with basic TK data to eliminate the need for the 

steady-state assumption altogether. At this time, the steady-state assumption appears to be 

the best available for HTE methods, and is a contributor to the overall uncertainty estimated in 

Wambaugh et al. (Wambaugh et al. 2013).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 80 of 111

The NHANES data reflect aggregate human exposures across all the pathways in Figure 4.2

and any unknown pathways. Thus, while the forward prediction models may only partially 

cover the exposure pathways in play, the estimated uncertainty will reflect all exposures that 

resulted in the chemical being present in a biomonitoring sample. The HTE model calibration 

performed with SEEM framework is designed to account for pathways not currently covered by 

the HTE models. However, the impact of exposure routes not reflected by NHANES 

monitoring, including occupational exposure, cannot be assessed in this manner.

A simple (i.e., yes/no) near field indicator variable had greater predictive power than more

complicated HTE environmental exposure models (Wambaugh et al. 2013). Generally, 

chemicals above the limit of detection in NHANES had indoor/consumer product use. The 

ability of a simple heuristic – whether or not chemicals were present in the home – to forecast 

exposure in the prior study raises the question of what other simple heuristics might be used to 

categorize chemicals. Given a variety of rapidly obtained data (putative chemicals use 

categories and physico-chemical properties largely obtained from QSAR), which factors best 

explain exposures inferred from the available NHANES biomonitoring data? Further, if near 

field sources of exposure tend to be the predominate source of chemical exposures indicated 

by biomonitoring, what data and models exist to address near field exposure for the thousands 

of chemicals with limited exposure data? 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 81 of 111

Figure 4.2: A graphical representation of human exposure space in relation to exposure pathways, receptors, and monitoring 

data.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 82 of 111

In the current (second generation) SEEM analysis, a total of 19 potential predictors were

considered, comprised of production volume, 13 chemical use categories, 3 physico-chemical 

properties, and two randomly generated descriptors (Wambaugh et al. 2014). For each subset 

size and sample, best subset selection was performed using complete enumeration (Morgan 

and Tatar 1972). The average Akaike Information Criteria (AIC) (Akaike 1974) across all 

samples was used to indicate the optimal number of factors. The frequency of occurrence of 

the factors among the best subset size was used to determine the optimal model – a five-factor 

model is suggested by AIC. The five factors are “Production volume” and four modified use 

categories from ACToR: “Consumer & Industrial” use, “Pesticide Active” use, “Pesticide Inert” 

use, and “Industrial no Consumer” use, where “Industrial” indicates use in chemical/industrial 

processes. The minimal predictive subset of factors we identified in this manner were 

consistent across all demographic groups and are described as the heuristics of environmental 

chemical exposure (Table 4.1). Using the factors identified by best subsets, a Bayesian 

regression was performed to jointly infer the regression coefficients, stoichiometric 

relationships, and the parent exposures from the NHANES subject-specific urine samples for 

the total NHANES sample and each demographic subset. For this analysis, Hamiltonian 

Monte Carlo was performed via Stan (http://mc-stan.org). The calibrated model, shown for the 

total population in Figure 4.3. has an R2 of approximately 0.5.

Figure 4.3: Separate linear regressions were made of the inferred exposures on the five 

optimal descriptors for different demographic groups monitored by the CDC NHANES studies. 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 83 of 111

Here we show the correlation of inferred exposures and the calibrated model for the total 

population. The solid line indicates the 1:1 line (perfect predictor). Bayesian analysis was 

used to distribute urine products using mass-balance, giving the 95% credible intervals (light 

lines) and medians (solid circles).

With the Bayesian regression methods and the biomonitoring data, we obtained a calibrated 

model that can predict exposure for unknown chemicals, and just as importantly, we also get 

an empirical estimate of our uncertainty. Any approximations, omissions, or mistakes should 

increase the estimated uncertainty when evaluated systematically across chemicals over time. 

This uncertainty reflects contributions from at least the following four factors: (1) 

incomplete/inappropriate chemical descriptors; (2) imperfect model assumptions; (3) 

measurement uncertainty and other sampling issues for the monitoring data; and (4) imperfect 

inferences from the monitoring data. The calibrated model was then used to predict exposure 

for 7968 chemicals. Figure 4.4 shows the median and the upper limit of the 95% credible 

interval for total population geometric mean predicted exposures (mg/kg BW/day). The highest 

overall upper 95% limit for the general populations was for diethyl phthalate, a chemical 

monitored by the NHANES program. Also plotted in Figure 4.4 are the median and upper 

95% predicted exposures for children aged 6-11 (the youngest group with NHANES urine 

biomonitoring data). The exposures for children are predicted to be systematically slightly 

higher than for the total population.

Figure 4.4: Ranking of 7968 chemicals with respect to the upper 95% confidence limit on the 

predicted geometric mean exposure (mg/kg/day) for the total U.S. population (predictions for 

children aged 6-11 also included). For each chemical the lower circle indicates the median 

and the upper circle indicates the upper limit of the 95% credible interval for predicted 

exposures (mg/kg BW/day) derived from the heuristics in Table 4.1. The dotted horizontal 

lines indicate the 25%, median, and 75% quantiles for the distribution of limits of detection in 

the NHANES samples.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 84 of 111

Table 4.1: High-throughput exposure heuristics

Heuristic Description

Number of 

Chemicals

Inferred NHANES ChemicalFull Chemical Library 

ACToR 

“Consumer use & 

Chemical /

Industrial 

Process use”

Chemical substances in consumer products (e.g.,

toys, personal care products, clothes, furniture, and 

home-care products) that are also used in industrial 

manufacturing processes. Does not include food or 

pharmaceuticals.

37 683

ACToR 

“Chemical /

Industrial 

Process use with 

no Consumer 

use”

Chemical substances and products in industrial 

manufacturing processes that are not used in 

consumer products. Does not include food or 

pharmaceuticals

14 282

ACToR UseDB 

“Pesticide Inert 

use”

Secondary (i.e., non-active) ingredients in a 

pesticide which serve a purpose other than 

repelling pests. Pesticide use of these ingredients 

is known due to more stringent reporting standards 

for pesticide ingredients, but many of these 

chemicals appear to be also used in consumer 

products.

16 816

ACToR 

“Pesticide Active 

use”

Active ingredients in products designed to prevent, 

destroy, repel, or reduce pests (e.g., insect 

repellants, weed killers, and disinfectants).

76 877

TSCA IUR 2006 

Total Production 

Volume

Sum total (kg/year) of production of the chemical 

from all sites that produced the chemical in quantities of 25,000 pounds or more per year. If 

information for a chemical is not available, it is 

assumed to be produced at <25,000 pounds per 

year.

106 7785

Two complementary statistical approaches (best subsets and Bayesian regression using 

horseshoe priors) were used to determine the optimal factors, with both methods agreeing on 

the importance of “Pesticide Inert”, “Consumer & Industrial” use, and “Production volume”, but 

with the best subsets method being less certain about the “Pesticide Active” and 

“Chemical/Industrial Process with no Consumer” use categories. Thus, although the heuristics 

in Table 4.1 seem plausible in retrospect, we should not be surprised if additional data 

diminishes or eliminates the significance of some factors. Similarly, plausible heuristics not 

supported by this analysis (e.g., vapor pressure) may be significant for chemicals not included 

in the current set of monitoring data. These quantitative predictions are useful for prioritization,
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 85 of 111

but the statistical analysis here may be even more useful as a means to organize and prioritize 

the types of data needed for predicting human exposure to the thousands of chemicals in our 

environment (Oreskes et al. 1994, Oreskes 1998). For chemicals that are similar to those

known to have high measured concentrations in the urine of the U.S. population, we need 

additional exposure information, such as biomonitoring and environmental sampling. New 

biomonitoring data will also be needed for the chemicals whose properties are most different 

from those currently being studied in the NHANES and by the U.S. EPA. In this manner, 

factors such as vapor pressure and other environmental factors may be shown to be important 

exposure heuristics. Chemicals without monitoring data, but that are similar heuristically to 

those chemicals with low exposure indicated by monitoring data, become lower priorities for 

further evaluation with respect to exposure.

Note that two of the predictive factors are modified ACToR categories “Consumer use and 

Chemical/Industrial Process use” (defined as the intersection of the two ACToR categories) 

and “Chemical/Industrial Process use, no Consumer use.” The final two columns of Table 4.1

illustrate the difference between the NHANES chemicals and those chemicals that make up 

the Tox21 library. Although there are a significant number of chemicals in Tox21 that have 

similar uses to the NHANES chemicals, the largest fraction of the total (4182 out of 7968) 

chemicals have no use identified by ACToR.

With respect to the five predictive factors in the optimal model, the ten demographic groups do 

not appear to vary substantially (Figure 4.5). When the optimal model with five predictive 

factors determined for the general population is run using data from ten demographic groups, 

resultant regression coefficients are very similar across demographic groups. The distribution 

of values for the regression coefficient in Figure 4.5 indicates the trend corresponding to each 

heuristic. For all demographic groups, exposures are estimated to be higher if: (1) both 

consumer use and chemical/industrial process use is indicated; (2) pesticidal inert use is 

indicated; and (3) production volume is high. For all demographic groups, exposures are 

estimated to be lower if chemical/industrial process use without consumer use, or pesticidal 

active use, is indicated. The difference between pesticidal inerts and actives is interesting. 

Most pesticide active ingredients monitored by NHANES are found to be below the limit of 

detection in the urine of most Americans, while the chemicals indicated as pesticidal inerts 

(e.g., bisphenol a, methyl paraben) are known to be inert ingredients in other classes of 

products that do not conform to the strict ingredient reporting standards of pesticides.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 86 of 111

Figure 4.5: Regression coefficients for the model from the second generation SEEM analysis 

across different demographic groups monitored by NHANES studies (abbreviations include 

Reproductive Age Females, children aged 6-11, children aged 12-19, adults aged 20-65, 

adults 66 and older, and people with body mass indices less or greater than 30). Separate 

linear regressions were made of the inferred exposures on the five optimal descriptors for 

different demographic groups monitored by the CDC NHANES studies. The posterior 

distribution of standardized regression coefficients for the top five factors is shown. Large 

positive coefficients indicate correlation with increased exposures, while large negative 

coefficients correspond to decreased exposures. Among the demographic groups considered 

there is little variation in the significance of those factors for the chemicals we have examined.

In the absence of any other exposure data, the heuristics of exposure in Table 4.1 provide a 

reasonable basis for initial prioritization of those chemicals most in need of additional study. 

These heuristics meet the need identified by the NRC for predictive tools that will enable 

development of exposure information on thousands of chemicals that are now in widespread 

use, and enable informed prioritization of those chemicals (National Research Council 2012). 

Although the state of knowledge on human exposure is not ideal, some easily obtained 

information correlates with the exposure as inferred from biomonitoring data for 106 

environmental chemicals. For the total U.S. population, as well as various demographic 

groups, including children aged 6-11 and women of child bearing age, our high-throughput 

heuristic model is able to explain roughly half of the variance in the inferred exposures. Out of 

the thousands of other chemicals without publicly accessible biomonitoring data, those with 

properties and uses that are most like the chemicals to which people are known to be highly 

exposed are logical targets for further investigation by more traditional exposure science 

methods.

For the thousands of chemicals with no other data on human exposure, the SEEM framework 

can determine a calibrated and empirical uncertainty estimate for predicting human exposure 

and provide a means to prioritize chemicals within the EDSP universe based on exposure.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 87 of 111

Section 5: Toxicokinetics / Dosimetry

TK provides a critical bridge between the concentrations that show bioactivity in the in vitro

assays and the administered doses needed to produce those concentrations at the tissue 

level. Unfortunately, for many of the compounds under consideration by the EDSP, the in vivo

TK data necessary to translate from the administered concentration in vitro (i.e., μM) to real 

world exposures (i.e., mg/kg BW/day) is not available, as is shown in Figure 5.1. Out of 239 

chemicals with in vitro bioactivity examined in the Wetmore et al. (2012) study, only 11 

chemicals had in vivo TK data. Thus, alternative, higher throughput methods are needed as 

an alternative to traditional TK model development with in vivo data. This section summarizes 

work on HTTK, which can be combined with HTS data. This material was covered in detail in 

the July 2014 FIFRA SAP review on exposure models. The complete white paper and all 

meeting materials can be found on the SAP website 

(http://www.epa.gov/scipoly/sap/meetings/2014/072914meeting.html).

Figure 5.1: Availability of in vitro bioactivity data, in vivo TK, and traditional exposure 

predictions. As in Egeghy et al. (Egeghy, Judson et al. 2012), there is a paucity of data for 

providing context to HTS bioactivity data.

For pharmaceutical compounds, successful methods have been developed for determining 

some aspects of TK (e.g., hepatic metabolic clearance and steady-state behavior) from limited 

in vitro methods and chemical structure-derived property predictions (Jamei et al. 2009, 

Lukacova et al. 2009, Jamei et al. 2013). In drug development HTTK methods allow an initial 

determination of the therapeutic doses for clinical studies. This approach is considered 

effective for pharmaceutical compounds and it is claimed that the predicted concentrations are 

typically on the order of the actual, in vivo concentrations (Wang 2010). Without the time and 

resources to generate in vivo TK data for the thousands of chemicals that require TK data, 

these HTTK methods for pharmaceuticals may be useful for environmental chemicals 

(Wetmore et al. 2012, Thomas et al. 2013).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 88 of 111

HTTK methods can estimate the dose (mg/kg BW/day) needed to produce steady-state 

plasma concentrations (Css, in units of μM) equivalent to the activation concentrations 

observed in vitro (this is RTK) (Rotroff et al. 2010; Wetmore et al. 2012; Wetmore et al. 2013). 

As is shown in Figure 1.2, considering the dose needed to produce a putative bioactive 

plasma concentration results in a much greater discrimination between the potencies of 

various compounds. In Figure 1.2, 239 chemicals investigated by Wetmore et al. (Wetmore et 

al. 2012) are compared both in terms of 50% activation concentration (AC50, in units of μM) for 

a bioactive assay (Figure 1.2A) and in terms of the oral equivalent dose (mg/kg BW/day) 

predicted to be necessary to cause that concentration in an adult human. Each plot point 

indicates an in vitro assay for which a systematic response was observed; some, more 

promiscuous chemicals that had activity in numerous assays may have many points, while 

other chemicals may have very few points plotted.

The HTTK model used for the EDSP chemicals, is relatively simple, making three key 

assumptions: (1) that the whole body is at the same concentration (i.e., plasma); (2) that the 

environmental exposure is constant and uniform (i.e., constant infusion), and (3) that enough 

time has passed that the plasma concentration is at steady-state with respect to the 

environment (Wetmore et al. 2012). We use a “three-compartment” model (Figure 5.2) to 

predict plasma concentrations as the result of a constant environmental exposure. We did not 

determine tissue partitioning, since at steady state we assume that the free concentration is in 

all tissues. This simple model featured elimination due to metabolism of the parent compound 

in the liver and renal elimination by passive glomerular filtration, adjusted for plasma protein 

binding (i.e., the product of glomerular filtration rate and fup). We modeled hepatic metabolism 

using the “well-stirred” approximation (Wilkinson and Shand 1975, Houston and Carlile 1997)

from the flow into the liver and the in vitro measured values for fraction unbound intrinsic 

hepatic clearance and the density of hepatocytes in liver (1.1 x 108 hepatocytes per gram liver 

and a liver density of 1.05 g/mL).

Figure 5.2: The general, three compartment TK model, used by SimCYP for RTK (Wetmore, 

Wambaugh et al. 2012).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 89 of 111

An analytic solution for plasma concentration (Equation 7.1) was derived from the differential 

equations describing the three-compartment model by assuming constant infusion dosing. 

This analytic solution is the equation used for steady state plasma concentrations (Css) in 

previous work (Wilkinson and Shand 1975, Rotroff et al. 2010, Wetmore et al. 2012, Wetmore

et al. 2013).

 























l u b int

int

u b l u b Q F *

GFR*F Q *F *

oral dose rate

Cl

Cl

Cs s

Equation (7.1)

Two types of in vitro data were used to parameterize the HTTK models: (1) CLint, the intrinsic 

metabolic clearance of the parent compound by primary hepatocytes (substrate depletion 

approach) in wells of a multi-compound plate (Shibata et al. 2002); and (2) fup, the fraction of 

chemical unbound in the presence of plasma protein as assessed using rapid equilibrium 

dialysis (RED) in which two wells are separated by a membrane that is permeable by smaller 

molecules, but prevents the plasma protein added to one well from migrating to the other well

(the relative chemical concentration in the two linked wells gives the free fraction of chemical) 

(Waters, Jones et al. 2008). For environmental chemicals, data on these two parameters were 

obtained primarily from Wetmore et al. (2012) and Tonnelier et al. (2012).

Once a chemical specific Css is predicted for a steady-state 1 mg/kg BW/day dose, we can use 

this Css value to perform RTK. We rely on the linearity of the simple TK model we are using –

a dose X times greater than 1 mg/kg BW/day will produce a steady-state concentration of X



Css, as in Figure 5.3A. Specifying the dose and predicting the Css is “forward” TK. RTK 

involves flipping the axes of Figure 5.3A to get Figure 5.3B. We take the AC50 from an in vitro

assay and assume that the steady-state plasma concentration must be equivalent to the AC50

in order to cause activity in vivo. We then use the linear Css prediction to infer a dose that 

would cause that concentration to occur (Figure 5.3B).

Monte Carlo (MC) was performed to simulate human variability in key physiological and TK 

parameters, as is illustrated in Figure 5.2. Log-normal distributions, were used with mean 

values drawn from the references above and a coefficient of variation of 0.3. Body weight, 

liver volume, blood flow to the liver, and CLint were all varied in this manner. The fup was drawn 

from a censored distribution with identical properties to the other distributions if the mean was 

above the limit of detection (assumed to be 1% unbound) and a uniform distribution between 

0% and 1% unbound otherwise. For each chemical, 1000 different combinations of 

parameters were used to determine Css, allowing estimation of the 5%, median, and 95% 

quantiles.

The 5% quantile from the MC simulation represents adult humans who would get lower plasma 

concentrations from the same fixed dose; for example, this could be due to higher than 

average expression of a specific metabolizing enzyme resulting in rapid clearance from the 

body. Conversely, the 95% quantile from the MC simulation represents adult humans who 

would get higher plasma concentrations – these are the more sensitive individuals. As is 

shown in Figure 5.3B, if the predicted Css is higher, then the daily dose predicted as 

necessary to achieve that concentration in plasma equal to the lowest bioactive concentration 

is predicted to be lower. Although MC simulation is used to characterize some aspects of 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 90 of 111

human variability (e.g., body weight), the assumptions we made of how to model these 

variations can influence the results of the predictions, especially for the extremes (e.g., the 

highest 95th percentile) (McLanahan et al. 2012). Further, any key determinants of variability 

that are not included in our simulation cannot be assessed for impact.

Figure 5.3: The Steady-State TK model used for Reverse Toxicokinetics (RTK) is linear. (A) 

We can calculate the predicted steady-state concentration (Css) for a 1 mg/kg/day dose and 

multiply to get concentrations for other doses. The “reverse” part of RTK is shown in (B) in 

which we swap the axes from (A) so that we can divide bioactive concentrations by the Css for 

a 1 mg/kg/day dose to get oral equivalent dose in mg/kg BW/day. We use Monte Carlo (MC) 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 91 of 111

methods to simulate human variability to determine the range of Css produced for the same 

exposure across the simulated population. The higher the predicted Css, the lower the oral 

equivalent dose, so the upper 95% predicted Css from the MC has a lower oral equivalent 

dose.

Figure 5.4: Monte Carlo (MC) Approach to Variability

We recognize that the HTTK methods we are using do not include prediction of important 

aspects of TK such as bioavailability and absorption rate, but we can make conservative 

assumptions for these values and test these assumptions against in vivo data. There are 

many ways in which HTTK models in general might omit other key drivers of TK, including: 

extra hepatic metabolism, secretion in bile/enterohepatic recirculation, unusual binding affinity 

for one tissue, substrates for transporters in renal, hepatic, and other tissues, and poor 

absorption (Rotroff et al. 2010). Until in vitro assays are developed for these processes, 

QSAR provides the best way forward. Unfortunately, QSAR is not yet capable of providing 

rate constants; rather it allows estimation of how likely or unlikely a process is to occur.

We have shown that provisional HTTK models can be developed from in vitro data for many 

environmental chemicals to allow comparison of HTS bioactivity data and HTS exposure 

predictions. While we expect that these methods will be refined and replaced as greater 

knowledge becomes available, we have provided the most informed methodology given the 

data available today.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 92 of 111

Section 6: Integrated Bioactivity Exposure Ranking (IBER) for 

Prioritization

Previous sections have described methods to quantitatively predict concentrations at which 

endocrine-pathway activity can occur; how to convert these concentrations to oral equivalent 

doses using HTTK methods; and how to estimate exposures for chemicals. In this section we 

describe how to put these pieces together to determine if a chemical has the potential to 

activate an endocrine-related pathway at exposures / doses that are likely to occur. We term 

this the Integrated Bioactivity-Exposure Ranking (IBER). We operate under the assumption 

that it is more important to perform follow-up testing on a chemical that has an overlap 

between expected exposure and bioactivity than one that does not. This is the basis of using 

all of these approaches for prioritization of chemicals for EDSP Tier 1 testing. This basic 

approach is illustrated in Figure 6.1. For each chemical of interest, we evaluate pathway 

activity in vitro, convert in vitro concentrations to in vivo doses using HTTK and then compare 

with exposure. This figure highlights another key aspect of this comparison, namely that each 

of these quantities (quantitative pathway activity, pharmacokinetic parameters and exposure 

estimates) are subject to uncertainty and variability, which leads to distributions in all of the 

relevant quantities.

Figure 6.1: Chemical prioritization can be conducted by using HTTK methods to translate 

high-throughput bioactivity data into putative human doses of concern (mg/kg BW/day) and 

then comparing those doses to HTE predictions.

6.1: Basic Approach for IBER

The basic approach for calculating the pathway-activity dose distribution was developed by 

Judson et al. (Judson et al. 2011). One first estimate a pathway-wise estimate of 

concentration at which the pathway is activated, uses HTTK to convert from concentration to 

dose, and then compare this dose against exposure. Each of the quantities activity 

concentration, concentration-to-dose conversion and exposure estimate, is subject to 

uncertainty and population variability. After combining point estimates and uncertainty and 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 93 of 111

variability ranges, the IBER (Integrated Bioactivity-Exposure Ranking) is the ratio between the 

lowest X-percentile (e.g. 5%) of the bioactivity dose range and the upper Y-percentile (e.g. 

95%) of the expected exposure range. The IBER is one proposed method for prioritization of 

limited testing resources for chemicals. We will illustrate this approach with data from 182 

ToxCast chemicals with in vitro bioactivity data, HTTK data, and exposure estimates. Of 

these, 40% have an IBER >100, and 25% have an IBER > 1000. We next provide somewhat 

more detail of the IBER approach.

1. The approach is built around biological pathways whose structure is derived from a 

large body of in vitro and in vivo studies. For EDSP, we focus on the estrogen and 

androgen receptor pathways already described.

2. For each pathway, we select a representative set of targets to probe, and develop in 

vitro assays to measure effects related to those targets. For human risk assessment, 

we would primarily focus on human targets and cells, although assays derived from 

other species that share significant sequence homology at the specific target gene can 

also be used. Assays can be as simple as binding to a single protein, or as complex as 

whole genome microarrays or other genome-scale measurements. Sections 2.1 and 

3.1 demonstrated the use of in vitro assays for this purpose.

3. All assays must be run in concentration-response format in order to yield values for the 

Biological Pathway Altering Concentration (BPAC).

4. Usually, it will be necessary to run several assays associated with a pathway and use a 

systems-level model to integrate the resulting pharmacodynamic (PD) data for 

estimating BPAC. The ER and AR models described above are specific examples of 

integrating data from multiple assays across a pathway.

5. We need to estimate PD-related uncertainties and population variability and must 

characterize the population distribution of the BPAC and its uncertainty using probability 

distributions. We have proposed a default approach to estimate uncertainty and 

variability which is similar to that used in in vivo studies. Developing better estimates of 

these quantities is a pressing research need.

6. HTTK modeling is then used to estimate the external dose that would lead to the 

internal BPAC (dose-to-concentration scaling function). As described above, the HTTK 

methods we use incorporate estimates of population variability. This dose is termed the 

Biological Pathway Altering Dose (BPAD). Note that the derived BPAD is a distribution 

and not a single value.

7. The BPAD and the exposure distributions are then compared as in Figure 6.1.

The mechanics of this process are straightforward, but appropriate treatment of uncertainty 

and variability is a challenge and a topic of ongoing research.

6.2: Modeling uncertainty and variability

For each of the components of IBER (PD, PK, Exposure), there are uncertainty and variability 

components. Table 6.1 summarizes the approaches we are currently taking for these. 

However, this is an area requiring significant new development.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 94 of 111

Table 6.1: Summary of approaches to uncertainty and variability in IBER

Uncertainty Variability

Pharmacodynamics (BPAC) Default.

(Judson et al. 2011)

Default.

(Judson et al. 2011)

Pharmacokinetics Future Research Direction

Chemical-specific, using 

Monte Carlo sampling of 

modeled subjects with 

different metabolic profiles 

(Section 5)

Exposure

Use of uncertainty in 

biomonitoring data; in 

production volume and 

other parameters. 

Future Research Direction

6.3: Interpretation and application: IBER

Over the past several years, EPA has initiated a step-wise scientific review of the status of 

high throughput tools and defined several potential uses for computational models in the EDSP 

program. A subset of in vitro assay and in silico methods were reviewed with the intended 

purpose of prioritizing chemicals based on ER bioactivity (Jan 2013 SAP). More recently, the 

agency requested scientific input on new high throughput exposure models and described the 

potential utility for estimating near-field exposures for thousands of chemicals (July 2014 SAP). 

In addition to these SAP reviews, numerous articles on high throughput in vitro assays (e.g.,

Reif et al. 2010, Kavlock et al. 2012, Rotroff et al. 2013, Judson et al. 2014), toxicokinetics 

(Rotroff et al. 2010, Wetmore et al. 2013, Wetmore et al. 2014), and exposure modeling

(Wetmore et al. 2011, Wambaugh et al. 2013, Wambaugh et al. 2014) have appeared in peerreviewed publications, broadly engaging the scientific community on the state of the science 

and potential use by EPA.

Rather than summarizing or reconsidering the research that has been presented previously, 

the aim of this document and subsequent SAP meeting is to request scientific input on EPA’s 

proposed method for applying computational tools to:

1. Prioritize the EDSP chemical universe;

2. Contribute to the weight of evidence determination on a chemical’s potential endocrine

bioactivity, and

3. Substitute for specific endpoints in the EDSP Tier 1 screening battery.

We describe the approach for integrating computational tools (Figure 6.2) to rank high priority 

chemicals for additional screening and testing, and demonstrate the scientific confidence and 

robustness of the various models used in EPA’s proposed approach using various data sets 

with the expectation that the three goals described for integrating computational tools into the 

EDSP framework require differing degrees of scientific confidence (i.e., “fit for purpose”).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 95 of 111

Figure 6.2: The flow-chart illustrating EPA’s proposed Integrated Bioactivity-Exposure 

Ranking (IBER) approach to integrating computational tools in the existing EDSP framework. 

Bioactivity HTTK = high throughput toxicokinetics.

Both the technology and the interpretation of high throughput methods have advanced greatly 

and EPA is now preparing to use these innovative methods to prioritize chemicals. The IBER 

is informed by computational models described in previous sections of this document. For 

chemicals in the EDSP universe, high throughput bioactivity estimates from the ToxCast 

models (Sections 2 and 3; Appendix 3) are used to calculate pseudo AC50 values and high 

throughput toxicokinetic models are then used to extrapolate in vivo estimates of bioactive 

concentrations from in vitro assay data (Section 5; Appendix 3). Human oral equivalent doses 

are then paired with exposure estimated from the SEEM framework (Section 4; Appendix 3). 

Plotted together, these data indicate chemicals for which activity and exposure may overlap or 

be close enough to prioritize for closer study (Figures 6.3 and 6.4).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 96 of 111

Figure 6.3: Plots of high throughput predictions of estrogen receptor (ER) bioactivity, 

converted to oral dose equivalents (Section 5) and high throughput predictions of 

environmental exposures (Section 4) for chemicals for which all data are available. Note: 

Graphs are intended to illustrate the approach for Integrated Bioactivity-Exposure Ranking and 

not all chemicals shown above are in the EDSP chemical universe. 

Figure 6.3: Plots of high throughput predictions of androgen receptor (AR) bioactivity, 

converted to oral dose equivalents (Section 5) and high throughput predictions of 

environmental exposures (Section 4) for chemicals for which all data are available. Note: 

Graphs are intended to illustrate the approach for Integrated Bioactivity-Exposure Ranking and 

not all chemicals shown above are in the EDSP chemical universe.

The extrapolated in vivo bioactive dose and the exposure estimate are then used rank 

chemicals (Figure 6.5; Appendix 3). The lowest IBER values are chemicals for which, the 

upper 95% confidence intervals of exposure estimates come closest to or overlap with (e.g.,

values < 1) the lower 95% confidence limits for doses estimated to be bioactive and these 

chemicals warrant highest priority for additional EDSP screening and testing. The proposed 

approach is illustrated for both estrogen receptor (ER IBER) and androgen receptor 

bioactivities (AR IBER). For each endocrine pathway, the greater of the two agonist (R1) or 

antagonist (R2) bioactivity score is used in rankings. Examining both bioactivity and exposure 

data illustrates that considering prioritizing environmental chemicals with the highest potential 

bioactivities may not be sufficiently protective for chemical which may be less potent but also 

have high estimated exposures. For example, two List 1 chemicals two List 1 chemicals have 

ER IBER rankings < 1000, despite having very low ER bioactivities (Figure 6.5). Highest

priority would be the six chemicals in the EDSP universe with ER IBER values and three AR 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 97 of 111

IBER values below 1, indicating estimated exposures exceed potential bioactive 

concentrations (Figure 6.5).

Figure 6.5: Integrated Bioactivity-Exposure Rankings (IBER) for estimated high throughput 

estrogen receptor (ER) and androgen receptor (AR) bioactivity of List 1, List 2, and the 

remaining chemicals for which all types of high throughput data are available. A ranking of 1 

indicates estimated bioactivity and exposure occur at the same concentrations (see Appendix 

3 for data).

Though there are recognized sources of uncertainty around bioactivity, toxicokinetic, and 

exposure model estimates, these data may still be useful to prioritize chemicals for additional 

endocrine screening and testing. For the thousands of industrial chemicals for which other 

sources of bioactivity and exposure information do not exist, this approach represents a 

method to begin to address potential endocrine effects. If the ER and AR AUC models are 

scientifically robust for detecting the endocrine bioactivity, as illustrated by sensitivity and 

specificity of model predictions when compared to in vitro reference chemicals, computational 

data also may be used, along with available Tier 1 (or “Tier 1-like”) data and other scientifically 

relevant information, to contribute to EDSP weight of evidence determination for chemicals. 

For some endocrine pathway activities, model performance may be sufficiently strong to 

consider using model scores as substitutes for specific Tier 1 endpoints. For example, 

agreement of ER agonist (R1) AUC model predictions with both Tier 1 in vitro assays and in 

vivo uterotrophic response described in Section 2 may be sufficiently robust that no additional 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 98 of 111

information would be gained by requiring Tier 1 ER binding, ERTA, or uterotrophic guideline 

studies for some chemicals. Further still, positive predictions from computational models when 

coupled with indications of in vivo positive endocrine response from other data sources (e.g.,

Part 158 studies, high-quality research in peer-reviewed literature) may identify a need for 

further testing.

To date, reviewing EDSP chemicals for potential bioactivity has proven to be a complex, 

costly, and time consuming endeavor. These limitations are compounded by the chemicals on 

EDSP List 1 and List 2, which are probably not the most endocrine bioactive chemicals in the 

EDSP universe. As an alternative, we have described an approach using computational tools 

to prioritize and screen EDSP chemicals and rapidly identify those with the greatest potential 

endocrine bioactivity, and moving away from lists of chemicals with relatively low or no 

potential endocrine activity. The IBER approach for prioritization considers both the biological 

activity of compounds and the potential environmental exposure. This approach takes 

advantage of both existing and innovative, emerging technologies to implement a scientifically 

robust and comprehensive chemical prioritization process for EDSP. The prioritization of 

chemicals in the EDSP universe would greatly reduce the time, cost, and animal use required 

to identify chemicals with the greatest associated endocrine activity and would be consistent 

with EPA’s strategic plan for evaluating chemical toxicity 

(http://www.epa.gov/spc/toxicitytesting/docs/toxtest_strategy_032309.pdf).
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 99 of 111

Section 7: Future directions

7.1: Computational tools, models and data

The computational tools described in earlier sections of this document focus on estrogen and 

androgen receptor-mediated bioactivity and rely on simple pharmacokinetics and second 

generation high throughput exposure models in a proposed approach for prioritizing chemicals 

for screening and testing. Significant ongoing research is underway to include non-receptor 

mediated endocrine effects, to include thyroid bioactivity, and to improve the accuracy of high 

throughput bioactivity, pharmacokinetics, and exposure estimates. The outputs of the 

computational models may change as the models continue to be refined and ongoing will be 

presented at future SAP meetings. Current and planned efforts include:

 Modeling the thyroid pathway: EPA/ORD has a large effort in generating HTS data around 

targets in the thyroid pathway, and building models to understand the consequences of 

perturbing these targets. Particular targets are: thyroid hormone receptor; thyroid releasing 

hormone; sodium-iodide symporter; multiple dioidinases; and thyroid peroxidase. Results 

of these models will help parameterize biologically-based dose-response models of the 

pregnant female. Models are being parameterized for both human and rat. Cell-level 

models of the developing brain are being incorporated into the BBDR model.

 Modeling the steroidogenesis pathway: We are generating high-throughput data using a 

modified variant of the EDSP Tier 1 T47D guideline assay that generates concentrationresponse profiles on almost all of the intermediate and final metabolites in this pathway. 

Raw data for the first 1000 chemicals is in house. A whole-pathway model is being 

developed to better predict where in this pathway a chemical is causing disruption.

 Metabolically-competent assays: One criticism of much in vitro toxicology is that the 

assays are typically not metabolically competent, so one can only measure the effect of the 

parent compound and not the complete set of metabolites. We are in discussion with a 

number of groups that have metabolically-competent cell systems that could be used in 

endocrine-related assays. We performed preliminary studies on the ToxCast chemicals 

using pretreatment with an S9 fraction followed by a protein complementation assays used 

in the current ER model. The results were promising in that many compounds showed 

decreased activity, while the 2 that showed increased potency were known to require 

metabolic activation (methoxychlor and benzo(a)pyrene). However, the assay was 

relatively insensitive in that only activity from very potent compounds could be seen. 

Further experiments of this type are being planned.

 Testing further chemicals: We are currently running more chemicals through the ER and 

AR assays already described. Most of these are in the EDSP universe. We are also 

running more chemicals through the HTTK assays (plasma protein binding and intrinsic 

clearance) to allow for more quantitative comparison with exposure estimates. These are 

being selected based on both inclusion in the EDSP universe and activity against ER and 

AR.

 Modeling of stress pathways: As we have shown, cell stress and cytotoxicity can cause 

non-receptor mediated activity in all of the HTS assays used, and can confound our ability 

to tell true from false positives. We are building structure-based models to predict the type 

of cell-stress process that chemicals are likely causing at high concentrations, with the aim 

of better understanding this cause of assay interference.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 100 of 111

 QSAR models: We have briefly described a crowd-sourced ER activity modeling project

(CERAPP). We are planning to carry out a parallel study for AR and other endocrinerelated targets as the HTS data becomes available.

 HTTK models: Improvements in the models, and QSAR and measurement of parameters 

are ongoing.

 Exposure models: The SEEM framework will continue be refined and calibrated for 

predicting human exposures. Models providing more detail on sources of human exposure 

(e.g., ground water and drinking water), as well as exposure models for ecological species 

(e.g., fish) are also being developed.

7.2: Data Release and Transparency: The EDSP21 Dashboard. 129

The EPA is committed to make public all of the data and models being used in the EDSP21 

prioritization effort described in this document. A primary route for making the HTS, exposure 

and dosimetry data public is through the EDSP21 dashboard. This web portal allows users to 

browse results by chemical or by output type. The dashboard will be available at URL:

http://actor.epa.gov/dashboard/edsp21. Figure 7.1 shows an example of assay concentrationresponse data for Bisphenol A. All data will be available for browsing through the dashboard 

and will be released as a database export and as a series of flat files (e.g. Excel worksheet).

In addition, all software used in these analyses are publically available. This public release 

policy applies to all data and models generated under EPA’s ToxCast and ExpoCast projects, 

and is not limited to the EDSP chemical universe.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 101 of 111

Figure 7.1: Screenshot of the EDSP21 dashboard showing sample concentration-response 

data for Bisphenol A.

7.3: EDSP Future Directions

In this document, we have presented a proposed Integrated Bioactivity Exposure Ranking to 

identify EDSP chemicals. We have demonstrated that the ToxCast models of estrogen and 

androgen receptor bioactivity are capable of detecting in vitro (E and A) and in vivo (E, A in 

progress) reference chemical with a high degree of sensitivity and specificity. Coupled with 

high throughput exposure predictions, these computational tools help to identify chemicals with 

the least separation between bioactive concentration and potential environmental exposures. 

With appropriate scientific confidence established at each level, these data may be useful for:

1. Prioritization of chemicals for additional EDSP screening and testing,

2. Contributing, along with other data, to the weight of evidence determination of a 

chemical’s potential for endocrine effects, and

3. Substituting for specific EDSP Tier 1 battery endpoints.

Together, these computational tools allow EPA to move away from resource-intensive 

examinations of relatively low activity chemicals and increase efficiency of endocrine screening 

and testing.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 102 of 111

For the current FIFRA SAP review, EPA has evaluated the performance of AUC models of ER 

bioactivity based on results for 18 HTS assays for the estrogen pathway, models of AR activity 

based on results of 9 HTS assays for the A pathway, and proposed an approach for integrating 

bioactivity with high throughput exposure predictions using HTTK/RTK conversion of in vitro

active concentration to human oral equivalent doses (i.e. Figure 6.2). Analyses presented in 

previous sections of this document demonstrate the scientific foundation of the computational 

tools used in the IBER approach. In addition, EPA believes the performance of the ER AUC 

models when compared with reference chemicals (Section 2.1), Tier 1 in vitro data (Section 

2.2.3), and in vivo published and Tier 1 uterotrophic studies (Sections 2.2.2 and 2.2.3) is 

sufficiently concordant to:

1. Prioritize chemicals for additional screening and testing based on potential ER 

bioactivity;

2. Contribute to weight of evidence decisions on the potential of a chemical to interact with 

the estrogen receptor; and

3. Suitably substitute for EDSP Tier 1 ER binding, ERTA, and uterotrophic endpoints.

The development of computational tools designed to predict ER interactions benefited greatly 

from the January 2013 SAP on the topic, and as such, the computational models for ER 

interactions are more mature than the parallel effort for androgen receptor-mediated 

bioactivity. None the less, the AUC AR models performed very well when compared with in 

vitro reference chemicals (Section 3.3.1). To further develop the AR model, a measure of in 

vitro potency must be established for the AR reference chemicals and weak positive chemicals 

must be included in the reference set to be sure the high throughput AR model detects weakly 

bioactive compounds. None the less, model performance of the current reference chemical 

set, which is far more extensive than those typically used to validate lower throughput in vitro

assays, provides strong confidence that predicted AR bioactivity is “true”. Based on analyses 

presented in Section 3, data support the assertion that chemicals with AR model scores > 0.05 

are potentially bioactive and the model can be used to prioritize chemicals for potential 

androgen bioactivity.

Computational tools for detecting AR interactions will continue to be developed and shaped by 

feedback from this SAP review. As reference chemicals spanning a wide range of potencies 

are run in the model and detected with a high degree of sensitivity and specificity, AR agonist 

and antagonist model scores can contribute to future weight of evidence determinations of 

androgen pathway bioactivity. Comparison of AR model predictions with in vivo Hershberger 

response is currently underway, but until analyses are completed and thoroughly reviewed, AR 

model scores will not proposed as substitutes for EDSP Tier 1 battery endpoints in the 

androgen pathway.

It is important to acknowledge that the focus of this white paper is restricted to estrogen and 

androgen receptor-mediated interactions. The computational models of ER bioactivity 

compare very well with lower throughput in vitro and in vivo responses (Section 2); however, in 

vivo analyses presented in this paper are restricted to ER and AR effects in animals 

interrupted hypothalamic pituitary gonadal axes (i.e., uterotrophic and Hershberger assays) 

and exclude the potentially confounding influence of a functional neuroendocrine system. We 

have not presented similar analyses of thyroid bioactivity, though examination of thyroid 

pathway interactions are also in the purview of the EDSP. Similar analyses of thyroid receptor 

(TR) bioactivity, estimated from the four HTS assays included in ToxCast, are underway, 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 103 of 111

though environmental chemicals are expected to affect thyroid homeostasis primarily by 

mechanisms other than receptor binding. As such, computational models designed to detect 

thyroid-active chemicals will include non-receptor mediated effects.

In addition to expanding analyses to examine non-receptor mediated effects, EPA is 

continually developing and improving HTTK and HTE models for the EDSP IBER application. 

In this paper, HTTK in vitro-to-in vivo extrapolation was limited to human oral equivalent doses, 

however, current models exist for in vitro extrapolation to rodent and fish. HTTK models for all 

taxa will continue to be refined. Future high throughput exposure models will simulate multiple 

exposure pathways for humans and the existing framework will be expanded to model 

ecological exposures, initially using water concentration data as a proof of concept.

The EDSP’s future directions for computational tool development is in a direct effort to meet 

goals outlined in the September 2011 EDSP21 Work Plan, specifically:

1. Ranking chemicals (setting priorities) for further evaluation,

2. Supporting screening determinations by substituting for a portion of the current Tier 1 

battery of assays, and

3. Supporting screening determinations by substituting for the full Tier 1 battery of assays.

We have made a case for using innovative computational toxicological methods to prioritize 

chemicals and to substitute for specific Tier 1 battery assays. To substitute for a larger portion 

of Tier 1 assays, and ultimately, the full Tier 1 battery, computational models for predicting the 

full suite of Tier 1 endpoints must be evaluated. Computational tools must be applied to key 

events along the adverse outcome pathway (AOP), the conceptual framework EPA has 

adopted for evaluating potential bioactivity of a chemical in the estrogen, androgen, and thyroid 

pathways. The AOP concept begins with the interaction of an agent with the molecular target 

(e.g., receptor binding) and through a number of increasingly complex events, results in an 

adverse effect. The use of the AOP conceptual framework for evaluating the EDSP weight of 

evidence was demonstrated in case studies presented to a July, 2013 SAP 

(http://www.regulations.gov/#!documentDetail;D=EPA-HQ-OPP-2013-0230-0004).

Progression from using computational tools for priority setting to substituting for the full Tier 1 

battery of assays requires confidence in computational tools to predict interactions along the 

endocrine AOPs. This “fit-for-purpose” approach to new methods for predicting increasingly 

complex endocrine effects will be accomplished in a step-wise evaluation of high throughput 

models of bioactivity or other appropriate computational tools. At each step, EPA will clearly 

define the intended use of the computational data and demonstrate the scientific confidence 

supporting the proposed use.

To address the goals of the 2011 EDSP21 Work Plan and expand the application of 

computational tools, EPA will leverage data available from peer-reviewed literature and List 

1/Tier 1 studies in an approach similar to that described for uterotrophic data (Section 2.2.1) 

and proposed for Hershberger assays (Section 3.2.1). EPA will initiate systematic reviews and 

curation efforts of high-quality studies methodologically similar to the Tier 1 Pubertal 

Development and Thyroid Function in Intact Juvenile/Peripubertal Male Rats (OSCPP 

890.1500), Pubertal Development and Thyroid Function in Intact Juvenile/Peripubertal Female 

Rats (OCSPP 890.1450), Fish short-Term Reproduction Assay, (OCSPP 890.1350), and 

Amphibian Metamorphosis Assay (OCSPP 890.1100). Data extracted from these studies of 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 104 of 111

neuroendocrine-intact animals will be compared to computational predictions of bioactivity. 

HTS assays measuring non-receptor mediated endocrine effects, such alterations of 

steroidogenesis, will be compared to low throughput in vitro and in vivo Tier 1 (and “Tier 1-

like”) results. These analyses will evaluate the scientific confidence in substituting 

computational predictions of bioactivity for additional Tier 1 battery endpoints across the AOPs, 

and move closer to full replacement of the Tier 1 screening battery. EPA intends to present 

these analyses for scientific peer-review at upcoming FIFRA SAP meetings.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 105 of 111

References

Ahn, K.C., Zhao, B., Chen, J., Cherednichenko, G., Sanmarti, E., Denison, M.S. and 

Hammock, B.D. (2008). "In vitro biologic activities of the antimicrobials triclocarban, its 

analogs, and triclosan in bioassay screens: Receptor-based bioassay screens."

Environmental Health Perspectives, 116(9), 1203–1210.

Akahori, Y., Nakai, M., Yamasaki, K., Takatsuki, M., Shimohigashi, Y. and Ohtaki, M. (2008). 

“Relationship between the results of in vitro receptor binding assay to human estrogen 

receptor alpha and in vivo uterotrophic assay: comparative study with 65 selected 

chemicals. Toxicology In vitro 22(1): 225-31.

Akaike, H. (1974). "A new look at the statistical model identification." Automatic Control, IEEE 

Transactions on 19(6): 716-723.

Arnot, J. A., Mackay, D., Webster, E. and Southwood, J.M. (2006). "Screening Level Risk 

Assessment Model for Chemical Fate and Effects in the Environment." Environmental 

Science and Technology 40: 2316-2323.

Auld, D.S., Thorne, N., Nguyen, D.T. and Inglese, J. (2008). "A specific mechanism for 

nonspecific activation in reporter-gene assays." ACS Chemicl Biology 3(8): 463-470.

Baell, J.B. and Holloway, G.A. (2010). "New substructure filters for removal of pan assay 

interference compounds (PAINS) from screening libraries and for their exclusion in 

bioassays." Journal of Medicinal Chemistry 53(7): 2719-2740.

Bejar, E., (1988). "Preliminary investigation of the estrogenic potential of grandiflorenic acid 

from Montanoa tomentosa.", Journal of Ethnopharmacology, 23(2–3): 329-331.

Bruns, R. F. and Watson, I.A. (2012). "Rules for identifying potentially reactive or promiscuous 

compounds." Journal of Medicinal Chemistry 55(22): 9763-9772.

Cappelletti, V., Miodini, P., Fioravanti, L., Di Fronzo, G.(1995) "Effect of progestin treatment on 

estradiol-and growth factor-stimulated breast cancer cell lines. Anticancer Res.

15(6B):2551-5.

CDC (2011). "Fourth National Report on Human Exposure to Environmental Chemicals." 

Atlanta, GA, Centers for Disease Control and Prevention, National Center for Health 

Statistics.

Christian, A.E., Haynes, M.P., Phillips, M.C., Rothblat, G.H. (1997). "Use of cyclodextrins for 

manipulating cellular cholesterol content." Journal of Lipid Research 38: 2264-2272.

Cohen-Hubal, E.A., Richard, A., Aylward, L., Edwards, S., Gallagher, J., Goldsmith, M.R., 

Isukapalli, S., Tornero-Velez, R., Weber, E. and Kavlock, R. (2010). "Advancing 

Exposure Characterization for Chemical Evaluation and Risk Assessment." Journal of 

Toxicology and Environmental Health, Part B 13(2-4): 299-313.

Dann, A.B. and Hontela, A. (2011). "Triclosan: Environmental exposure, toxicity and 

mechanisms of action". Journal of Applied Toxicology, 31(4), 285–311.

Ding, D., Xu, L., Fang, H., Hong, H., Perkins, R., Harris, S., Bearden, E. D., Shi, L., Tong, W. 

(2010). "The EDKB: an established knowledge base for endocrine disrupting 

chemicals." BMC Bioinformatics 11 Suppl6:S5.

Dorfman, R. I. (1969). "Androgen and anabolic agents"> In: Methods in Hormone Research, 

Volume IIA. (Dorfman, R. I., ed.) New York: Academic Press, 151-220.

Egeghy, P. P., Judson, R., Gangwal, S., Mosher, S., Smith, D., Vail, J. and Cohen Hubal, E.A. 

(2012). "The exposure data landscape for manufactured chemicals." Science of The 

Total Environment 414: 159-166.

Gawienowski, A.M., Stadnicki, S.S. and Stacewicz-Sapuntzakis, M. (1977). "Synergistic 

uterotrophic effect of gibberellic acid and estradiol in the immature mouse", Life 

Sciences, 20(5): 785-788.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 106 of 111

Gawienowski, A.M., and Chatterjee, D. (1980). "Effect of prostaglandin inhibitor on the 

uterotrophic response of estradiol and gibberellic acid." Life Sciences 27(15): 1393-

1396.

Gee, R.H., Charles, A., Taylor, N. and Darbre, P.D. (2008). "Oestrogenic and androgenic 

activity of triclosan in breast cancer cells." Journal of Applied Toxicology 28(1): 78–91.

Go, V., Garey, J., Wolff, M.S. and Pogo, B.G. (1999). "Estrogenic potential of certain pyrethroid 

compounds in the MCF-7 human breast carcinoma cell line." Environmental Health 

Perspectives 107(3): 173-7.

Houston, J. B. and D. J. Carlile (1997). "Prediction of hepatic clearance from microsomes, 

hepatocytes, and liver slices." Drug Metab Rev 29(4): 891-922.

Huang, R., Sakamuru, S., Martin, M.T., Reif, D.M., Judson, R.S., Houck, K.A., Casey, W., 

Hsieh, J., Shockley, K.R., Ceger, P., Fostel, J., Witt, K.L., Tong, W., Rotroff, D.M., Zhao, 

T., Shinn, P., Simeonov, A., Dix, D.J., Austin, C.P., Kavlock, R.J., Tice, R.R. and Xia, M. 

(2014). "Profiling of the Tox21 10K compound library for agonists and antagonists of the 

estrogen receptor alpha signaling pathway." Scientific Reports 4: 1-9

Jamei, M., Marciniak, S., Feng, K., Barnett, A., Tucker, G. and Rostami-Hodjegan, A. (2009). 

"The Simcyp population-based ADME simulator." Expert Opinion on Drug Metabolism & 

Toxicology 5(2): 211-223.

Jamei, M., Marciniak, S., Edwards, D., Wragg, K., Feng, K., Barnett, A. and RostamiHodjegan, A. (2013). "The Simcyp Population Based Simulator: Architecture, 

Implementation, and Quality Assurance." In silico Pharmacology 1(1): 1-14.

Judson, R.S., Houck, K.A., Kavlock, R.J., Knudsen, T.B, Martin, M.T, Mortensen, H.M., Reif, 

D.M., Rotroff, D.M., Shah, I., Richard, A.M. and Dix, D.J. (2010). "In vitro screening of 

environmental chemicals for targeted testing prioritization: the ToxCast project." 

Environmental Health Perspectives 118(4): 485-492.

Judson, R.S., Kavlock, R.J., Setzer, R.W., Cohen-Hubal, E.A., Martin, M.T., Knudsen, T.B., 

Houck, K.A., Thomas, R.S., Wetmore, B.A. and Dix, D.J. (2011). "Estimating ToxicityRelated Biological Pathway Altering Doses for High-Throughput Chemical Risk 

Assessment." Chemical Research in Toxicology 24(4): 451-462.

Judson, R.S., Houck, K., Martin, M., Knudsen, T., Thomas, R.S., Sipes, N., Shah, I., 

Wambaugh, J. and Crofton, K. (2014). "In vitro and modelling approaches to risk 

assessment from the U.S. Environmental Protection Agency ToxCast programme." 

Basic & Clinical Pharmacology & Toxicology 115(1): 69-76.

Kavlock, R., Chandler, K., Houck, K., Hunter, S., Judson, R., Kleinstreuer, N., Knudsen, T., 

Martin, M., Padilla, S., Reif, D., Richard, A, Rotroff, D., Sipes, N. and Dix, D. (2012). 

"Update on EPA’s ToxCast Program: Providing High Throughput Decision Support 

Tools for Chemical Risk Management." Chemical Research in Toxicology 25(7): 1287-

1302.

Knudsen, T., Houck, K., Judson, R.S., Singh, A., Mortensen, H., Reif, D., Dix, D.J. and 

Kavlock, R.J. (2011). "Biochemical Activities of 320 ToxCast Chemicals Evaluated 

Across 239 Functional Targets." Toxicology. 282(1-2):1-15.

Kojima, H., Katsura, E., Takeuchi, S., Niiyama, K. and Kobayashi, K. (2004). "Screening for 

estrogen and androgen receptor activities in 200 pesticides by in vitro reporter gene 

assays using Chinese hamster ovary cells." Environmental Health Perspectives 112(5): 

524-31.

Lukacova, V., Woltosz, W. and Bolger, M. (2009). "Prediction of Modified Release 

Pharmacokinetics and Pharmacodynamics from In vitro, Immediate Release, and 

Intravenous Data." The AAPS Journal 11(2): 323-334.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 107 of 111

Mage, D.T., Allen, R.H., Gondy, G., Smith, W., Barr, D.B. and Needham, L.L. (2004). 

"Estimating pesticide dose from urinary pesticide concentration data by creatinine 

correction in the Third National Health and Nutrition Examination Survey (NHANES-III)." 

Journal of Exposure Analysis and Environmental Epidemiology 14(6): 457-465.

Martin, M.T., Dix, D.J., Judson, R.S., Kavlock, R.J., Reif, D.M., Richard, A.M., Rotroff, D.M., 

Romanov, S., Medvedev, A., Poltoratskaya, N., Gambarian, M., Moeser, M., Makarov, 

S.S. and Houck, K.A. (2010). "Impact of environmental chemicals on key transcription 

regulators and correlation to toxicity end points within EPA's ToxCast program." 

Chemical Research in Toxicology 23(3): 578-590.

McLanahan, E.D., El-Masri, H.A., Sweeney, L.M., Kopylev, L.Y., Clewell, H.J., Wambaugh, 

J.F. and Schlosser, P.M. (2012). "Physiologically based pharmacokinetic model use in 

risk assessment--Why being published is not enough." Toxicological Sciences 126(1): 

5-15.

Miller, J.R., Thanabal, V., Melnick, M.M., Lall, M., Donovan, C., Sarver, R.W., Lee, D.Y., 

Ohren, J. and Emerson, D. (2010). "The use of biochemical and biophysical tools for 

triage of high-throughput screening hits - A case study with Escherichia coli 

phosphopantetheine adenylyltransferase." Chemical Biology and Drug Design 75(5): 

444-454.

Mitchell, J., Arnot, J.A., Jolliet, O., Georgopolous, P. Isukapallie, S., Dasguptaf, S., Pandiang, 

M., Wambaugh, J., Egeghy, P., Cohen-Hubal, E.A. and Vallero, D.A. (2013). 

"Comparison of Modeling Approaches to Prioritize Chemicals Based on Estimates of 

Exposure and Exposure Potential." Science of The Total Environment 458: 555-567.

Morgan, J.A. and Tatar, J.F. (1972). "Calculation of the Residual Sum of Squares for all 

Possible Regressions." Technometrics 14(2): 317-325.

National Research Council (2007). Toxicity Testing in the 21st Century: A Vision and a 

Strategy, The National Academies Press.

National Research Council (2009). Science and Decisions: Advancing Risk Assessment, The 

National Academies Press.

National Research Council (2011). Sustainability and the U.S. EPA, The National Academies 

Press.

National Research Council (2012). Exposure Science in the 21st Century: A Vision and a 

Strategy, The National Academies Press.

OECD (2005) "Environment, Health and Safety Publications Series on Testing and 

Assessment No. 34: Guidance Document on the Validation and International 

Acceptance of New or Updated Test Methods for Hazard Assessment."

OECD (2010). "Draft Report of Pre-Validation and Inter-laboratory Validation for Androgen 

Receptor (AR) Mediated Stably Transfected Transcriptional Activation (AR-STTA) 

Assay to Detece Androgenic and Anti-androgenic Activities." OECD Publishing.

OECD (2012). "OECD Test No. 457: BG1Luc Estrogen Receptor Transactivation Test Method 

for Identifying Estrogen Receptor Agonists and Antagonists."

Oliveira, E.J. and Watson, D.G. (2000). "In vitro glucuronidation of kaempferol and quercetin 

by human UGT-1A9 microsomes", Federation of European Biochemical Societies 

Letters 471(1): 1-6

Oreskes, N., Shrader-Frechette, K. and Belitz, K. (1994). "Verification, Validation, and 

Confirmation of Numerical Models in the Earth Sciences." Science 263(5147): 641-646.

Oreskes, N. (1998). "Evaluation (not validation) of quantitative models." Environmental Health 

Perspectiives 106 Suppl 6: 1453-1460.

Reif, D.M., Martin, M.T, Tan, S.W., Houck, K.A., Judson, R.S., Richard, A.M., Knudsen, T.B., 

Dix, D.J. and Kavlock, R.J. (2010). "Endocrine profiling and prioritization of 
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 108 of 111

environmental chemicals using ToxCast data." Environmental Health Perspectives

118(12): 1714-20.

Rosenbaum, R.K., Bachmann, T.M., Swirsky Gold, L., Huijbregts, M.A.J., Jolliet, O., Juraske, 

R., Koehler, A., Larsen, H.F., MacLeod, M., Margni, M., McKone, T.E., Payet, K., 

Schumacher, M., van de Meent, D. and Hauschild, M.Z. (2008). "USEtox -- the UNEPSETAC toxicity model: recommended characterization factors for human toxicity and 

freshwater ecotoxicicty in life cycle impact assessment." International Journal of Life 

Cycle Assessment 13: 532-546.

Rotroff, D.M., Wetmore, B.A., Dix, D.J., Ferguson, S.S., Clewell, H.J., Houck, K.A., Lecluyse, 

E.L., Andersen, M.E., Judson, R.S., Smith, C.M., Sochaski, M.A, Kavlock, R.J., 

Boellmann, F., Martin, M.T., Reif, D.M., Wambaugh, J.F. and Thomas, R.S. (2010). 

"Incorporating human dosimetry and exposure into high-throughput in vitro toxicity 

screening." Toxicological Sciences 117(2): 348-358.

Rotroff, D.M., Dix, D.J., Houck, K.A., Kavlock, R.J., Knudsen, T.B., Martin, M.T., Reif, D.M., 

Richard, A.M., Sipes, N.S., Abassi, Y.A., Jin, C., Stampfl, M. and Judson, R.S. (2013). 

"Real-Time Growth Kinetics Measuring Hormone Mimicry for ToxCast Chemicals in T47D Human Ductal Carcinoma Cells." Chemical Research in Toxicology.

Routledgea, E.J., Parkera, J., Odumb, J., Ashbyb, J. and Sumptera, J.P. (1998) "Some Alkyl 

Hydroxy Benzoate Preservatives (Parabens) Are Estrogenic." Toxicology and Applied 

Pharmacology 153(1):12–19.

Safe, S.H., Pallaroini, L., Yoon, K., Gaido, K., Ross, S., Saville, B. and McDonnell, D. (2001). 

"Toxicology of Enviornmental Estrogens." Reproduction, Fertility and Development 

13(4): 307-315.

Scientific Committee on Consumer Safety (SCCS) (2010) "Opinion on Cyclomethicone 

Octamethylcyclotetrasiloxane (Cyclotetrasiloxane, D4) and 

Decamethylcyclopentasiloxane (Cyclopentasiloxane, D5)."

Shen, J., Xu, L., Fang, H., Richard, A. M., Bray, J. D., Judson, R. S., Zhou, G., Colatsky, T. J., 

Aungst, J. L., Teng, C., Harris, S. C., Ge, W., Dai, S. Y., Su, Z., Jacobs, A. C., Harrouk, 

W., Perkins, R., Tong, W., and Hong, H. (2013). "EADB: An Estrogenic Activity 

Database for Assessing Potential Endocrine Activity". Toxicological Sciences. 135 (2), 

277-291

Shibata, Y.,Takahashi, H., Chiba, M. and Ishii, Y. (2002). "Prediction of Hepatic Clearance and 

Availability by Cryopreserved Human Hepatocytes: An Application of Serum Incubation 

Method." Drug Metabolism and Disposition 30(8): 892-896.

Sipes, N.S., Martin, M.T., Kothiya, P., Reif, D.M., Judson, R.S., Richard, A.M, Houck, K.A., 

Dix, D.J., Kavlock, R.J., and Knudsen, T.B. (2013). "Profiling 976 ToxCast chemicals 

across 331 enzymatic and receptor signaling assays." Chemical Research in Toxicology

26(6): 878-895.

Stitzel, R.E. (1976) "The biological fate of reserpine." Pharmacological Reviews 28(3):179-208.

Tange, S., Fujimoto, N., Uramaru, N., Sugihara, K., Ohta, S. and Kitamura, S. (2014) "In vitro

metabolism of cis- and trans-permethrin by rat liver microsomes, and its effect on 

estrogenic and anti-androgenic activities." Environmental Toxicology and Pharmacology

37(3): 996-1005.

Thomas, R.S., Philbert, M.A., Auerbach, S.S., Wetmore, B.A., Devito, M.J., Cote, I., Rowlands, 

J.C., Whelan, M.P., Hays, S.M., Andersen, M.E., Meek, M.E., Reiter, L.W., Lambert, 

J.C., Clewell 3rd, H.J., Stephens, M.L., Zhao, Q.J., Wesselkamper, S.C., Flowers, L., 

Carney, E.W., Pastoor, T.P., Petersen, D.D., Yauk, C.L. and Nong, A. (2013). 

"Incorporating new technologies into toxicity testing and risk assessment: moving from 

21st century vision to a data-driven framework." Toxicological Sciences 136(1): 4-18.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 109 of 111

Thorne, N., Auld, D.S. and Inglese, J. (2010). "Apparent activity in high-throughput screening: 

origins of compound-dependent assay interference." Current Opinion in Biotechnology

14(3): 315-324.

Tonnelier, A., Coecke, S. and Zaldívar, J.M. (2012). "Screening of chemicals for human 

bioaccumulative potential with a physiologically based toxicokinetic model." Archives of 

Toxicology 86(3): 393-403.

U.S. Environmental Protection Agency, (2011), Standard Evaluation Procedure Uterotrophic 

Assay OCSPP 890.1600 

www.epa.gov/endo/pubs/toresources/seps/Final_890.1600_Uterotrophic_Assay_SEP%

209.22.11.pdf

U.S. Environmental Protection Agency, (2012). "Universe of Chemicals and General 

Validation Principles" 

http://www.epa.gov/endo/pubs/edsp_chemical_universe_and_general_validations_whit

e_paper_11_12.pdf

Wambaugh, J.F., Setzer, R.W., Reif, D.M., Gangwal, S., Mitchell-Blackwood, J., Arnot, J.A., 

Jolliet, O., Frame, A., Rabinowitz, J., Knudsen, T.B., Judson, R.S., Egeghy, P., Vallero, 

D. and Cohen-Hubal, E.A. (2013). "High Throughput Models for Exposure-Based 

Chemical Prioritization in the ExpoCast Project." Environmental Science & Technology

47(15): 8479-8488.

Wambaugh, J.F., Wang, A., Dionisio, K.L., Frame, A., Egeghy, P., Judson, R., Setzer, R.W. 

(2014). "High Throughput Heuristics for Prioritizing Human Exposure to Environmental 

Chemicals." Environmental Science & Technology 2014 Oct 24. [Epub ahead of print]

Wang, Y.H. (2010). "Confidence Assessment of the Simcyp Time-Based Approach and a 

Static Mathematical Model in Predicting Clinical Drug-Drug Interactions for MechanismBased CYP3A Inhibitors." Drug Metabolism and Disposition 38(7): 1094-1104.

Wan, Y. and Nordeen, S. K. (2002) "Overlapping but distinct gene regulation profiles by 

glucocorticoids and progestins in human breast cancer cells." Molecular Endocrinology. 

16(6): 1204-14.

Waters, N.J., Jones, R., Williams, G. and Sohal, B. (2008). "Validation of a rapid equilibrium 

dialysis approach for the measurement of plasma protein binding." Journal of 

Pharmaceutical Sciences 97(10): 4586-4595.

Wetmore, B.A., Wambaugh, J.F., Ferguson, S.S, Sochaski, M.A., Rotroff, D.M., Freeman, K., 

Clewell 3rd, H.J., Dix, D.J., Andersen, M.E., Houck, K.A., Allen, B., Judson, R.S., Singh, 

R., Kavlock, R.J., Richard, A.M. and Thomas, R.S. (2012). "Integration of dosimetry, 

exposure, and high-throughput screening data in chemical toxicity assessment." 

Toxicological Sciences 125(1): 157-174.

Wetmore, B.A., Wambaugh, J.F., Ferguson, S.S., Li, L., Clewell, H.J., Judson, R.S., Freeman, 

K., Bao, W., Sochaski, M.A., Chu, T.M., Black, M.B., Healy, E., Allen, B., Andersen, 

M.E., Wolfinger, R.D. and Thomas, R.S. (2013). "Relative Impact of Incorporating 

Pharmacokinetics on Predicting In vivo Hazard and Mode of Action from HighThroughput In vitro Toxicity Assays." Toxicological Sciences 132(2): 327-346.

Wetmore, B.A., Allen, B., Clewell 3rd, H.J., Parker, T., Wambaugh, J.F., Almond, L.M., 

Sochaski, M.A. and Thomas, R.S. (2014). "Incorporating Population Variability and 

Susceptible Subpopulations into Dosimetry for High-Throughput Toxicity Testing." 

Toxicological Sciences 2014 Aug 21 [Epub ahead of print]

Wilkinson, G.R. and Shand, D.G. (1975). "Commentary: a physiological approach to hepatic 

drug clearance." Clinical Pharmacology & Therapeutics 18(4): 377-390.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 110 of 111

Zava, D.T. and Duwe, G. (1997). "Estrogenic and antiproliferative properties of genistein and 

other flavonoids in human breast cancer cells in vitro." Nutrition and Cancer 27(1): 31-

40.
Integrated Bioactivity and Exposure Ranking

November 3, 2014 Page 111 of 111

List of Appendices

Appendix 1: ER AUC plots for reference chemicals 

Appendix 2: ER AUC plots for all chemicals 

Appendix 3: Supplementary Data: ER, AR, and IBER

Appendix 4: NICEATM_UT_GL_DB_All_Data.xls

Appendix 5: Uterotrophic study radar plots

Appendix 6: Examination of ER in vitro-in vivo discordance 

Appendix 7: AR AUC plots for reference chemicals

Appendix 8: AR AUC plots for all chemicals