Comments on EPA’s White Paper

“Integrated Bioactivity and 

Exposure Ranking: A 

Computational Approach for the 

Prioritization and Screening of 

Chemicals in the Endocrine 

Disruptor Screening Program”

Richard A. Becker Ph.D. DABT

Dec. 3, 2014
Additional work by EPA, however, should be 

considered to improve the scientific basis of the 

methods used, and to fully document model 

development and performance.

The approach presented in EPA’s IBER Profiling 

White Paper represents a significant advance in 

bringing such methods to the forefront

We support EPA’s conceptual approach to use 

activity in screening assays measuring E and A 

endocrine-related endpoints and to combine this 

with HT exposure information to provide an 

improved scientific basis for priority setting

2
Recommendations for the Path Forward: 

Enhance Transparency & Refine Methodology

• We understand that the Agency intends to release model codes 

in the future, but the lack of such information at the present 

time renders a comprehensive independent verification of 

modeling by outside experts and peer reviewers very difficult. 

• ACC’s comments include an analysis prepared for ACC by Dr. 

Tony Cox; specific comments on the strengths and limitations of 

the area under the curve (AUC) approach for activity profiling 

are noted, and recommendations for improvement are made. 

• We request that the SAP and Agency give due attention to Dr. 

Cox’ report as the SAP conducts its evaluation and the Agency 

moves forward to revise the IBER Profiling method for EDSP 

priority setting.

3
Becker et al. 2014. Interpreting Estrogen Screening 

Assays in the Context of Potency and Human Exposure 

Relative to Natural Exposures to Phytoestrogens

4
Assist ACC groups in developing science and 

research projects, identifying collaborations and 

design of strategies for utilization of results in 

product stewardship & regulatory activities

Development and application of Exposure : Activity Ratio 

(EAR) and Relative Endocrine Exposure Activity Quotient 

(REEAQ) to ToxCast & T21 Estrogen Assay Results

5

An Exposure:Activity Profiling Method for Interpreting High 

Throughput Screening Data for Estrogenic Activity – Proof of 

Concept (Becker et al., 2014 Accepted for Publication in Reg Tox 

Pharm)
Assist ACC groups in developing science and 

research projects, identifying collaborations and 

design of strategies for utilization of results in 

product stewardship & regulatory activities

Development and application of Exposure : Activity Ratio 

(EAR) and Relative Endocrine Exposure Activity Quotient 

(REEAQ) to ToxCast & T21 Estrogen Assay Results

• Oral equivalent doses (mg/kg/d) from Wetmore et 

al. 2012. 

• EARs: compare: 

(1) human exposure estimates (oral, mg/kg/d) to

(2) predicted oral equivalent doses (mg/kg/d), 

corresponding to HTS assay AC50.

• EAR values for GEN are based on steady state 

plasma levels. 

6
Assist ACC groups in developing science and 

research projects, identifying collaborations and 

design of strategies for utilization of results in 

product stewardship & regulatory activities

Activity Results: ToxCast Estrogen Activity

(ATG_ERa_TRANS)

Log Scale

Disulfoton

Oxyfluorfen

Metolachlor

Clomazone

Cyproconazole

Benfluralin

Cycloate

Bensulide

Tefluthrin

Diphenylamine

Pendimethalin

Esfenvalerate

Fenamidone

Fenarimol

Fenhexamid

Flusilazole

Trifluralin

Myclobutanil

Permethrin

Chlorpyrifos-methyl

Endosulfan

Bifenthrin

Dicofol

Bisphenol A

Piperonyl butoxide

Fenbuconazole

Ethalfluralin

Cyprodinil

2-Phenylphenol

Genistein

Boscalid

0.001

0.010

0.100

1

1 0

100

Activity (AC50 in mM)

ATG_ERa_TRANS

7
Assist ACC groups in developing science and 

research projects, identifying collaborations and 

design of strategies for utilization of results in 

product stewardship & regulatory activities

Log Scale

Exposure: Activity Ratios (EAR) 

(ATG_ERa_TRANS)

Disulfoton

Oxyfluorfen

Metolachlor

Clomazone

Cyproconazole

Benfluralin

Cycloate

Bensulide

Tefluthrin

Diphenylamine

Pendimethalin

Esfenvalerate

Fenamidone

Fenarimol

Fenhexamid

Flusilazole

Trifluralin

Myclobutanil

Permethrin

Chlorpyrifos-methyl

Endosulfan

Bifenthrin

Dicofol

Bisphenol A

Piperonyl butoxide

Fenbuconazole

Ethalfluralin

Cyprodinil

2-Phenylphenol

Genistein

Boscalid

1

0 .1

0 .0 1

0 .001

0 .0001

0 .00001

0 .000001

0 .0000001

Exposure:Activity Ratio

ATG_ERa_TRANS

8
Assist ACC groups in developing science and 

research projects, identifying collaborations and 

design of strategies for utilization of results in 

product stewardship & regulatory activities

Log Scale

Relative Endocrine Exposure: Activity 

Quotient (REEAQ) (ATG_ERa_TRANS)

Disulfoton

Oxyfluorfen

Metolachlor

Clomazone

Cyproconazole

Benfluralin

Cycloate

Bensulide

Tefluthrin

Diphenylamine

Pendimethalin

Esfenvalerate

Fenamidone

Fenarimol

Fenhexamid

Flusilazole

Trifluralin

Myclobutanil

Permethrin

Chlorpyrifos-methyl

Endosulfan

Bifenthrin

Dicofol

Bisphenol A

Piperonyl butoxide

Fenbuconazole

Ethalfluralin

Cyprodinil

2-Phenylphenol

Genistein

Boscalid

1

0 .1

0 .0 1

0 .001

0 .0001

0 .00001

0 .000001

0 .0000001

REEAQ

ATG_ERa_TRANS

9
Assist ACC groups in developing science and 

research projects, identifying collaborations and 

design of strategies for utilization of results in 

product stewardship & regulatory activities

Utility of EAR and REEAQ Approaches: 

Proof of Concept

• Risk (Hazard & Exposure), therefore:

prioritization should be (Activity &Exposure) 

• EAR and REEAQ provide a scientifically sound approach 

for placing HTS results into context for priority setting. 

• GEN as a reasonable benchmark for EAR and REEAQ: 

-component of a healthy diet

-[GEN]serum levels in US population available

• Potential chemical candidates for deprioritization for 

screening and testing (e.g. EDSP) might follow: 

EARchemical << EARGEN (i.e., REEAQ < 1)

10
Recommendations for the Path Forward: 

Taking a Closer Look at the AUC Metric 

• The AUC methodology is not sufficiently described in the EPA White 

Paper (but clarified to a degree by EPA’s 12/2/14 ppts to the SAP). 

However the non-linear transformation to yield AUC values creates 

complexity & complicates interpretation & this is compounded when 

combined with linear exposure estimates to yield IBERs.

• Literature values for relative potency in estrogen assays (see EPF 

comments) indicate many orders of magnitude difference between 

strong & weak agonists (e.g. genistein is 100 to >10000 less potent 

than DES). 

• The scientific rationale for this compression of AUC values is not 

adequately described or discussed in the EPA White Paper.

• The SAP should evaluate these issues and the Agency should address 

this in its response to comments / revision of the IBER profiling method 

for EDSP priority setting.

11
Recommendations for the Path Forward: 

Taking a Closer Look at the AUC Metric 

From: Appendix 1 ER AUC Scores, Reference Chemicals

http://www.regulations.gov/contentStreamer?objectId=0900006481911aed&disposition=attachment&contentType=pdf

12

EPA’s DES AUC = 1.01 

EPF DES Relative Potency to 

17Estradiol = 1.63E-1

EPA’s GEN AUC = 0.77 

EPF GEN Relative Potency to 

17Estradiol = 1.13E-5
Recommendations for the Path Forward: 

Taking a Closer Look at the AUC Metric 

• A Scientific Confidence Framework for evaluating the AUC 

methodology (adopted from Cox et al. 2014. Developing 

scientific confidence in HTS-derived prediction models: Lessons 

learned from an endocrine case study. Regul. Toxicol. 

Pharmacol. 69, 443-50.

• Analytical Validation 

• Qualification

• Utilization

13
Recommendations for the Path Forward: 

Taking a Closer Look at the AUC Metric 

Analytical Validation:

Assessment of the biological basis and analytical performance of 

assays. 

• Each assay should map to a defined mechanistic endpoint (e.g., a 

key event in the mode of action or AOP). 

• Documentation of assay performance characteristics (reliability, 

sensitivity, and specificity) 

• A defined chemical domain of applicability

• Transparent data sets (to enable independent verification) should 

be readily available.

• Note: This framework was used to form the basis of the draft 

OECD guidance “Characterizing non-guideline in vitro test 

methods to facilitate their consideration in regulatory 

applications” 

14
Recommendations for the Path Forward: 

Taking a Closer Look at the AUC Metric 

Qualification:

Assessment of the prediction model derived from the assays.

• A defined algorithm for each prediction model. 

• Appropriate measures of goodness-of-fit, robustness and 

predictivity of the prediction models (models may be quantitative 

or qualitative). 

• Known limitations of each prediction model should be 

summarized. 

• Prediction models should be characterized in sufficient detail to 

facilitate review, reconstruction and independent verification of 

results. 

15
Recommendations for the Path Forward: 

Taking a Closer Look at the AUC Metric 

Utilization:

Contextual and weight-of-evidence analysis of the use (qualitative or 

quantitative) of the prediction model for a specific purpose. 

• Defining the intended purpose of the prediction model

• Documenting/justifying applications, based on weight of 

evidence, of the scientific confidence to support the use of the 

method for specific purposes:

• For example:

(1) priority setting, where the model is used to identify priority substances 

for more detailed evaluation; 

(2) chemical categorization for subsequent read-across

(3) screening level assessment of a biomarker, where model is used as a 

surrogate data point for a biochemical endpoint or a biomarker; 

(4) integrated testing strategy, or where the model is used to 

describe/predict a hazard property in lieu of a traditional tox study 

(5) to predict an adverse outcome.

16
Enhancing product stewardship and regulatory decision 

making through innovative science & research

17

• The approach presented in EPA’s IBER Profiling 

White Paper represents a significant advancement 

in using HTS and exposure information to support 

decision making

• Additional work by EPA should be considered to 

improve the scientific basis of the methods used, 

and to fully document model development and 

performance.

Thank you for your attention