1

February 4 , 2011

Don Bergfelt Office of Science Coordination and Policy (7203M) Environmental Protection Agency 1200 Pennsylvania Ave., NW. Washington, DC 20460

– 0001

Re: 75 FR 67964
, November 4 , 2010, Docket ID number EPA

– HQ

– OPPT

– 20 10

– 0877 ; Endocrine Disruptor Screening Program; Draft for Weight

- of

- Evidence Guidance Document: Evaluating Results of EDSP Tier 1 Screening to Identify
Candidate Chemicals for Tier 2 Testing

Dear M r. Bergfelt :

These comments are submitted on behalf of People fo r the Ethical Treatment of Animals and the Physicians Committee for Responsible Medicine whose members are national animal protection, health, and scientific advocacy organizations with a combined constituency of

more than two million Americans . We share the common goal of promoting reliable and relevant regulatory testing methods and strategies that protect human health and the environment while reducing, and ultimately eliminating, the use of animals.

In the Federal Register on November 4 , 2010, the Env ironmental Protection Agency (EPA )

, announced the availability of a D raft Weight

- of

- Evidence (WoE) Guidance Document: Evaluating Results of EDSP Tier 1 Screening to Identify Candidate Chemicals for Tier 2 Testing to be used for evaluating results of Tier 1 screening and determining whether a chemical should undergo Tier 2 analysis. The purpose of this document “is to set forth some of the general principles, criteria, and considerations EPA generally believes to b e relevant under a WoE approach for evaluating data submitted as part of EPA’s two

- tiered paradigm for screening and testing chemicals for endocrine activity (i.e., estrogen, androgen, and thyroid hormonal systems; E, A, and T) under the EDSP (75 FR 67964) .” It is also noted in the Federal Register that principles articulated in the draft WoE are applicable not only to evaluation of data from individual assays with multiple endpoints, but across the whole suite of assays in the Tier 1 screening battery. A dditionally, these same principles will be applied to review of Other Scientifically Relevant Information (OSRI) submitted as potential substitutes/equivalents for designated Tier 1 screening assays.2

General Comments

During the review of the EDSP Phase I and Tier 1 assays, several stakeholders expressed concern over a lack of clarity in EPA’s decision

- making process, either for determining positive/negative outcomes of each Tier 1 assay, or in determining how decisions will be made to proceed to Tier 2 t esting (available at Regulations.gov in Docket no. EPA

- HQ

OPPT

- 2007

- 1081 )

. As a result, i n its Terms of Clearance to EPA regarding the original Information Collection Request, the Office of Management and Budget (OMB) requested that
EPA “ensure s ufficient opportunity prior to submission of any revision to this collection for public comment and peer review of the EPA tools to be developed to guide agency decisions

on whether a chemical must proceed to Tier II, including the Weight of the Evidenc e Approach and Standard Evaluation Procedures (emphasis added) .” In addition, in report language, the House Appropriations Committee for the Interior and Environment ( FY 2010 )
directed EPA to: “

develop and publish criteria for evaluating the results of Tier 1 screening and determining whether a chemical should undergo Tier 2 analysis within one year of

enactment

. The process should allow for public input (US Congress, 2010) (emphasis added) .” EPA’s Draft Wo E Guidance Document ( GD ) is intended to fulfil l these instructions; however, the content is inadequate to satisfy either the letter or intent of both OMB and Congress.

The draft Wo E GD contains very little information that is specific to Tier 1 test results; rather, the draft describes the history of the EDSP, outlines W o E approaches in general and reiterates its definition of Other Scientifically Relevant Information (OSRI). Out of a nine page document, only a single paragraph relates to evaluation of Tier 1 studies specifically and
includes the fol lowing points:

 In general, however, Tier 1 in vivo results would carry greater weight than in vitro
results

 Concordant effects found in multiple interrelated endpoints generally imply a compromise in endocrine function, in contrast to isolated or discor dant effects.

 “Generally, consistent positive or negative effects across studies and taxa increase
confidence...”

 “other available information may also be appropriate” and “could include consideration of other critical effects (non

- endocrine), dose respons e, what is understood about the underlying basis (i.e., toxicity MOA) of these critical effects (i.e., non

- endocrine or an endocrine MOA not covered by Tier 1) and their human relevance, and potential for exposure.”

The draft does not describe specifical ly what criteria the EPA will use to determine whether a chemical does or does not interact with the EAT hormone system(s) or what
information will be used to determine whether a chemical will proceed to Tier 2 testing.
Nor has EPA provided “Standard Evaluation Procedures,” satisfying neither the intent

n or the letter of the OMB Terms of Clearance or the FY 2010 House appropriations language.3

WoE GD F ails to P rovide S tated G oal of T ransparency

In a paper developed through the International Life Sciences Institute’s (ILSI) Health and Environmental Sciences Risk Assessment Weight of Evidence Project, Schreider et al. (2010) state that “[t]ransparency and documentation of the decision process are at the core of a credible risk assessment and, in addition, are essential in the presentation of a weight

- of

evidence

- based approach (emphasis added).” When describing benefits attributable to a WoE framework in regulatory decisions, Krimsky (2005) cites Walker’s (1996) three objectives of a WoE analysis: (1) it provides a “clear and transparent framework” for evaluating the evidence in a risk determination; (2) it offers regulatory agencies a consistent and standardized approach to ev aluating toxic substances; and (3) it helps to identify the discretionary assumptions in risk determinations from experts. Unfortunately, as Krimsky goes on to say, most applications of WoE in support of public policy “seem to (by inference or lack of spe cification) use a process methodology that is low on transparency and high on subjectivity.” While it is stated by EPA in the Draft WoE that “this document provides a transparent scientific approach for broadly evaluating Tier 1 screening data to determin e if additional Tier 2 testing is necessary”, and later goes on to add that for each chemical “[a] summary of the WoE analysis is expected to transparently state and explain conclusions”, we find little in the document that transparently and clearly articu lates the methods EPA will use to determine which OSRI will be considered relevant to the objectives of the EDSP and how
EPA will evaluate Tier 1 results to determine whether a chemical will have to undergo Tier 2 testing.

EDSP

- specific WoE A

pproach is L acking ; S uggestions for I mprovement

To create an objective and unbiased assessment of the hypothesis that a particular chemical is acting via an endocrine mechanism of action to cause adverse effects on animals in
laboratories

, wildlife populations

, and humans, all of the relevant information needs to be considered in an organized and structured manner (WHO, 2002). In the absence of a single definitive study, WoE analyses must rely on a rational integration of mul tiple sources, qualities, and types of data, as well as consideration of uncertainty factors and variability in
results. As par t of a general description of Wo E, t he Wo E GD outlines how individual studies

are analyzed , for example, by characterizing the “nature of the effects ”, specificity and sensitivity, dose

- and time

- dependent changes, potency and magnitude of changes and consistency among assays. Also, study “quality” will be considered

, i.e., studies performed under GLP will be given more weight. At the very least, EPA should develop a scheme for grading evidence similar to what the National Toxicology Program (NTP, 2009) has done in developing criteria to classify outcomes from no n

- cancer studies. NTP uses five categories or
“levels of evidence” to describe non

- cancer findings: clear evidence, some evidence, equivocal evidence, no evidence and inadequate stud y, and states the type of results that
would characterize each level. Th ere is nothing analogous to this in the Draft WoE GD presented by EPA.

The GD lists a number of factors that contribute to an integrated analysis of data and acknowledges

, in general terms

, that some factors may be considered more important than
others :4

 Quality of data and replicability of results

 Strengths and limits of in vitro and in vivo results

 Number and type of effects; potency and magnitude of effects

 Consistency, pattern, range, and interrelationships of effects observed across studies, species, strains, and sexes

 Dosage, route, duration conditions under which effects occur

 Understanding of MOA and biological plausibility of responses

While the above factors represent most of those that should be considered in a WoE analysis,
there is no description of how these general principles will be specifically applied to Tier 1

assays . Nor is there an explanation or scheme provided which clearly describes how these
factors will be assessed or integrated , what relevancy weight will be assigned to each, and
how the final result “score” will be determined

. W ithout a clear explication of how evidence will be “weighed” and “weighted”, we find it difficult to see how there can be

cons ist ency between assessors or between assessments of different chemicals . Nor will it
be clear to the recipients of test orders or the public how decisions have been made.

EPA gives some insight as to the importance of in vitro studies versus in vivo studies, stating that “[i]n general, Tier 1 in vivo results would carry greater weight than in vitro results
because in vitro assays inherently lack physiological conditions associated with whole tissues or organs and, therefore, have nil or very limited ability to represent metabolic processes and pathways leading to endocrine disruption.” Will all in vivo studies be treated equally and always be given greater weight over in vitro studies? For example, will results from specific
mechanistic tests such as the Her shberger and uterotrophic assays be weighted more heavily
than if results from s everal in vitro tests for the same pathway concur ? And will mechanistic

in vi vo tests be weighted the same as less specific tests such as the pubertal assays? Are all of the endpoints within each assay weighted similarly? For example, are histological changes weighted the same as hormone levels? How will EPA differentiate between relevant in vivo findings and those that are not ? For example, how will it be determined that a change in fecundity seen in the short

- term fish assay is due to an endocrine

- related mechanism?

These are a few of the many questions that EPA has failed to answer in its D raft WoE GD because it has not clearly laid out the steps that must be taken in order to provide a transparent, clearly defined decision

- making process based on specific results of Tier 1 assays and how they relate to an endocrine

- related mode or mechanism of action

. To meet this goal

we recommend the following elements be included in the WoE guidance :

1. Clearly state the foundation of data evaluation in relation to endocrine disruption
based on what we currently know, including relevance , applicability and limit ation s of that knowledge

.
2. Define the hypotheses to be tested (e.g., estrogen agonist, estrogen antagonist, androgen agonist, etc.) and how each test relates to those hypotheses.5

3. Clearly state the rules that will be followed when evaluating OSRI a nd its relevance to objectives of EDSP testing

.
4. Identify and provide rationale for a priori assumptions or judgments used to weight
data.

5. Develop an empirically

- based quantitative weighting scheme to evaluate OSRI, Tier 1 and Tier 2 data (see discussion of Calabrese et al., 1997, below).

6. Establish a n explicit basis for determining cause and effect.

Calabrese et al. (1997) developed a quantitative weighting system to evaluate estrogen

- related
endocrine effects of chemicals on human health. Criteria providing the predictive framework included: (1) endocrine disruption as a multistage process (stages b eing receptor binding, transcription, macromolecule synthesis, cell proliferation, clinical manifestations) with effects on the later stages having more weight , (2) phylogenetic relatedness , (3) model system ( in
vitro versus in vivo ), and (4) estrogenic po tency. When developing such a weighting system,
it i s necessary to consider that scientific evidence for a hypothesis generally appears in gradations, and preserving the continuous nature of these data can provide more valuable information than turning it into a dichotomous (above or below a threshold) variable. If continuous data must be reduced to ordered or dichotomous judgments, then the rationale for doing this should be provided. In computing chemical rankings, t he authors state that estrogenic pot ency reflects quantitative differences with respect to species and stage in the endocrine system, and

, as such

, was assigned greater importance than the other criteria. Similarly, weighting of phylogenetic factors was based on the assumption that organism s more closely related phylogenetically were more likely to respond in a similar manner to
endocrine

- disrupting agents than more distantly related ones. Thus, when evaluating assay results in relation to a particular organism, a test done on a species wit hin the same genus would be given much more weight and have a lower uncertainty factor, than a test done on an animal of the same family within an order. Using the system proposed by Cal a brese et al. , a high final value would indicate that the study used a more apical endpoint of high predictive value, generated data using a closely related species, was performed in a whole animal experimental model, and the chemical tested had a potency similar to that of the model high

potency es trogen, estradiol. While this framework may not receive universal agreement from experts, it clearly lays out the factors used, their relative weights, any empirically

- based and judgment

- based assumptions that were mad e , and the rationale behind each element of the
framework.

In the EDSP, separate WoE assessments will be needed for determining:

a) whether a substance exhibits the potential for interaction with androgen, estrogen, or thyroid pathways

, based on Tier 1 screening results and OSRI;

b) whethe r a substance should be further evaluated in Tier 2 tests, based on Tier 1 results
and OSRI , and if so, which Tier 2 tests ;
c) whether a substance exhibits adverse effects based on Tier 2 tests ;6

d) whether the adverse effects observed in Tier 2 tests are a consequence of an endocrine

- related mode or mechanism of action , based Tier 1 results , OSRI, Tier 2 tests, and as necessary, additional mode

- of

- action experiments, and;

e) whether endocrine

- mediated adverse effects on humans or wildlife are possible at environmentally relevant exposure levels.

T he predictive framework of Calabrese et al. (1997) may not be exactly applicable to evaluating Tier 1 and 2 testing data and OSRI, but it certainly provides an example of the type of weighting system that is needed to ensure consistent and transparent eval uation within the
EDSP . Krimsky (2005) notes that while there may be certain pitfalls and disagreement among experts as to how and what to weight in a WoE analysis, “[w]hen WoE is used consistently and uniformly by a regulatory body, it enables that body to develop a strong comparative approach for assessing the potential health and environmental effects of products.”

Without clearly articulated

, standardized methods in place for each of the above determinations, we do not see how rational, transparent decision

- making on the part of EPA
will be achieved. The WoE GD states that it was developed “to provide guidance to EPA staff and managers who will be reviewing data submitted in response to Orders for Tier 1 screening that began October 29, 2009 under the EDSP” and to inform outside parties as to how the Tier 1 screening results are being evaluated. It is also expected , by EPA, to comply
with OMB Terms of Clearance for the ICR as well as the 2010 House Appropriations Report language. As currently written, the WoE document fails to achieve any of these objectives.

Thank you for considering our comments.

Sincerely,

Patricia L. Bishop, M.S.

Resea r ch Associate Regulatory Testing Divis ion People for the Ethical Treatment of Animals

Catherine Willett, PhD Science Policy Advisor Regulatory Testing Division People for the Ethical Treatment of Animals7

Kristie Sullivan , M

. P

. H

. Science & Policy Advisor Physicians Committee for Responsible Medicine

References

Calabrese, E.J., L.A. Baldwin, P.T. Kostecki, and T.L Potter. (1997). A toxicologically based weight

- of

- evidence methodology for the relative ranking of chemicals of endocrine disruption potential. Regulatory Tox. & P harma. 26 : 36

- 40.

Krimsky, S. (2005) The weight of scientific evidence in policy and law . Am J Public Health. 95 (S1): 129

- 136.

National Toxicology Program. ( 20 09)

. NTP Non

- Cancer Evaluation Criteria

. U.S. Department of Health and Human Services. Accessed at http://ntp.niehs.nih.gov/?objectid=CD5E017E

- F1F6

- 975E

- 7D86DAA5256F6AB8

Schreider, J., C. Barrow, N. Birchfield, K. Dearfield, D. Devlin, S. Henry, M. Kramer, S. Schappelle,. K. Solomon, D. L. Weed, and M. R. Embry (2010). Enhancing the credibility of decisions based on scientific conclusions: Transparency is imperative.
Toxicol

. Sci

. 116(1) : 5

- 7

.

U.S. Congress. House Report 111

– 180. Department of the Interior, Environment, and Related Agencies Appropriations Bill, 2010. pp. 105

– 106. Available at: http://www.gpo.gov/fdsys/pkg/CRPT

- 111hrpt180/pdf/CRPT

- 111hrpt180.pdf (Accessed 10 Jan, 2011)

Walker

, V.R. (1996) Risk characterization and the weight of evidence: Adapting gatekeeping concepts from the courts. Risk Analysis 14: 793

- 799.

World Health Organization. (2002) Chapter 7 of Global assessment of the state

- of

- the

science of endocrine disruptors: Causal Criteria for Assessing Endocrine Disru ptors

- A Proposed Framework. WHO/PCS/EDC/02.2. WHO, Geneva Switzerland. 180p.