### December

**1 2**  **- 13, 2013**  **COMMENTS AT** 

**THE EP A**  **BI -** **MONTHL**

**Y LISTENING**  **SESSION: RDX, ETBE,**  **and tert** **- Butanol**  **O n behalf of**  **ACC and**  **the Center for**  **A dvancing Risk**  **Assessment Science**  **and Policy Nancy Beck,**  **PhD, DABT Senior**  **Dir ector**  **, ACC** 

## ACC and **the Center f**  **or A**  **dvancing Risk**  **Assessment Science**  **and Policy (ARASP)**  **ACC:**

• Represents the leading companies engaged in the business of chemistry
. • Committed to improved environmental, health and safety performance through Responsible Care ®


### . 

## ARASP:

• Coalition of 19 organizations focused on development and application of scientifically sound methods for conducting chemical assessments.

• Members include chemical specific panels and other trade associations. See : http://arasp.americanchemistry.com/

## IRIS Enhancements  July 2013 IRIS enhancements are a constructive step forward.  Early stakeholder engagement, particularly before a draft is developed will help strengthen assessments and move them to completion in a more timely manner
.  Planning and scoping will help in understanding parameters that will be assessed.  Complex scientific issues will ideally be discussed earlier in the process.  Identification of critical studies and their summaries should help stakeholders understand the direction the agency is heading.  Appropriate exposure response tables will help provide context
. 

###  The release of evidence tables, while a helpful start, is not suf ficiently consistent with the IRIS enhancements.  Further improvements are necessary.

## IRIS Pr**ocess Step 1**   The revised IRIS process documentation includes the following in the details about Step 1:  Begins with planning and scoping, including public meeting on technical problem f orm ulation and release of planning and scoping summary.  Conducts literature search and critical study selection
.  Develops evidence tables that succinctly summarize the critical studies to be considered in developing the assessment .  Publicly releases literature search, literature search strategy , critical study selection criteria, evidence tables for critical studies, and exposure
response figures (which graphically depict responses at dif ferent exposure levels for studies in evidence tables ).  Convenes public meeting to discuss literature search, evidence tables, exposure - response figures, and key issues
.  Only one of these five elements is complete in the RDX, tert - butanol 

### and ETBE releases.

# Knowing is not enough; we must apply
. Wi lling is not enough; we must do
.

- Johann Wo lfgang von Goethe

## Evidence **Ta ble**  **Releases Fall**  **Short (1)**  No planning and scoping summary is provided.  There is no understanding of the questions being asked or issues to be addressed.  No context is provided in the released evidence tables.  EP A’ s definition of a systematic review is related only to the literature search.  Docum ent entitled “Systematic review of the ETBE literature” on IRIS takes readers to a HERO webpage for identified studies.  Systematic review must be more than a first step literature search strategy.  All studies identified in the literature search, that provide an endpoint where a change is seen, are deemed ‘critical studies’.  There are no ‘critical study’i dentification criteria.  Study quality must be an essential element of ‘critical study’ identification.

## Evidence **Ta ble**  **Releases Fall**  **Short (2)**  Evidence tables present all studies that show a change in an endpoint, not necessarily ‘critical studies’.  There is no discussion of which studies should be treated as ‘critical studies’due to quality

, methodology

, adversity of ef

fect, or any other criteria.  Studies negative for statistical changes are excluded from evidence tables, therefore making them incomplete and misleading.  While an approach like this may work where there are limited studies identified in a literature review , as more complex chemicals are reviewed, the EP A approach will be unworkable
.  Burden is on stakeholders to review and comment on every study identified in the literature review , not just those in the table
. 

###  Evidence tables focus only on endpoints, ignoring all ‘critical studies’that relate to mode of action (MOA).

## Evidence **Ta ble**  **Releases Fall**  **Short (3)**  Exposure
- response arrays are misleading  All studies are treated as being of equal quality , when this is not the case.  Exposure
- response arrays are limited to positive endpoints, ignoring studies with negative endpoint findings and also ignoring all mode of action information.

## Impr**oving Evidence**  **Ta bles**  1) Release planning and scoping summary along with evidence tables. 2) Don’ t confuse a literature search with a systematic review. Clarify terminology and use it consistently
. 3) Determine, a priori , criteria for ‘critical study’ identification. This should include not only endpoint specific data, but also mode of action information.
• It should be more specific than including all relevant studies identified in the literature review
. 4) Critical study criteria should include, at a minimum, a review of study quality, methodology, relevance, and adversity of ef fect (if relevant).
• This review should be released by EP A along with the evidence tables. 5) Evidence tables and exposure
- response arrays should be developed for mode of action information (see Kushman et. al., for example). 6) Exposure
- response arrays should include only those studies of suf ficient quality (e.g. those meeting critical study criteria) and quality of studies should be clear in the arrays.

### December

**1 2**  **- 13, 2013**  **COMMENTS AT** 

**THE EP A**  **BI -** **MONTHL**

**Y LISTENING**  **SESSION: RDX O**  **n behalf of ACC**  **and the Center**  **for A** **dvancing Risk Assessment**  **Science and Policy**  **Nancy Beck, PhD, DABT**  **Senior Dir** **ector ,**  **ACC**

## RDX General **Recommendations** 1) Release planning and scoping summary along with evidence tables. 2) Don’ t confuse a literature search with a systematic review. Clarify terminology and use it consistently
. 3) Determine, a priori , criteria for ‘critical study’ identification. This should include not only endpoint specific data, but also mode of action information.
• It should be more specific than including all relevant studies identified in the literature review
. 4) Critical study criteria should include, at a minimum, a review of study quality, methodology, relevance, and adversity of ef fect (if relevant).
• This review should be released by EP A along with the evidence tables. 5) Evidence tables and exposure
- response arrays should be developed for mode of action information (see Kushman et. al., for example). 6) Exposure
- response arrays should include only those studies of suf ficient quality (e.g. those meeting critical study criteria) and quality of studies should be clear in the arrays.

## RDX Specific **Concerns (1)**   Literature search approach needs improvement  EP A apparently excluded mechanistic, genotoxicity , and toxicokinetics data. These were kept as additional data but not primary sources. Thus none of these studies are considered ‘critical’.  However, EP

A created tables using genotoxicity studies (table 2
- 14). Use of additional data, in evidence tables, seems inconsistent at this stage and also not consistent with EP

A description of what is displayed in bold in section 1.2.1. Is all genotox information in the tables? If not, why not?  How and when will EP A use all additional studies in the assessment?  All studies should be evaluated for quality and relevance, consistent with developed criteria.  When EP

A searched DTIC, one study was not brought forward due to study quality considerations.  What were the criteria used? How were these criteria applied to all other studies in the evidence tables?  Seems like an inconsistent approach as Ta ble 2
- 1 contains at least one study with no exposure measurements. Shouldn’ t this also have been excluded?

## RDX Specific **Concerns (2)**   Why are ecosystem ef fects information maintained for an IRIS assessment that is theoretically only evaluating human health ?

• Releasing the scoping and planning document may help clarify this. 

###  Date limits on literature search should be clear and consistent.

• For som e searches mo nth, date, and year information is provided; for others only a year or month and year. Further transparency is required to improve reproducibility
. 

###  References to tables and arrays as including “Key Study Data” should be corrected and clarified.  All mode of action and mechanistic information, which could be critical to the assessment is excluded from tables and arrays.  Other than a literature review , EP A has done no study evaluation to determine which studies should be “key studies”.

## RDX Specific **Concerns (3)**   EP A states that tables include references “to identify those most pertinent for evaluating human health ef fects..”

• Mode of action information and mechanistic information MUSTbe considered pertinent.

• Evidence tables for mechanistic information can be created (see, for example, Kushman et al. 2013, which includes NCEA authors). 

###  Endpoints for which no ef fects are seen in any studies are completely excluded from evidence tables.

• Negative information should be considered pertinent and informative to IRIS assessments. Should not be excluded from the toxicological review
. At a minimum, this information can help inform the database uncertainty factor
.

## RDX Specific **Concerns (4)**   For mortality and neurological endpoints, EP A identified ef

fects based on biological significance, rather than statistical significance.

• EP A stated that for studies with a low number of animals, this may preclude identifying a statistically significant change. EP A implies that statistical significance could be achieved with a higher num ber of animals, however this is not known, nor is it known whether there is any causal relationship (see Goodman et al., 2010 in Reg To x and Pharm).

• EP A is missing any clear definition of what constitutes “biological significance.”

• Is this consistent with common practice? 

###  EP A includes studies with no exposure measurements in the tables

• Its not clear how these data can be ‘critical studies’. Som e evaluation of study quality and relevance is necessary
.

## RDX Specific **Concerns (5)**   Evidence tables don’ t present all the information.

• Key information about endpoints or timepoints that were evaluated, but not statistically significant are missing from the tables.

• Clarity about which changes are statistically significant is missing from the tables, although it is noted in some arrays.

• Is this consistent with common practice? 

###  EP A conducted their own statistical analysis

• Is this appropriate at this stage? S houldn’ t study quality be considered first?

• For other evidence tables, no further analysis was conducted. Why did EP A decide to do their own analyses here?

### December

**1 2**  **- 13, 2013**  **COMMENTS AT** 

**THE EP A**  **BI -** **MONTHL**

**Y LISTENING**  **SESSION: ETBE AND**  **T ert** **- BUT**  **ANOL O**  **n behalf of ACC**  **and the Center**  **for A** **dvancing Risk Assessment**  **Science and Policy**  **Nancy Beck, PhD, DABT**  **Senior Dir** **ector ,**  **ACC**

## ETBE and t**- Butanol**  **General Recommendations**  1) Release planning and scoping summary along with evidence tables. 2) Don’ t confuse a literature search with a systematic review. Clarify terminology and use it consistently
. 3) Determine, a priori , criteria for ‘critical study’ identification. This should include not only endpoint specific data, but also mode of action information.
• It should be more specific than including all relevant studies identified in the literature review
. 4) Critical study criteria should include, at a minimum, a review of study quality, methodology, relevance, and adversity of ef fect (if relevant).
• This review should be released by EP A along with the evidence tables. 5) Evidence tables and exposure
- response arrays should be developed for mode of action information (see Kushman et. al., for example). 6) Exposure
- response arrays should include only those studies of suf ficient quality (e.g. those meeting critical study criteria) and quality of studies should be clear in the arrays.

## ETBE and **tert -** **Butanol Specific**  **Concerns (1)**   Literature search approach needs improvement.  EP A apparently excluded structure activity information. Why?  To xicokinetic data were kept as additional data but not primary sources. Thus none of these studies are considered ‘critical’.  Specific rational for excluding individual studies is not clear
. More documentation is necessary
. • How and when will EP A use all additional studies in the assessment? If a study is ‘pertinent’but not ‘primary’what exactly does this mean?  All studies should be evaluated for quality and relevance, consistent with necessary criteria. 

###  Date limits on literature search should be clear and consistent
.

 Range of years searched is not clear
.

## ETBE and **tert -** **butanol Specific**  **Concerns (2)**   Referrals to tables and arrays as including most pertinent references should be corrected and clarified.  Mode of action and mechanistic information, which could be critical to the assessment is excluded from tables and arrays (unless it is in an endpoint study).  Other than a literature review , EP A has done no study evaluation to determine which studies should be most important
. 

###  Endpoints for which no ef fects are seen in any studies are unfortunately excluded from evidence tables.  Negative information should be considered pertinent and informative to IRIS assessments. Should not be excluded from the toxicological review
. At a minimum, this information can help inform the database uncertainty factor
.

## ETBE and **tert -** **butanol Specific**  **Concerns (3)**   More clarity on endpoints in tables is needed.  It is not know if all endpoints from a positive study are included or if only the positive endpoints are included.  Criteria for extraction of information into the tables is necessary.  Cancer is treated dif

ferently in each assessment. Should it consistently get its own table and section ?  When percent change is shown, should also show the mean and standard deviation for controls.  Exposure
- arrays are misleading.  A quality review of the data is necessary before providing summary statistics and graphics.  Criteria for reviewing data should be developed and used before presenting information graphically
.

# Thank **Yo u!**   EP A has taken a strong first step. However improvements are needed.  Consistency with intent of IRIS process enhancements will go a long way towards improving the evidence tables.  Most importantly

, EP A must conduct a review of the quality and relevance of studies before moving them forward as ‘critical studies’ for an IRIS assessment.  Wi thout changes and improvements, the utility of evidence tables, following the current structure, may not help suf ficiently speed the finalization of IRIS assessments.  Small changes, such as those noted in our general recommendations slide will lead to much improvement.

# Knowing is not enough; we must apply
. Wi lling is not enough; we must do
.

- Johann Wo lfgang von Goethe