## Interim Guidance of the Evaluation Criteria for

Ecological Toxicity Data in the Open Literature

PHASES I and II
PROCEDURES FOR IDENTIFYING, SELECTING AND ACQUIRING TOXICITY DATA PUBLISHED IN THE OPEN LITERATURE FOR USE I

N ECOLOGICALRI SK ASSESSMENTS July 16, 2004 Office of Pesticide Programs U.S. Environmental Protection Agency2 

# Table of Contents

Table of Contents ................................ ................................ ................................ ............................. 2 Introduction ................................ ................................ ................................ ................................ ...... 4 Purpose ................................ ................................ ................................ ................................ . 4 Background ................................ ................................ ................................ .......................... 4 Phase I : Literature Acceptanc e Criteria for Effects Data ................................ ................................ . 5 Background on the ECOTOX Database ................................ ................................ .............. 5 ECOTOX Eligibility Criteria ................................ ................................ ................... 5 ECOTOX Literature Identification Procedures ................................ ........................ 5 Manual Literature Searches ................................ ................................ .......... 5 Electronic Literature Searches ................................ ................................ ..... 6 Identification of Relevant Literature ................................ ................................ .................... 6 Papers Eligible for the ECOTOX Database ................................ ............................. 6 EFED Acceptability

Criteria ................................ ................................ ........ 7 Data Summary

Tables ................................ ................................ .................. 7 Bibliographic Files ................................ ................................ ....................... 9 Papers Excluded from the ECOTOX Database ................................ ....................... 9 Phase II: Categorization of Studies and Determination of Use in Risk Assessment ..................... 11 Introduction ................................ ................................ ................................ ........................ 11 Studies Identified as Acceptable Based on Phase I Analysis ................................ ............. 11 Standardization of Data Summary Table ................................ ............................... 11 Data Comparison and Summary Tables ................................ ................................ 12 Studies with Concentrations/Doses of Concern ................................ ......... 13 Identification of Data for Taxonomic Groups not Normally Considered in RQ C alculations ................................ ................................ ............ 14 Identification of Data with Ecologically Relevant Effects not Typically Addressed in 40 CFR Section 158 ................................ ................. 14 Studies Excluded from the ECOTOX Database and EFED Screen ................................ .. 15 Determining the Use of Open Literature in the Risk Assessment ................................ ..... 16 Rationale ................................ ................................ ................................ ................ 16 Data Quality

Objectives ................................ ................................ ......................... 17 Determining if Study

Should be Used Quantitatively vs. Qualitatively ................ 17 Roles and Responsibilities ................................ ................................ ................................ ............. 20 ORD’s Role ................................ ................................ ................................ ........................ 203

EI SB’s Role ................................ ................................ ................................ ........................ 20 Risk Assessor’s Role ................................ ................................ ................................ .......... 20 Appendix I : ECOTOX QA/QC Support Documents ................................ ................................ ..... 22 Appendix II: Acceptability Criteria For Effects Data ................................ ................................ .... 23 Appendix III: Completion of the Data Summary Table ................................ ................................ . 26 Example Data Summary Table ................................ ................................ .......................... 29 Appendix IV. ECOTOX Exclusion Categories ................................ ................................ ............ 30 Appendix V. Example of Summary of Registrant- Submitted Data ................................ .............. 32 Appendix VI . EI SB Summary Table QA/QC Procedures ................................ ............................ 34 Appendix VII. Rationale for Ordering Papers ................................ ................................ .............. 37 Appendix VIII. Rationale for the Use of Open Literature in Risk Assessment ............................ 38 Appendix IX. Qualitative Study Summary ................................ ................................ ................... 3945 

## Introduction Purpose This guidance is for use by s cientists) in the U.S. Environmental Protection Agency ’s (EPA) Office of Pesticides Program (OPP). It is intended to ensure consistent consideration and use of information in the open literature by OPP risk assessors for ecological risk assessments of pesticide effects on non-target organisms. Use of this guidance will assist OPP risk assessors in screening papers in an efficient and consistent manner. This guidance, in final form, will be applied across the division to ensure consistency and transparency

in the risk assessment process. 

### Background Open literature can provide useful information in assessing the ecological risk of pesticides. F or most pesticides there is a paucity of research conducted beyond that required for registration. An exhaustive literature search for a lesser- known pesticide will frequently provide few

u seful papers for risk assessment. F or certain older pesticides, several hundred papers des cribing toxic effects may be available and many papers may provide similar information and results. Replication of results can provide confidence in conclusions. I n choosing whether or not to review or include a paper in a risk assessment, the risk asses sor should evaluate that paper’s potential to improve the risk assessment through adding confidence to the assessment conclusions or helping to reduce uncertainties. F or pesticides with many papers presenting similar conclusions, it may not be justified for the risk assessor to address all of the papers, but instead a significant sampling of them, with the desired result of presenting sound and confident risk conclusions without using unnecessary resources. T he intent of this guidance is to provide consistent and transparent methods for selection of appropriate studies, but it is also important that risk assessors execute best professional judgement, with documentation of the rationale, when choosing to include or exclude papers in the final risk assessme nt. The method described under the section entitled “Phase I: Literature Acceptance Criteria for Effects Data,”is intended to provide a coarse filter for identifying potentially useful ecotoxicology papers for OPP’s ecological risk assessments, thereby reducing the expenditure of resources on the review and evaluation of papers with a low likelihood of usefulness. F or papers that pass this acceptance criteria, a summary table of selected information from each study w ill be generated. U nder the section e ntitled “Phase II: Categorization of Studies”, the data provided in the summary table will be compared to OPP’s registrant- submitted data set with the intent of increasing confidence and decreasing uncertainties in the risk assessment conclusions.6



## Phase I: Literature Acceptance Criteria for Effects Data Background on the ECOTOX Database Staff scientists in OPP will consider data from the open li terature that the EPA Office of Research and Development’s (ORD) Mid - Continental Ecology Division (MED) locate using the search strategy developed for EPA’s Ecotoxicology database (ECOTOX). ECOTOX is a publicly available database summarizing the ecologica l effects of single chemicals to aquatic and terrestrial plants and animals ( http://www.epa.gov/ecotox ). ECOTOX Eligibility

Criteria Papers acquired using the ECOTOX literature searching and acquisition protocols must meet five minimum data requirements to be considered eligible for inclusion in the database. T he paper is eligible for the ECOTOX database if it reports (1) toxic effects related to a single chemical exposure; (2) toxic effects on an aquatic or terrestri al plant or animal species; (3) a biological effect on live, whole organisms; (4) a concurrent environmental chemical concentration/dose or application rate; and (5) an explicit duration of exposure. ECOTOX Literature Identification Procedures For the ECOTOX database, literature is identified through comprehensive and well documented literature searches (See http://lester.dul.epa.gov/qa/sop/#DAM for standard operating procedures) either manually (e.g., skimming of reference sections of review/summary articles and skimming literature held in ORD/MED’s library) or electronically (e.g., searches of electronic abstracting services such as STN, Cambridge Scientific Abstracts, Toxline). Q uality Assurance/Qu ality Control (QA/QC) procedures for the ECOTOX database are outlined in the document Overview of the Ecological Risk Assessment Process in the Office of Pesticide Programs, U.S. Environmental Protection Agency ( January 2004) and the support documents are itemized in Appendix I. Manual Literature Searches For papers identified using a manual literature search method, citations for potentially applicable articles are entered into an electronic bibliographic file. Abst racts are not available in the electronic bibliographic file for these papers. As articles are received, papers that meet the minimum data requirements for ECOTOX are assigned an ECOTOX reference number and a copy of the paper is maintained in the ECOTOX file room. ORD/MED also maintains copies of some non - applicable papers that are useful to the ECOTOX database effort, and these are also assigned an ECOTOX reference number.7

These papers include review or summary articles that are not the primary source of data but are used in manual literature searches, and papers that document test method procedures. All papers that are received and identified as non - applicable, including any retained in the ECOTOX file room, have annotations added to the electronic bi bliographic entry documenting the reason the paper was excluded from consideration for the ECOTOX database effort. C opies of non - applicable papers, other than methods and review articles, are discarded. Electronic Literature Searches For papers identified using an electronic literature search method, the citation and abstract information are downloaded electronically into a bibliographic file. Each citation and abstract are reviewed to determine if the paper is p otentially applicable to the ECOTOX database. W hen a paper is identified as not applicable to ECOTOX, the entry is annotated with the reason the study was excluded, and a copy of the paper is not acquired. The citation and abstract are moved to a separate electronic bibliographic file that stores citations for all papers identified via the ECOTOX electronic literature search method, which were never acquired. C itations for papers (excluding the abstract) that are identified as potentially applicable are moved to another electronic bibliographic file that tracks papers that have been ordered via the ORD/MED library or other sources, and the paper is acquired. As papers are received, the ECOTOX eligibility criteria are applied, and citations for any papers that are identified as not applicable to ECOTOX are transferred to another bibliographic file that tracks papers that were received and subsequently rejected. A ll citations for rejected studies are annotated with the reason for exclusion, and the paper i s not retained. A ll papers that are received and identified as applicable are assigned an ECOTOX reference number and placed in the ECOTOX file room. 

### Identification of Relevant Literature The results that O RD/MED obtains through the ECOTOX literature search can be divided into two broad categories:

· Papers identified as applicable to the ECOTOX database (i.e., those that meet the five minimum eligibility criteria); and

· Papers located using the ECOTOX search strategy but rejected for entry into the ECOTOX database Papers Eligible for the ECOTOX Database Wi thin the category of papers identified as acceptable to the ECOTOX database effort, are two sub - categori es: (a) papers with data available in the online version of8

ECOTOX; and (b) papers that are in the queue for data abstraction and inclusion in the ECOTOX database. A ll papers identified as applicable to the ECOTOX database, with the exception of the OPP d atabase (ECOTOX Reference Number 344), will undergo an acceptability screen by ORD/MED to discriminate between papers that are potentially applicable to the OPP risk assessment (i.e., papers that meet all the criteria) from those papers that have a low pro bability of usefulness in the OPP risk assessment (i.e., papers that do not meet one or more of the criteria). OPP Acceptability Criteria The OPP Technical Teams identified criteria that will assist in eliminating pape rs that have a low probability of usefulness in the hazard identification component of a screening level risk assessment. See Appendix II for the data form and guidance on determining the acceptability of an ecotoxicology study. Papers must meet all of t he following Acceptability Criteria:

· The paper reports toxicology information for a chemical of concern to OPP;

· The article is published in English language;

· The study is presented as a full article. Abstracts will not be considered;

· The paper is a publ icly available document;

· The paper is the primary source of the data;

· The paper reports that treatment(s) were compared to an acceptable control;

· The paper reports an explicit duration of exposure;

· The paper reports a concurrent environmental chemical con centration/dose or application rate;

· The paper reports the location of the study (e.g., laboratory vs. field);

· The paper reports a biological effect on live, whole organisms;

· The paper reports the species that was tested; and this species can be verified i n a reliable source;

· The paper reports effects associated with exposure to a single chemical. Data Summary Tables Once the OPP acceptance criteria have been applied to all the papers meeting the ECOTOX eligibility requirements, ORD/MED will summarize the data associated with the9

potentially acceptable studies for use in the screening level risk assessment. F or each paper that passes the Acceptance Criteria, ORD/MED will summarize the parameters listed below in separate fields within a Data Summary Spreadsheet, and provide the results in an electronic format to the appropriate staff person within OPP’s Environmental Information Services Branch (EISB). See Appendix III for instructions on the completion of the Data Summary Tabl e and an example table.

· Chemical identification information

· Chemical Abstract Services registry number

· common name of chemical

· Taxonomic information including:

· phylum

· class

· order

· family

· genus

· species

· species common name

· Effect information including:

· major effect category (e.g., reproduction)

· specific result measurement (e.g. number of viable eggs)

· calculated endpoint (e.g., LC50)

· Species habitat (aquatic and/or terrestrial)

· Test location (laboratory vs. field)

· Number of treatment levels

· Concentrat ion/dose/application rate associated with the observed effect response

· Concentration units (e.g., ug/l)

· Concentration type (e.g., active ingredient vs. formulated)

· Percent purity

· Media (freshwater vs. marine)

· Duration of the study

· Duration units (e.g., h ours)10

· For avian and mammalian studies, identify the route of exposure (e.g., dietary, oral, etc.)

· pH ranges

· Hardness

· Organic Carbon Concentrations

· Remarks

· ECOTOX reference number Bibliographic Files The following proce dures will be followed for all papers identified as acceptable to the ECOTOX database:

· ORD/MED will provide OPP/EISB staff with a ProCite file that includes citations for all papers identified as acceptable to the ECOTOX database effort;

· Each entry will i nclude an ECOTOX reference number;

· Each entry will include complete citation information;

· Studies that pass the acceptance criteria will be marked as acceptable;

· Each entry will identify the paper’s coding status at it relates to the ECOTOX database (i.e. , papers that are in the public version of the ECOTOX database and papers that are in the queue);

· Each citation will identify whether the paper presents data for aquatic and/or terrestrial species;

· Each citation will include a code identifying the chemical (s) For studies that did not pass the acceptance criteria, each entry will include a code that denotes the specific criterion/criteria that the paper did not meet. Papers Excluded from the ECOTOX Database ORD/MED maintains electronic bibliographic files of all rejected papers that include codes documenting the reason the paper was rejected. See Appendix IV for a11

list of exclusion criteria used within the scope of the ECOTOX database. The following proce dures will be followed for these excluded studies:

· ORD/MED will provide OPP with a ProCite database of citations and abstracts for papers that were rejected, and never acquired. The compilation will include the following:

· Citations for papers that have been retained in the ECOTOX literature holdings, but that do not meet the eligibility requirements;

· Citation for papers identified as potentially applicable to the ECOTOX effort, which were rejected once the full article was acquired;

· Citation and abstract, when available, for publications excluded based on information provided in the literature search (i.e., without acquisition of the paper)

· All records will include the following information:

· The reason the paper was not considered applicable to the ECO TOX database;

· All available citation information;

· Abstracts when available (Note: These will only be available for electronic search results rejected prior to acquisition of the full publication.)12



## Phase II: Categorization of St udies and Determination

of Use in Risk A

ssessment

` 

### Introduction Although registrants are required to submit data on the toxicity of pesticide s, data from the open literature can provide additional information that may improve the risk assessment through adding confidence to the assessment conclusions, reducing uncertainties, and identifying additional effects that will require further data and analysis. The purpose of Phase II of this guidance is to provide OPP risk assessors an efficient and consistent manner in which to screen the open literature studies that may be used in ecological risk assessments. T he risk assessor will provide the rati onale for the inclusion or exclusion of literature in their risk assessments. 

### Studies Identified as Acceptable Based on Phase I Analysis The process outlined in this section of the gui dance document was developed to provide OPP scientists an efficient way to locate studies reporting toxic responses that are more sensitive than those reported in the registrant submitted data sets, to quickly identify receptors and ecologically relevant e ndpoints that are not contained in current EPA guideline studies, and identify, when possible, studies reporting direct toxicity i nformation on endangered species. W hile guidance has been developed for the selection of appropriate open literature for po ssible inclusion in a risk assessment, the exercise of best professional judgement is also required. S cientists should use their best professional judgement on whether to include open literature and if necessary consult with the appropriate technical team . It is important that the risk assessor document the rationale associated with all choices made regarding open literature data in the assessment. This documentation must follow the Agency “TCCR” rules for T ransparency i n p rocess followed, and C larity , C onsistency and Reasonableness in its ecological risk assessments.13

Standardization of Data Summary Table In order to categorize data effectively, the data summary file must be in a standardized format. Th e following manipulations will be made by ORD/MED staff to the Data Summary Table:

· For studies reporting pesticide concentrations based on formulated product and/or the chemical was not measured in the test media (i.e., nominal concentrations), the values will be converted to a concentration/dose based on active ingredient

· For pesticides not reporting measured chemical concentration/dose, the percent purity will be used to convert the value to the active ingredient

· For metals reporting concentrations (or doses) based on the amount of compound used (e.g., mg/L of copper sulfate), the molecular weight of the compound will be used to convert to elemental metal (e.g., mg copper/L)

· All effect concentrations/doses will be converted to a standard unit:

· For aqua tic studies (fish, invertebrate, algae and plant species) convert to mg/L for aqueous phases and mg/kg (dry weight) for matrices such as sediment

· For avian, reptilian and mammalian dietary studies convert to ppm

· For avian oral studies and mammalian oral and intravenous studies, convert to mg/kg bw

· For aquatic phase amphibians, convert to mg/L; for terrestrial phase amphibian oral studies, convert to mg/kg bw; for terrestrial phase amphibian dietary studies convert to ppm

· For reptilian oral and dietary stu dies, convert to mg/kg bw

· For inhalation studies, report in units presented in the study

· For dermal studies, report in units presented in the study

· All durations will be converted to days The following tables will be generated by EISB staff from the Data Summary Tables received from ORD/MED:

· Each Data Summary Table will be separated into taxonomic groups

· Create a freshwater fish table

· Create a saltwater fish table

· Create an algae table

· Create a freshwater invertebrate table

· Create a saltwater invertebrate table

· Create an aquatic plant table

· Create a terrestrial plant table

· Create a mammal table

· Create a bird table14 · Create an amphibian table

· Create a reptile table Data Comparison and Summary Tables As needed, the EI SB staff will apply the following criteria to develop comparison tables. Prior to preparing comparison tables, the EI SB team should have the results from the Data Evaluation Reviews (DER) available from the Risk Assessor. It will be the Risk Asse ssor’s responsibility

to provide the summarized DER results in a formatted table (see example template table in Appendix V). EISB staff will provide the risk assessor with the following three comparison tables and the original Data Summary Tables received from ORD/MED. QA/QC procedures for the development and production of summary tables by EI SB are located in Appendix VI . Table 1: Studies with Concentrations/Doses of Concern : The purpose of this procedure is to identify studies associated with a taxonomic group that report more toxic endpoints than results summarized in registrant- submitted data (i.e. data submitted in support of registration or re -registration actions according to 40 CFR Section 158). This procedure potentially relates to the following taxonomic groups: Class Osteichthyes; all plant studies; Class Branchiopoda; Order Rodentia with oral or dietary routes of exposure; and Class Aves with oral or dietary routes of exposure.

· For each taxonomic category with registrant- submitted test data, the toxicological endpoint and the exposure duration associated with that endpoint (acute vs. chronic) will be identified. See Appendix V for an example of how to summarize the data;

· For each taxonomic category, effects that are not included in the registrantsubmitted data set are removed;

· For birds and mammals, all exposure routes other than dietary or oral are removed;

· For invertebrates, all terrestrial species are removed;

· Data removed from Table 1 by this process will be placed in Table 2 or 3.

· The Data Summary Table will be sorted in descending order by the concentration field;

· The row in the Data Summary Table will be marked where measurement endpoint values above the line are greater (less sensitive) than the values provided in the15 registrant- submitted studies, and data below the line are less (more sensitive) than the registrant- submitted data;

· The risk assessor will review this table and order all papers below the line (i.e., more toxic than the registrant- submitted data). This information will be communicated to EI SB staff and the papers will be ordered from ORD/MED;

· Risk assessors should use best professional judgement when identify ing additional papers above the line for acquisition from ORD/MED; however, documentation is required as to the rationale underlying the decision (see Appendix VII for documentation for rationale for ordering and not ordering papers). Table 2: Identification of Data for Taxonomic Groups not Normally Considered in RQ Calculations The purpose of this procedure is to identify potentially

useful toxicity

data/measurement endpoints on taxonomic groups that are not addressed by registrant- submitted data (i.e. data submitted in support of registration or re -registration actions according to 40 CFR Section 158). This procedure relates to the following species groups: mammalian data with the exception of rodent studies and all data on terrestrial invertebrates, reptiles, and amphibians. The resulting tables will be sorted by effect and concentration. OPP scientists will review the tables and identify studies of concern. Consideration should be given to sensitive species groups and ecologically relevant effects not ty

pically addressed in an OPP risk assessment. Scientists should use best professional judgement to identify papers of concern; however, documentation is required as to the rationale supporting the use or exclusion of studies (see Appendix VIII). The reprint request will be communicated to EI SB staff and the papers will be ordered from ORD/MED. Table 3: Identification of Data with Ecologically Relevant Effects not Typically Add ressed in 40 CFR Section 158 The purpose of this procedure is to identify test results for ecologically relevant endpoints that are not addressed by

registrant- submitted data. This procedure potentially relates to the following taxonomic groups: Class Osteichthyes; Class Branchiopoda; all plant studies; Order Rodentia; and Class Aves.16

If there were no registrant- submitted data for the taxonomic group, the resulting table will be sorted by

effect and concentration. For each taxonomic category with registrantsubmitted test data, the effects that are included in the registrant- submitted data set will be removed. OPP risk assessors will review the tables and identify studies of concern . Consideration should be given to sensitive species groups and measurement endpoints not ty pically addressed identified in guideline studies. Scientists should use best professional judgement to identify

papers of interest, however documentation is required as to the rationale supporting in inclusion or exclusion of open literature studies (see Appendix VIII); Paper requests will be communicated to EI SB staff and ordered from ORD/MED. 

### Studies Excluded from the ECOTOX Database and OPP Screen ECOTOX- rejected studies that may be of interest to the OPP risk assessor will include papers related to modeling, monitoring, mixtures, in v itro studies, incident reports, fate studies, and review articles. Scientists should use best professional judgement to identify papers of interest, however documentation is required as to the rationale supporting their use or exclusion. ECOTOX rejection codes may be cited to explain exclusion of literature or after review, the risk assessor may use best professional judgement for exclusion (Appendix VIII) .

· ORD/MED will provide OPP with a ProCite database of all citations for papers that were reviewed an d subsequently rejected. Each citation will be annotated with the reason the paper was excluded from consideration.

· These papers are not retained at MED, but as papers are acquired and rejected, those relevant to chemicals under review will be sent via s urface mail to EISB upon OPP’s request based on criteria discussed below.

· EISB will use ProCite to print a bibliography of the rejected papers associated with each chemical of concern for the OPP risk assessment teams.

· The bibliography will contain a fi eld entitled “KEYW ORD” which reports the coded rationale for why the paper was rejected for ECOTOX (e.g. REVIEW , NON - ENGLISH, PUBL AS, etc.; see Appendix I).17

· OPP scientists will use these rejection codes as a guide to filter out papers that they wish to exclude from further consideration in their risk assessment. It is important to remember the five minimum eligibility criteria that must be met in order to be included in ECOTOX (chemical, species, effect, concentration/dose, and exposure must be provide d in the paper) since these are also important elements of reporting for OPP.

· OPP staff will review the bibliography of rejected papers to identify those likely to provide useful information for the risk assessment. In evaluating citations and abstracts (which will be provided when available), the risk assessor will consider the same OPP acceptance criteria outlined above. Wh en considering papers not meeting ECOTOX inclusion criteria, the risk assessor should consider the following questions: Is the pa per likely to provide useful information on

· the effects of exposure to other pesticides?

· the effects of other environmental stressors in combination with the pesticide?

· information on population - level effects?

· exposure or toxicity to pesticide degradates ?

· If the risk assessor determines that the papers could address uncertainties or provide other useful information in assessing pesticide risk, reprints will be requested from ORD/MED through EISB. 

### Determining the Use of Open Literature in the Risk Asses sment Rationale Reprints from ORD/MED will be distributed to the risk assessors by EISB staff. Based on the selection of requested literature from the summary tabl es, the risk assessor may encounter effects data that can provide additional information on existing measurement endpoints commonly used in the screening risk assessment, insight on endpoints not routinely considered in risk quotient calculations, and effe cts data on non - guideline receptors (e.g., amphibians, mussels, etc). It is the risk assessor’s responsibility to closely review the study. Professional judgement is used and documented by the risk assessor to determine whether and how available data on other toxicological endpoints are included in the risk assessment. This evaluation may include reference to data quality objectives (see below) for specific types of studies, the degree to which adequate documentation is available to evaluate18

the technical merit of the study, and whether the study’s measurement endpoints are relevant to the assessment endpoints established for ecological risk assessment. To decide if data are applicable to assessment endpoints, the risk assessor uses best professional ju dgement and available lines of evidence to determine if the toxicological endpoints can be linked to assessment endpoints in a reasonable and plausible manner. There must be a determination of whether there are clear, and reasonable links between the effect and survival or reproductive capacity of organisms in the field in accordance with the screening assessment endpoints of survival and reproductive capacity. It is imperative that the risk assessment process be transparent and, to that end, that risk a ssessments themselves be clear, consistent and reasonable relative to other risk assessments of similar scope. The risk assessment must clearly document the criteria used in selecting additional literature and its potential effects on the confidence of th e overall risk assessment conclusions. Data Quality

Objectives Defining Best Scientific and Commercial Data Available Guidelines were issued by the Office of Management and Budget (OMB) in response to the Data Quality Act mandated by Congress in Section 515(a) of the Treasury and General Government Appropriations Act for FY 2001 (Public Law 106 - 554; H.R. 5658). The EPA then developed the policy and procedural guidance Guidelines for Ensuring and Maximiz

ing the Qualit

y, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency (EPA/206R- 02 - 008, October 2002). The following criteria apply to defining best scientific and commercial data available:

· Quality is a term that encompasses the other criteria below. The overall quality of a study must be determined by evaluating these additional criteria.

· Utility refers to the usefulness of the literature to its intended purpose and use. The studies, data, information and methods m ust only be relied upon to the extent their use is scientifically justified.

· Objectivity refers to whether the information in the study is being presented in an accurate, clear, complete, appropriate and unbiased manner.19

· Transparency refers to the clarity of the process. Are the uncertainty and error sources, assumptions, statistical methods, and justification identified? A high degree of transparency facilitates reproducibility by third parties.

· Quantity refers to study design. Is the magnitude of the effe ct, number of studies, number of replicates, sample size, or power identified?

· Consistency refer to the extent similar findings are reported using similar study design and methods.

· Integrity refers to scrutiny to ensure the information is not compromised t hrough alteration or improper interpretation. EPA data quality objectives are met by OPP’s current procedure of evaluating studies through the data evaluation record (DER) process. Studies are identified as core, supplemental, or invalid. Studies classi fied as invalid cannot be used for risk assessment purposes. Open literature that will be used quantitatively will undergo the same level of scrutiny as registrant- submitted data. Determining if Study

Should be Used Quantitatively vs. Qualitatively To determine if a study should be used quantitatively versus qualitatively, one question may be posed: 

### Does the study show clear and reasonable links between the effect and survival or reproductive capacity? Quantitative Use in Calculating Risk Quotients If the answer to this question is “yes”, decisions are to be made regarding the quality of the data and their utility in the risk assessment (e.g., a review of the validity and reliability of study protocols), relative to the Agency’s risk assessment guidance. The extent to which such additional data are either employed or rejected should be clearly described. Regardless of the amount of data beyond the guideline - required toxi city studies, the screening - level risk assessment relies on selecting the most sensitive species tested from acceptable studies. Acceptable studies are determined on the basis of the design and conduct of the experiment from which the data were derived. In evaluating the experimental design, OPP scientists will consider methods generally recognized as acceptable by the scientific community (e.g. ASTM), following reasonable statistical design and rigor, and the proper use of controls in all phases of the experiment. The study is evaluated to20

determine whether it conformed to its experimental design. The review of the study is documented in the contractor- prepared DER. If the DER deems the study to core or supplemental and the paper has identified a more se nsitive toxicity value, the study may be used to determine risk quotients. Qualitative and Descriptive Use in Risk Characterization If the answer to the question is “no” and the study is of acceptable quality and addresses other endpoints of concern to the risk assessment , the study may be used descriptively (qualitatively) in the risk characterization. Additionally a paper not used in quantitative risk quotient calculations may still be a seminal document for line of evidence reasons (e.g. endangered sp ecies concerns). Therefore, it may be beneficial to document the rationale for the study’s use in the assessment by preparing a qualitative study summary to be archived in the chemical file (see Appendix IX for a qualitative study summary template). The risk assessor will use established guidance documents such as the Guidelines for Ecological Risk Assessment (EPA/630/R- 95/002F, April 1998), Risk Characterization Handbook (EPA/100 - B - 00 - 002, December 2000), and the Peer Review Handbook, 2nd Edition (EPA/ 100 - B - 00 - 001, December 2000) to prepare the risk characterization. In addition, the risk assessor will follow the process as outlined the document Overview of the Ecological Risk Assessment Process in the Office of Pesticide Programs, U.S. Environmental Protection Agency ( January 2004). Clear, Consistent and Reasonable Rationale for Use of Papers In accordance with established risk assessment guidance, the risk assessor will identify in the risk assessment:

· The measurement endpoints from the literature that were considered in the risk assessment;

· The rationale for why such data were incorporated in the risk assessment;

· The rationale for not including data obtained from the literature Appendix VIII provides a list of rationales for the inclusion and exc lusion of literature in the risk assessment.21 

## Roles and Responsibilities ORD’s Role Incooperation with OPP, ORD/MED will do the following tasks:

· Apply

the OPP acceptance criteria outlined above;

· Summarize the effects data for acceptable studies in a Data Summary Table as outlined in this document;

· Convert the Data Summary Table to a standard format;

· For studies that do not meet one or more of the OPP acceptance criteria, ORD/MED will document in the electronic bibliographic file the specific criterion/criteria that were not met;

· ORD/MED will provide OPP/EI SB with reprints as outlined in this document;

· ORD/MED will provide ProCite files including citations for all studies identified in ECOTOX literature searches including studies that meet the OPP acceptance criteria, studies that do not meet the OPP acceptance criteria, and papers that did not meet the ORD/MED ECOTOX database inclusion criteria. 

### EISB’s Role OPP’s EIS B is responsible for the following tasks:

· EI SB will serve as the OPP repository for reprints provided by

ORD;

· EI SB will generate tables of different taxonomic groups from the standardized data summary

tables provided by

ORD/MED;

· EI SB will categorize the data presented in the Data Summary Table following procedures outlined in this document, and EI SB will provide to the risk assessors the following results related to the chemical of concern:

· Results for taxonomic groups included in the registrant- submitted data

· Results for non-target species not addressed in the registrant- submitted guideline studies;

· Results for observed effects not addressed in registrant- submitted data

· EI SB will provide summary tables and database to OPP risk assessor;

· EI SB staff will coordinate all communication with ORD/MED staff related to the tasks outlined in this document. 

### Risk Assessor’s Role The risk assessor is responsible for the following tasks:22

· Providing EI SB with summary of toxicity

results from Data Evaluation Records;

· Communicate to EI SB staff which papers should be ordered from ORD/MED;

· Reviewing the papers and bibliographies in terms of their utility relative to ecological assessment endpoints;

· Using best professional judgement in deter mining the utility

of papers in strengthening the risk assessment conclusions and reducing uncertainty;

· Describing the method used to evaluate open literature and the rationale for either including or excluding it, for use in the risk assessment.

· Prepare a summary sheet (Appendix IX) for studies used qualitatively in the risk assessment.23



## Appendix I: ECOTOX QA/QC Support Documents #72 Section 1. Database and Documentation Overview. MED ECOTOXICOLOGY DATABASE SOP’s. September 1997. #73 ECOTOX. ECOTOXicology Database System. Literature Search and Citation Identification. MED ECOTOXICOLOGY DATABASE SOP’s. April 2001. #74 ECOTOX. ECOTOXicology Database System. AQUIRE Coding Guidelines. August 2003. #75 ECOTOX Data Entry Procedures (AQUIRE). MED ECOTOXICOLOGY DATABASE. June 2000. #76 ECOTOX. ECOTOXicology Database System. Chemical Verification and Database Entry Procedures (EcoChem). April 2001. #77 ECOTOX . ECOTOXicology Database System. Taxonomic Name Verification Procedures (CRITTERS). September 2001.24



## Appendix II: Acceptability Criteria For Effects Data ECOTOX Reference No.: ____________ General instructions : If more than one experimental design is used in the study, multiple Literature Acceptance Criteria Checklist forms may b e required, but the acceptability o f the paper is based on at least one experimental design meeting all the Acceptability Criteria. No. Criteria / Instructions Yes / No 1 The paper reports toxicology information for a chemical of concern to OPP. Check chemical list for chemicals tested. If a chemical in appears on the list answer Yes. If the chemical is not on the list answer No. 2 The article is published in the English language. The full article is published in the English language. Translations will not be conducted. 3 The study

is presented as a full article. Abstracts from journal publications where the full article is published in non -English, the abstract is published in English and conference proceedings published as brief abstracts will not be considered. 4 The paper is a publicly available document. Publications that are not publicly available; i.e., internal memoranda, government reports that are not available from NTIS or other government sources will not be considered. 5 The paper is the primary source of the data. A document is considered a primary source if at least one of the investigators who conducted the toxicity test is an author, and the authors do not cite another publication as the original source of the data. 6 The paper reports a calculated endpoint. For the purposes of th is evaluation, an endpoint is defined as the quantification of an observed effect obtained through statistics or other means of calculation for the expressed purpose of comparing equivalent effects (e.g., LC50, BCF, NOAEL). If within a single experiment, the authors report the same endpoint at multiple durations, only code the duration most relevant to the OPP SEPs (see Table below), and note the other endpoints in the remarks field. If NOEC/NOEL and LOEC/LOEL25 endpoints are not explicitly reported by the authors, reviewers should interpret endpoints based on levels of significance reported in the paper. If more than one measurement (e.g., juveniles per litter, juveniles per females, etc.) is observed for a particular effect (e.g., reproduction), then only the most sensitive measurement is coded, and the remaining measurements are noted in the remarks field. 7 The paper reports that treatment(s) were compared to an acceptable control. The control treatment must be comparable to the other treatmen ts, and must be free of the chemical stressor. Appropriate controls include: baseline or background control - parameters of actual or representative test species measured before and after administration of test chemical, though not as part of the same test scenario; negative control - organisms maintained under conditions identical to exposed organisms except for the absence of the test substance; positive controls - organisms maintained under conditions identical to the exposed organisms except the test substance is replaced with a substance known to elicit a consistent toxic response; and solvent controls - organisms exposed to carrier or solvent that is used as a vehicle for administrating the test substance to exposed organisms. The number of treatments (other than controls) should be reported in the data summary table. 8 The paper reports an explicit duration of exposure. Authors must explicitly report the duration of the exposure related to the observed effect. Durations can include qualitative terms (e.g., at hatch, at harvest). 9 The paper reports a concurrent environmental chemical concentration/dose or application rate. Authors clearly report a concentration/dose or application rate associated with the observed effect respon se. If the study results are only available in a graphical format, add a comment to the remarks field. 10 The paper reports the location of the study (e.g., laboratory vs. field). Authors clearly state the locations of the study, either in a con trolled laboratory setting or in the fields. Field studies can include natural or artificial settings (e.g., outdoor pen studies; enclosures). 11 The paper reports a biological effect on live, whole organisms. The authors clearly identify an ob served effect response related to the exposure of a live organism to the chemical of concern. In vitro studies are not considered. Positive effects will be recorded, with a note in the26 remarks field flagging the response as such. 12 The paper reports the species that was tested; and this species can be verified in a reliable source. The authors clearly identify the test species and the organism’ s scientific name can be verified in a reliable reference. The preferred scientific name should be reported. If a specific genus/species is not reported, reviewers should code the species information at the lowest taxonomic level. 13 The paper reports effects associated with a single chemical exposure. Tests concerning synergism, additivity, p otentiation, or antagonism, whether it involves two or more chemical stressors or a combination of chemical and non
chemical stressors are not acceptable studies under this criteria. Effluents, leachates, drilling muds, fly ashes, sediments, and sludges are not considered single chemicals. In addition, the single chemical cannot be introduced as a component of an effluent, etc. Formulated products, such as emulsifiable concentrates and wettable powders are considered single chemicals. Also, chemicals that are mixtures of structurally similar organic chemicals are considered single chemicals because the mixtures have similar biological, chemical, physical, and toxicological properties (e.g., toxaphene).27



## Appendix III: Completion of the Data Summary Tab le General Instructions: All entries should be made in an Excel spreadsheet using version 9.0 (Excel 2000) or higher. Each sub - category within a general parameter must be data entered into a sep arate data field within the spreadsheet. Entries should be based on information provided by the authors, unless clarified in the instructions for each parameter. No. Parameter Instructions 1 Chemical Identification In separate data fields, enter the Chemical Abstract Services registry number and the chemical name as listed in Appendix III. 2 Percent Purity Percent purity of compound tested as reported by the authors. 3 Taxonomic Identification In separate data fields, enter the following taxonomic information:

ECOTOX species number, phylum, class, order, family, genus, species, and the common name of the organism involved in the exposure. 4 Habitat Identify whether the study

is an aquatic or terrestrial study. 5 Test Location In a sep arate data field, enter the test location using codes presented in Appendix H of the ECOTOX Code Appendix (www.epa.gov/ecotox/codeappendix.pdf). 6 Media The ty pe of exposure media reported using codes presented in Appendix L of the ECOTOX Code Appendix (www.epa.gov/ecotox/codeappendix.pdf). 7 Route of Exposure Identify the route of exposure using codes presented in Appendix J of the ECOTOX Code Appendix (www.epa.gov/ecotox/codeappendix.pdf). 8 Number of concentrations/ doses Enter the number of treatment levels (other than controls) that were used in

the study. 9 Duration The exposure duration associated with the observed effect, as reported by the authors. 10 Duration Units Units associated with the reported duration. 11 Observed Effect and Calculated Endpoint Inseparate data fields, enter the major group effect, observed effect,

and measurement codes presented in Appendix S of the ECOTOX Code Appendix (www.epa.gov/ecotox/codeappendix.pdf ). In a separate data field enter the appro priate calculated endpoint using the codes presented in Appendix T of the ECOTOX Code Appendix28 (www.epa.gov/ecotox/codeappendix.pdf). If more than one measurement (e.g., juveniles per litter, juveniles per female, etc.) is observed for a particular effect (e.g., reproduction), then only the most sensitive measurement is coded, and the remaining measurements are noted in the remarks field. If within a single experiment, the authors report the same endpoint at multiple durations, only code the duration most relevant to the OPP SEPs (see Table below), and note the other endpoints in the remarks field. 12 Concentratio n Type Report if the chemical concentration is based on active

ingredient or formulation. If chemical concentration is reported as measured, the concentration type should be coded as active. If chemical concentration is reported as nominal, concentration type should be coded as formulation. If the authors do not clearly state whether or not the chemical concentration was measured or whether t he reported concentrations are based on formulated product vs. active ingredient, code as formulations, but make a comment in the remarks field that concentration type could not be determined from paper. 13 Concentratio n, Dose, or Application Rate In a separate data field, enter the chemical concentration,

dose, or application rate associated with the observed effect. If both a concentration and application rate are presented, only report the concentration. If a dose is presented and a concentration, only present the dose. In separate data fields enter the unit associated with the concentration/dose/application rate entry, the concentration type (F if formulated; A if active ingredient); and the percent purity of the compound as reported by the autho rs. If the authors clearly state that the compound is measured in the test matrix, enter the concentration type as ‘A’. 14 Remarks General comments on other endpoint durations, graphical data

coding, concentration type, and positive effects. 15 Reference Number In a separate field, enter the ECOTOX reference number for

the paper.29 

### Example Data Summary Table CAS # Chemic al Nam e Phylum Class Order Family Genus Species Common nam e Effect Meas Endp Media pH O. C . Hard ness Dur Dur

Unit Conc Type Conc Conc

Units Ref # 298000 Methyl parathion Arthropoda Maxillopoda Calanoida Acartiidae Acartia tonsa Calanoid copepod BEH EQUL EC50 SW 96 h A 890 ug/L 742 298000 Methyl parathion Arthropoda Insecta Diptera Culcidae Aedes nigromaculi s Mosquito MOR MORT LC50 FW 24 h A 0.008 ppm 4431 298000 Methyl parathion Chordata Actinoptergyii Siluriformes Ictaluridae Ameiurus melas Black bullhead MOR MORT LC50 FW 24 h F 7910 ug/L 6797 298000 Methyl parathion Chordata Acti noptergyii Siluriformes Ictaluridae Ameiurus melas Black bullhead MOR MORT LC50 FW 96 h F 6640 ug/L 6797 298000 Methyl parathion Chordata Actinoptergyii Cyprinodontiform es Poecillidae Gambusia affinis We stern mosquitofish BEH THML NR FW 48 h A 5 ug /L 5149 298000 Methyl parathion Chordata Actinoptergyii Cyprinodontiform es Poecillidae Gambusia affinis We stern mosquitofish MOR MORT NR FW 24 h F 5000 ug/L 776230



## Appendix IV. ECOTOX Exclusion Categories General Instructions: The following is a list of ECOTOX rejection codes, exclusion terms and definitions utilized under the ECOTOX database efforts. Each citation that is identified as not applicable to the ECOTOX database (i.e., NON-APPLICABLE entered in field #37 of the ProCite bibliographic file) will have one or more of these codes entered in the KEYWORD field (number 45) of the ProCite bibliographic files to identify the reason the study

was excluded.
Code Used in ProCite File Description REVIEW All toxicity tests reported elsewhere. If the publication is applicable to one of the ECOTOX databases, the bibliography is skimmed and any applicable articles are ordered. ME THODS Methods paper with no usable toxicity tests. Reports of methods of conducting tests, determination or purification of chemicals, etc. Methods publications are selected to be ordered for the ECOTOX toxicology methods information file. PUBL AS Author publishes data in multiple formats. For instance, an author pu blishes the same study results in a Ph.D. thesis and in a peer-reviewed journal. In order to reduce redundancy of data in the ECOTOX database, only one document is encoded; with preference given to peer-reviewed publications.. NON-ENGLISH or FORE Paper is published in a foreign language. Modeling Paper reports models only, no new organism exposure data. Modeling studies may report original toxicity tests performed as comparisons or as a basis for extrapolation; order the paper if it is not clear from the abstract. Air pollution Air pollution and emissions, no specific chemical. Other ambient conditions Effects on organisms from changes in conditions other than addition of chemicals, including radioactivity, ultraviolet light (UV), temperature, pH, salinity, dissolved oxygen (DO), or other water, air, or soil parameters. Biological Toxicant Biological toxicants; includes venoms, fungal toxins, Bacillus thuringiensis , other plant, animal, or microbial extracts or toxins. Drug Testing for dr ug effects and side - effects specific to human health. Effluent Studies reporting effects associated with exposure to effluent, sewage, or polluted runoff. Mixture Mixture study without data on single chemical exposures. Nutrient Nutrient studies -- in situ chemicals tested as nutrients. Oil Studies with oil and petroleum products as the only test chemical.31

No Species No organism present or tested, unable to verify a species, or exposure of dead organism. In Vitro In vitro studies, including exposure of cell cultures and excised tissues. Bacteria Bacteria as test organism, including Microtox tests, or other microbial organisms. Yeast Yeast as the test organism. No Tox Data Publications w hich are not toxicology studies; no toxicity data are presented in the paper. Human Health Human health effects; studies with human subjects or with animal subjects as surrogates for human health risk assessment. No Conc No usable dose or concentration reported; identified after examination of full paper. Includes lead- shot studies which lack dose information or give only number of pellets. Also includes papers with concentrations reported only in log units. Sediment Conc Chemical concentration reported in sediment. Sediment studies are coded if a water concentration of the added chemical is also reported; order the publication if unclear from the abstract. No Duration Duration is not reported, identified after examination of full paper. Incident Report of incidents --reports of animal deaths by poison, etc. Lacks usable concentration or duration or both. Survey Survey studies --measuring amounts of chemical present, but no usable quantification of exposure. Lacks either usable concentration or duration or both. Fate Studies reporting only what happens to the chemical in abiotic matrices Food Studies Food studies, no chemical and effects information are reported32



## Appendix V. Example of Summary of Registrant- Submitted Data Aquatic Animals Acute Assessment Lowest tested EC50 or LC50 for freshwater fish Lowest tested EC50 or LC50 for freshwater invertebrates Lowest tested EC50 or LC50 for estuarine/marine fish Lowest tested EC50 or LC50 for estuarine/marine invertebrates Chronic Assessment Lowest NOEC for freshwater fish Lowest NOEC for freshwater invertebrates Lowest NOEC for estuarine/marine fish Lowest NOEC for estuarine/marine invertebrates Terrestrial Animals Acute Avian Assessment Lowest LD50 (single oral dose) Lowest LC50 (subacute dietary) Chronic Avian Assessment Lowest NOEC for 21 -we ek avian reproduction test Acute Mammalian Assessment Lowest LD50 from single oral dose test Chronic Mammalian Assessment Lowest NOEC for two - generation reproduction test Plants Terrestrial non- endangered Lowest EC25 values for seedling emergence for monocots Lowest EC25 values for vegetative vigor for monocots33 Lowest EC25 values for seedling emergence for dicots Lowest EC25 values for vegetative vigor for dicots Aquatic vascular and algae Lowest EC50 for vascular plants Lowest EC50 for algae Terrestrial Endangered Lowest EC5 or NOEC for seedling emergence for monocots Lowest EC5 or NOEC for vegetative vigor for monocots Lowest EC5 or NOEC for seedling emergence for dicots Lowest EC5 or NOEC for vegetative vigor for dicots34



## Ap pendix VI. EISB Summ ary Table QA /QC Procedures Ap plication Name Version Number

QA Summary Summary Developer: Person or Team Name and Team Member Names Date: dd/mmm/yyyy Application Developer: Person or Team Name and Member Names Phone, Address (e
-mail and postal) Application Contact: Primary Contact Name and Alternate Contact Name Phone, Address (e
-mail and postal) Application Ty

pe: Empirical, deterministic,stochastic(?), Aquatic, Ecological, Health Effects, etc. Historical Insight: Method of Version Control and Notification: process and names involved in version control Li st of All Prior Application Versions and Dates of their QA Summaries: Version Number, QA Summary Date: dd/mmm/yyyy Version Number, QA Summary Date: dd/mmm/yyyy Application Calibration Reports: Include a reference to where this information is available. Application Evaluation Reports: Include a reference to where this information is available. Summerize Application evaluation. Refer to: Guidance for QA Project Plans for Applicationing (QA/G- 5M) EPA/600/R- 02/007 http://www.epa.gov/quality /qa_docs.html Peer Review Report Summary - Dated: dd/mmm/yyyy Include a reference to where this information is ava ilable. Application Access Mechanism or Application Site: Web Site or Access Location User Manual Access Mechanism or Site: Web Site or Access Location Intended Use of Output (risk characterization, support decision, answer question, etc.): Intended Use of Output justification goes here.35

Regulatory

Context, if any : Regulatory Context justifying existence of the Application goes here. Process for archiving and documentation:36 Ap plication Name Version Number

QA Summary Summary

Developer: Person or Tea m Name and Team Member Names

Date: dd/mmm/yyyy Application Output Description: Include output description(s) including the format(s). And/Or include a reference to where this information is available. Examples of How Application Output is Used: Include examples of Application output. And/Or include a reference to where this information is available. An appendix in the User’ s Manual might be appropriate. Limitations on Application Use: Describe any limitations on using the Application. And/Or include a r

eference to where this information is available. An appendix in the User’ s Manual might be appropriate. Description of Application Input including Input Sources: List the Application Input and Input Sources. Describe details as necessary and/or include a reference to where this information is available. This should refer to a particular portion of the User’ s Manual.37 

## Appendix VII. Rationale for Ordering Papers
Code Rationale R1 Paper reports a more sensitive endpoint than registrant data R2 Paper supports the line of evidence approach R3 Paper identifies a species of concern (e.g., endangered species or sensitive species group) that are not ty

pically addressed in registrant data R4 Paper reports ecologically relevant endpoints not ty pically addressed by

registrant submitted data R5 Although these papers are rejected by ECOTOX as a toxicity

study, they

may provide the risk assessor with useful nontoxicity information. These studies would include incident, fate, in vitro, mixture, review, and modeling studies. R6 Paper not ordered - deemed not pertinent38



## Appendix VIII. Rationale for the Use of Open Literature in

Risk Assessment

Code Rationale QUANT The study has been evaluated by

a DER, determined to be supplemental, and deemed appropriate for use in risk quotient calculations QUAL The paper is not appropriate for quantitative use but is of good quality, addresses issues of concern to the risk assessment and is used in the risk characterization discussion INV The study has been deemed inappropriate for use in risk quotient calculations (quantitatively ) and in the risk characterization (qualitatively ) because it does not address endpoints of concern to the risk assessment and/or is of poor quality



## Appendix IX. Qualitative Study Summary39

Chemical Name: ECOTOX Record Number and Citation:
DP Barcode: Date of Assessment: Brief Summary

of Study Findings:

Description of Use in Document:
Rationale for Use:
Limitations of Study :

Reviewer: