Science Policy Council
 
EPA 100/B-03/001 

June 2003 

U.S. Environmental Protection Agency
 

A Summary of General Assessment Factors for
 

Evaluating the Quality of Scientific and Technical
 

Information
 

Prepared for the U.S. Environmental Protection Agency
 

by members of the Assessment Factors Workgroup, a group of the
 

EPA’s Science Policy Council
 

Science Policy Council
 

U.S. Environmental Protection Agency
 

Washington, DC 20460
 
Page ii Assessment Factors 

DISCLAIMER 

This document has been reviewed in accordance with United States Environmental 

Protection Agency policy and approved for publication and distribution. Mention of trade names 

or commercial products does not constitute endorsement or recommendation for use. 
Assessment Factors	 Page iii
 

TABLE OF CONTENTS
 

FOREWORD ............................................................ Page iv
 

ACKNOWLEDGMENTS .................................................. Page vi
 

A Summary of General Assessment Factors for Evaluating the Quality of Scientific
 

and Technical Information ............................................ Page 1
 

1.	 Introduction ................................................. Page 1
 

1.1	 Overview ............................................. Page 1
 

1.2	 Purpose ............................................... Page 1
 

1.3	 Background ........................................... Page 2
 

2.	 Assessment Factors ........................................... Page 4
 

2.1	 General Assessment Factors .............................. Page 4
 

2.2	 Examples of Questions Raised by Consideration of the
 

Assessment Factors ..................................... Page 5
 

2.3	 Relationship Between the General Assessment Factors and the
 

Elements of Quality in EPA’s Information Quality Guidelines .... Page 8
 

3.	 Summary .................................................. Page 10
 

References ............................................................. Page 11
 
Page iv Assessment Factors 

FOREWORD 

This document was prepared under the auspices of the Science Policy Council (SPC) to 

describe the assessment factors and considerations generally used by the Agency to evaluate the 

quality and relevance of scientific and technical information. These general assessment factors 

are founded in the Agency guidelines, practices and procedures that make up the EPA 

information and quality systems, including existing program-specific quality assurance policies. 

As such, the general assessment factors do not constitute new quality-related considerations, nor 

does this document describe a new process for evaluating information. This document is 

intended to raise the awareness of the information-generating public about EPA’s ongoing 

interest in ensuring and enhancing the quality of information available for Agency use. Further, 

it complements the Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and 

Integrity of Information Disseminated by the Environmental Protection Agency (EPA 

Information Quality Guidelines). This summary of Agency practice is also an additional resource 

for Agency staff as they evaluate the quality and relevance of information, regardless of source. 

Consistent with the Agency’s approach to the development of the EPA Information 

Quality Guidelines, this document is the product of an open, collaborative process between EPA 

and the public. During the development of this document, EPA obtained public comments on a 

draft version of the document released in September 2002 and commissioned the National 

Academy of Sciences to host a workshop in January 2003 to discuss key aspects of this document 

from a scientific and technical perspective. 

We want to acknowledge and thank the Assessment Factors workgroup for its steady and 

insightful work in assembling this document under stringent time constraints and scrutiny. We 

particularly appreciate the efforts of the co-chairs, Halûk Özkaynak (ORD) and Greg Schweer 

(OPPTS), who successfully led and shepherded the workgroup. 

It is with great pleasure that we present the Summary of General Assessment Factors for 

Evaluating the Quality of Scientific and Technical Information. 

Paul Gilman, Ph.D. Elaine Stanley 

Science Advisor to the Agency Director, Office of Information Analysis 

Chair, Science Policy Council and Access 

Office of Environmental Information 
Assessment Factors Page v 

Science Policy Council 

Paul Gilman, Science Advisor to 

the Agency, Chair 

Devereaux Barnes, OSWER 

Robert Brenner, OAR 

Jerry Clifford, OIA 

William Farland, ORD 

Geoffrey Grubbs, OW 

Diana Love, OECA 

Bharat Mathur, Region 5 

Albert McGartland, OPEI 

Stan Meiburg, Region 4 

Joseph Merenda, OPPTS 

William Muszynski, Region 2 

Joanne Rodman, OCHP 

Michael Ryan, OCFO 

Michael Shapiro, OW 

Elaine Stanley, OEI 

Ramona Trovato, OEI 

Vanessa Vu, SAB 

Anna Wolgast, OGC 

Science Policy Council Steering Committee 

Randolph Perfetti, Chair 

Thomas Baugh, Region 4 

Michael Brody, OCFO 

Reginald Cheatham, OEI 

Patricia Cirone, Region 10 

John Diamante, OIA 

Michael Firestone, OCHP 

A. Robert Flaak, SAB 

Jerri-Anne Garl, Region 5 

Roland Hemmett, Region 2 

Lee Hofmann, OSWER 

Daniel Malloy, OCFO 

Kate Mahaffey, OPPTS 

Carl Mazza, OAR 

James Nelson, OGC 

Jennifer Orme-Zavaleta, ORD 

Rosemarie Russo, ORD 

Rita Schoeny, OW 

Margaret Stasikowski, OPPTS 

Kevin Teichman, ORD 

Mary Ellen Weber, OPPTS 

William Wood, ORD 

Tracey Woodruff, OPEI 

Science Policy Council Staff 

Edward Bender Kerry Dearfield Kathryn Gallagher 
Page vi Assessment Factors 

ACKNOWLEDGMENTS 

Many people worked on and contributed to the assessment factors effort that ultimately 

resulted in this document. Many EPA employees from the Agency’s Offices and Regions 

provided input and we would like to specifically acknowledge the efforts and contributions made 

by the following individuals: 

Halûk Özkaynak, ORD, Co-Chair Greg Schweer, OPPTS, Co-Chair 

Mary Belefski, OPPTS Kevin Kubik, Region 2 

Katherine Biggs, OECA Karen Martin, OAR 

Connie Bosma, ORD Amy Mills, ORD 

Ming Chang, OEI Barbara Pace, OGC 

Weihsueh Chiu, ORD Nicole Paquette, OEI 

Evangeline Cummings, OEI Devon Payne-Sturges, OPEI 

Kerry Dearfield, SPC Staff Ward Penberthy, OPPTS 

Thomas Eagles, OAR James Quackenboss, ORD 

Kathryn Gallagher, SPC Staff Mary Reiley, OW 

Staci Gatica-Hebert, ORD Bruce Rodan, ORD 

Roland Hemmett, Region 2 Terry Simpson, ORD 

Annie Jarabek, ORD Elaine Stanley, OEI 

Stephen Kroner, OSWER Lanelle Wiggins, OPEI 
Assessment Factors Page 1 

U.S. Environmental Protection Agency 

A Summary of General Assessment Factors for Evaluating the
 

Quality of Scientific and Technical Information
 

1. Introduction 

1.1 Overview 

As part of the ongoing commitment of the United States Environmental Protection 

Agency (USEPA) to ensure the quality of the information it uses, the Agency is publishing this 

summary of general assessment factors in an effort to enhance the transparency about EPA’s 

quality expectations for information that is voluntarily submitted to or gathered or generated by 

the Agency for various purposes. This Assessment Factors document is intended to inform 

information-generating scientists about quality issues that should appropriately be taken into 

consideration at the time information is generated. It is also an additional resource for Agency 

staff as they evaluate the quality and relevance of information, regardless of source. The general 

assessment factors are drawn from the Agency’s existing information quality systems, practices 

and guidelines that describe the types of considerations EPA takes into account when evaluating 

the quality and relevance of scientific and technical information used in support of Agency 

actions. As such, the general assessment factors do not constitute new quality-related 

considerations, nor does this document describe a new process for evaluating information. This 

document is intended to raise the awareness of the information-generating public about EPA’s 

ongoing interest in ensuring and enhancing the quality of information available for Agency use. 

1.2 Purpose 

The Agency believes that the summary of general assessment factors provided in this 

document will serve to increase the extent to which the information-generating public builds 

quality considerations into the generation and documentation of their information products. The 

Agency expects that the resulting improvements in the quality of such information will enable the 

Agency to more fully utilize and disseminate such information. Thus, this document is intended 

to complement the Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and 

Integrity of Information Disseminated by the Environmental Protection Agency (EPA 

Information Quality Guidelines) (EPA, 2002) and other Agency efforts to ensure and enhance 

information quality, as discussed below in Section 1.3. This document is not a regulation and is 

not intended to create any legal rights or impose legally binding requirements or obligations on 

EPA or the information-generating public. 
Page 2 Assessment Factors 

Although the assessment factors as presented are intended to most generally apply to 

individual pieces of information, they can also be used as part of a broader evaluation of a body 

of evidence that is collectively evaluated through a process typically referred to as a “weight-ofevidence” approach. The weight-of-evidence approach considers all relevant information in an 

integrative assessment that takes into account the kinds of evidence available, the quality and 

quantity of the evidence, the strengths and limitations associated with each type of evidence and 

explains how the various types of evidence fit together. Details as to the Agency’s approach to 

integrating a body of evidence depend on the type of decision or action being undertaken, and are 

not the subject of this document. For instance, the Guidelines for Carcinogen Risk Assessment, 

Review Draft (EPA, 1999) provides guidance on characterizing the weight-of-evidence for 

carcinogenicity. Similarly, the Guidelines for Ecological Risk Assessment (EPA, 1998) describes 

the development of “lines of evidence” to reach a conclusion regarding an ecological risk 

estimate. 

The general assessment factors are presented and discussed more fully in Section 2.1. 

Section 2.2 presents illustrative examples of the types of questions that consideration of these 

factors raise in the process of evaluating the quality and relevance of different types of 

information for different uses. The relationship between these general assessment factors and the 

elements of quality contained in the EPA Information Quality Guidelines is discussed in Section 

2.3. 

1.3 Background 

In October 2002, EPA made available the EPA Information Quality Guidelines. The 

EPA Information Quality Guidelines were developed in response to guidelines issued by the 

Office of Management and Budget (OMB, 2002) under Section 515 of the Treasury and General 

Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554; H.R. 5658). The 

EPA Information Quality Guidelines set forth the Agency’s policy and procedural guidance for 

ensuring and maximizing the quality of information disseminated by EPA, regardless of the 

source of the information, and articulate the Agency’s ongoing commitment to ensuring and 

maximizing information quality through existing policies, systems and programs. Thus, the EPA 

Information Quality Guidelines build upon the Agency’s numerous existing systems, practices 

and guidelines that address information quality, and provide new policies and administrative 

mechanisms that respond to OMB’s guidelines. 

The EPA Information Quality Guidelines also recognize that, as part of its efforts to 

ensure information quality, the Agency does not wait until the point at which information is 

disseminated to consider important quality principles. Rather, the Agency recognizes that it is 
Assessment Factors Page 3 

important to assure the quality of information through processes that incorporate quality 

principles starting at the point at which information is generated. 

The Agency uses and disseminates information that is generated by a variety of sources, 

including EPA itself as well as other parties that produce information through EPA contracts, 

grants and cooperative and interagency agreements or in response to a requirement under a 

statute, regulation, permit, order or other mandate. EPA generally has considerable control or 

influence over the quality of this information at the time the information is generated. Existing 

quality controls that EPA applies to the generation of information from these sources are based 

on EPA’s Quality System (EPA, 2000a; EPA, 2000b), Peer Review Policy (EPA, 1994), Risk 

Characterization Policy (EPA, 1995) and other agency-wide and program-specific policies, as 

well as specific provisions in contracts, grants, agreements, regulations and statutes. A few 

additional useful web sites for obtaining further information on EPA’s Quality System and 

various regulatory policies and decisions are provided under the References section at the end of 

this document. 

The Agency also receives information that is voluntarily submitted by or collected from 

external sources, the generation of which does not come under the direct control of the Agency’s 

internal information quality systems. This information may include scientific studies published 

in journal articles, testing or survey data, such as environmental monitoring or laboratory test 

results, and analytic studies, such as those that model environmental conditions or that assess 

risks to public health. Since EPA has placed great emphasis on the management of 

environmental issues on a cooperative basis with its many stakeholders, the amount of 

information submitted to EPA by external sources is increasing. Such sources include other 

federal, state, tribal, local and international agencies; national laboratories; academic and 

research institutions; business and industry; and public interest organizations. Although EPA’s 

existing quality systems are not applied at the time this information is generated, EPA does apply 

appropriate quality controls when evaluating this information for use in Agency actions and for 

its dissemination consistent with the EPA Information Quality Guidelines. The Agency hopes 

this document will inform the public of EPA’s objectives and enlist them in its effort to 

disseminate quality information and make quality decisions. 

During the development of this document, EPA requested public input in a variety of 

ways. EPA distributed a draft document for public comment in September 2002 and hosted a 

public meeting in Washington, DC. In January 2003, EPA commissioned the National Academy 

of Sciences to host a workshop to discuss key aspects of this document from a scientific and 

technical perspective. EPA revised this document based on the input received through these 

public outreach opportunities. 
Page 4	 Assessment Factors 

2.	 Assessment Factors 

2.1	 General Assessment Factors 

When evaluating the quality and relevance of scientific and technical information, the 

considerations that the Agency typically takes into account can be characterized by five general 

assessment factors: 

!	 Soundness - The extent to which the scientific and technical procedures, 

measures, methods or models employed to generate the information are 

reasonable for, and consistent with, the intended application. 

!	 Applicability and Utility - The extent to which the information is relevant for the 

Agency’s intended use. 

!	 Clarity and Completeness - The degree of clarity and completeness with which 

the data, assumptions, methods, quality assurance, sponsoring organizations and 

analyses employed to generate the information are documented. 

!	 Uncertainty and Variability - The extent to which the variability and uncertainty 

(quantitative and qualitative) in the information or in the procedures, measures, 

methods or models are evaluated and characterized. 

!	 Evaluation and Review - The extent of independent verification, validation and 

peer review of the information or of the procedures, measures, methods or 

models. 

These assessment factors reflect the most salient features of EPA’s existing information 

quality policies and guidelines. Whether the information consists of scientific theories, computer 

codes for modeling environmental systems, environmental monitoring data, economic analyses, 

social survey or demographic data, chemical toxicity testing, environmental fate and transport 

predictions or a human health risk assessment, EPA generally evaluates information by weighing 

considerations that fit within these five assessment factors. Thus, these factors encompass 

considerations that are weighed in the process of evaluating the quality and relevance of 

information. The appropriate level of quality for any particular information product is 

necessarily related to how and in what context the information is to be used. If EPA chooses to 

later “disseminate” the information, that dissemination would be covered by the Information 

Quality Guidelines which describe EPA policy and procedures for reviewing and substantiating 

the quality of information before EPA disseminates it. 
Assessment Factors	 Page 5 

When EPA considers using information for a particular purpose, careful judgment is 

applied to evaluate the information for quality and relevance in the context of the specific 

Agency action being developed. For instance, in the context of a given action, EPA may need to 

weigh the appropriateness of using information with significant, but known uncertainties to fill 

“data gaps,” relative to using default assumptions or committing additional resources to generate 

new information. 

2.2	 Examples of Questions Raised by Consideration of the Assessment Factors 

Example questions that could be raised by the consideration of each of the assessment 

factors for various types of information are provided below. Given the very general nature of 

these assessment factors, the Agency felt that a compilation of such illustrative questions would 

most clearly convey the intended nature and breadth of the assessment factors, and how they 

would be reflected in an evaluation of various types of information. However, the applicability 

of these factors depends on the individual situation, and EPA retains discretion to consider and 

use factors and approaches on a case-by-case basis that may differ from the illustrative 

considerations presented below. 

2.2.1	 Soundness 

The extent to which the scientific and technical procedures, measures, methods or 

models employed to generate the information are reasonable for, and consistent 

with, the intended application. 

a)	 Is the purpose of the study reasonable and consistent with its design? 

b)	 To what extent are the procedures, measures, methods, or models 

employed to develop the information reasonable and consistent with sound 

scientific theory or accepted approaches? 

c)	 How do the study’s design and results compare with existing scientific or 

economic theory and practice? Are the assumptions, governing equations 

and mathematical descriptions employed scientifically and technically 

justified? Is the study based on sound scientific or econometric 

principles? 

d)	 In the case of a survey, have the questionnaires and other survey 

instruments been validated (e.g., compared with direct measurement data)? 

Were checks for potential errors made during the interview process? 
Page 6	 Assessment Factors 

e)	 How internally consistent are the study’s conclusions with the data and 

results presented? 

2.2.2	 Applicability and Utility 

The extent to which the information is relevant for the Agency’s intended use. 

a)	 How useful or applicable is the scientific or economic theory applied in 

the study to the Agency’s intended use of the analysis? 

b)	 How relevant are the study’s purpose, design, outcome measures and 

results to the Agency’s intended use of the analysis (e.g., for a chemical 

hazard characterization)? 

c)	 Are the domains (e.g., duration, species, exposure) where the model or 

results are valid useful to the Agency’s application? 

d)	 How relevant is the study to current conditions of interest? For example, 

in the case of a survey, are conditions likely to have changed since the 

survey was completed (i.e., is the information still relevant)? Is the 

sampled population relevant to the Agency’s current application? How 

well does the sample take into account sensitive subpopulations? 

2.2.3	 Clarity and Completeness 

The degree of clarity and completeness with which the data, assumptions, 

methods, quality assurance, sponsoring organizations and analyses employed to 

generate the information are documented. 

a)	 To what extent does the documentation clearly and completely describe 

the underlying scientific or economic theory and the analytic methods 

used? 

b)	 To what extent have key assumptions, parameter values, measures, 

domains and limitations been described and characterized? 

c)	 To what extent are the results clearly and completely documented as a 

basis for comparing them to results from other similar tests? 
Assessment Factors	 Page 7 

d)	 If novel or alternative theories or approaches are used, how clearly are they 

explained and the differences with accepted theories or approaches 

highlighted? 

e)	 Is the complete data set accessible, including metadata, data-dictionaries 

and embedded definitions (e.g., codes for missing values, data quality 

flags and questionnaire responses)? Are there confidentiality issues that 

may limit accessibility to the complete data set? 

f)	 In the case of a modeling exercise, have the definitions and units of model 

parameters been provided? To what extent have the procedures for 

applying the model been clearly and completely documented? How 

available and adequate is the information necessary to run the model 

computer code? 

g)	 To what extent are the descriptions of the study or survey design clear, 

complete and sufficient to enable the study or survey to be reproduced? 

h)	 Have the sponsoring organization(s) for the study/information product and 

the author(s) affiliation(s) been documented? 

i)	 To what extent are the procedures for quality assurance and quality control 

of the data documented and accessible? 

2.2.4	 Uncertainty and Variability 

The extent to which the variability and uncertainty (quantitative and qualitative) 

in the information or in the procedures, measures, methods or models are 

evaluated and characterized. 

a)	 To what extent have appropriate statistical techniques been employed to 

evaluate variability and uncertainty? To what extent have the sensitive 

parameters of models been identified and characterized? 

b)	 To what extent do the uncertainty and variability impact the conclusions 

that can be inferred from the data and the utility of the study? What are 

the potential sources and effects of error and bias in the study design? 
Page 8	 Assessment Factors 

c)	 Did the study identify potential uncertainties such as those due to inherent 

variability in environmental and exposure-related parameters or possible 

measurement errors? 

2.2.5	 Evaluation and Review 

The extent of independent verification, validation and peer review of the 

information or of the procedures, measures, methods or models. 

a)	 To what extent has there been independent verification or validation of the 

study method and results? What were the conclusions of these 

independent efforts, and are they consistent? 

b)	 To what extent has independent peer review been conducted of the study 

method and results, and how were the conclusions of this review taken 

into account? 

c)	 Has the procedure, method or model been used in similar, peer reviewed 

studies? Are the results consistent with other relevant studies? 

d)	 In the case of model-based information, to what extent has independent 

evaluation and testing of the model code been performed and documented? 

2.3	 Relationship Between the General Assessment Factors and the Elements of 

Quality in EPA’s Information Quality Guidelines 

The definition of quality in the EPA Information Quality Guidelines consists of three 

components, consistent with the definition of quality in OMB’s Guidelines: objectivity, utility 

and integrity of disseminated information. “Objectivity” focuses on the extent to which 

information is presented in an accurate, clear, complete and unbiased manner; and, as a matter of 

substance, the extent to which the information is accurate, reliable and unbiased. “Utility” refers 

to the usefulness of the information to the intended users. “Integrity” refers to security, such as 

the protection of information from unauthorized access or revision, to ensure the information is 

not compromised through corruption or falsification. 

The five general assessment factors presented in this document are consistent with the 

quality elements of objectivity and utility, but do not extend to the distinct element of integrity 

(which refers to the separate matter of security issues). The assessment factor applicability and 

utility is most directly related to the element of utility in the OMB and EPA Information Quality 
Assessment Factors Page 9 

Guidelines. The other four assessment factors relate to the element of objectivity, which itself 

encompasses a number of issues related to both presentation and substance. In particular, the 

factor clarity and completeness is most directly related to some aspects of the presentation of 

information (including whether the information is “presented in an accurate, clear, complete and 

unbiased manner”). The factors soundness, uncertainty and variability and evaluation and 

review most directly relate to the substantive aspects of the element of objectivity (related to 

whether the information itself is “accurate, reliable and unbiased”), although they also play a role 

in enhancing aspects of the presentation of the information. Thus, the general assessment factors 

are fully consistent with the related information quality elements described in the OMB and EPA 

Information Quality Guidelines, and do not constitute a conceptually different or unrelated basis 

for evaluating information quality. 

It is important to note that the EPA Information Quality Guidelines apply to 

“information” that EPA disseminates to the public. The EPA Information Quality Guidelines 

apply to information generated by third parties if EPA distributes information prepared or 

submitted by an outside party in a manner that reasonably suggests that EPA endorses or agrees 

with it; if EPA indicates in its distribution that the information supports or represents EPA's 

viewpoint; or if EPA in its distribution proposes to use or uses the information to formulate or 

support a regulation, guidance, policy or other Agency decision or position (EPA 2002). Please 

refer to the EPA Information Quality Guidelines for additional information regarding their 

applicability to information EPA disseminates. 
Page 10 Assessment Factors 

3. Summary 

This document describes the assessment factors and considerations generally used by the 

Agency to evaluate the quality and relevance of the broad range of scientific and technical 

information used by the EPA. These factors are founded in the Agency guidelines, practices and 

procedures that make up the EPA information and quality systems including existing 

program-specific quality assurance policies. Consistent with the Agency’s approach to the 

development of the EPA Information Quality Guidelines, this document is the product of an 

open, collaborative process between EPA and the public. 

The Agency believes that the summary of general assessment factors provided in this 

document will serve to increase the extent to which the information-generating public builds 

quality considerations into the generation and documentation of their information products. The 

Agency expects that the resulting improvements in the quality of such information will enable the 

Agency to more fully utilize and disseminate such information. Thus, this document is intended 

to complement the EPA Information Quality Guidelines and other Agency efforts to ensure and 

enhance information quality. 
Assessment Factors Page 11 

References 

US Environmental Protection Agency, 1994, Peer Review and Peer Involvement at the U.S. 

Environmental Protection Agency, Science Policy Council, 

http://www.epa.gov/osp/spc/memo0607.htm 

US Environmental Protection Agency, 1995, Policy for Risk Characterization, Science Policy 

Council, http://www.epa.gov/osp/spc/rcpolicy.htm 

US Environmental Protection Agency, 1998, Guidelines for Ecological Risk Assessment, 

EPA/630/R095/002F, Risk Assessment Forum, 

http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12460 

US Environmental Protection Agency, 1999, Guidelines for Carcinogen Risk Assessment, 

Review Draft, NCEA-F-0644, Office of Research and Development, 

http://cfpub.epa.gov/ncea/raf/cancer.cfm 

US Environmental Protection Agency, 2000a, “Policy and Program Requirements for the 

Mandatory Agency-wide Quality System,” (EPA Order 5360.1 A2), 

http://www.epa.gov/quality/qs-docs/5360-1.pdf 

US Environmental Protection Agency, 2000b, “EPA Quality Manual for Environmental 

Programs,” (EPA Order 5360 A1), http://www.epa.gov/quality/qs-docs/5360.pdf 

US Environmental Protection Agency, 2002, Guidelines for Ensuring and Maximizing the 

Quality, Objectivity, Utility, and Integrity of Information Disseminated by the 

Environmental Protection Agency, EPA 260R-02-008, Office of Environmental 

Information, http://www.epa.gov/oei/qualityguidelines 

US Office of Management and Budget, 2002, Guidelines for Ensuring and Maximizing the 

Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal 

Agencies, (67 FR 8452), http://www.whitehouse.gov/omb/fedreg/reproducible2.pdf 

Additional Useful Web Sites 

EPA Quality System web site: http://www.epa.gov/quality 

EPA Science Policy Council web site: http://www.epa.gov/osp/spc 

EPA Information Quality Guidelines web site: http://www.epa.gov/oei/qualityguidelines