# Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency

10101010010101 1010101011101101010111010 01100010101010 0110100001010010101110011

information quality guidelinesEPA/260R-02-008 October 2002



# Guideli nes for Ensuring an d Maximizin g the Quali ty, Obje ctivi ty, Ut ilit y, and Integrity,of Info rmation Di sseminat ed by the Environmental Protection Ag ency

Prepared by: U.S. Environmental Protection Agency Office of Environmental Information (2810) 1200 Pennsylvania Avenue, NW Washington, DC 20460Addendum 06/24/2004 This addendum updates the contact information for submittal of Requests for Correction under the Information Quality Gu idelines (Section 8.2 of the Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by EPA , October, 2002) An affected person may submit an RFC vi a any one of the methods listed here:
• E-mail at quality@epa.gov
• Fax at (202) 565-2441
• Mail to Information Quality Guidelines Staff, Mail Code 2811R, U.S. EPA, 1200 Pennsylvania Ave., N.W ., Washington, DC, 20460
• By courier or in person to Information Quality Guide lines Staff, Ronald Reagan Building, Room M1200, 1300 Pennsylvania Ave., N.W., Washington, DC

Addendum 05/13/2005 This addendum updates the link for the EPA In tegrated Error Correction Process found in Section 4.4, footnote 8, page 12 of the Guidelines for Ensuring and Maximizing the Quality, Objectivity, Util ity, and Integrity of Information Disseminated by EPA , October, 2002.

_____________________________ 8 Integrated Error Correction Process for Environmental Data. http://oaspub.epa.gov/enviro/ets_gra b_error.smart_form**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  G u ide line s forEnsur i ng and M axim i z i ng the Quality, Obje c t ivity, Utility, and Inte gr ity of In f o rm ati on Di ssem i n a ted b y th e E n vi ron m en tal P r otecti on Agen cy Table of C o nt e n t s 1 Introduction

............................................................ 3 2 EPA Mission a nd Commitme nt to Qua lity

.................................... 5 2.1 EPA’s Mission and Commitment to Public Access

........................ 5 2.2 Inf orm at i on M a na g e m e nt i n E P A

..................................... 5 2.3 E P A ' s R e l at i ons hi p w i th S t at e , T ri ba l , a nd L oc a l G ove rnm e nt s .............. 8 3 O M B G ui de l i ne s ....................................................... . 9 4 E x i sting Polic ie s a nd Proc e dure s thatEnsure a nd Ma x imize In formation Qua lity

..... 10 4.1 Qua lity Sy ste m ................................................... 10 4.2 P e e r R e vi e w P ol i c y ............................................... 11 4.3 A c t i on D e ve l opm e nt P r oc e s s ........................................ 12 4.4 Int e g r at e d E r r or C orr e c t i on P r oc e s s ................................... 12 4.5 Inf orm at i on R e s our c e s M a na g e m e nt M a nua l

........................... 13 4.6 Risk Characteriz ation Policy and Handbook

............................ 13 4.7 P r og r a m - S pe c i f i c P ol i c i e s .......................................... 13 4.8 EPA Commitment to Continuous I mprovement
......................... 14 4.9 Summary of Ne w Ac tivitie s a nd In itiative s ............................. 14 5 G uide line s Sc ope a nd Applic ab ility

........................................ . 15 5.1 Whatis “ Q ua lity ” Ac c o rding to the Guide line s ?

........................ 15 5.2 W hat i s the P ur pos e of the s e G ui de l i ne s ?

.............................. 15 5.3 W he n D o the s e G ui de l i ne s A ppl y ?

................................... 15 5.4 W hat i s N ot C ove r e d by the s e G ui de l i ne s ?

............................ 16 5.5 WhatHappe ns if In formation is In itia lly Not Cove re d by the s e Guide line s , but E P A S ubs e que nt l y D i s s e m i nat e s i t t o the P ubl i c ?

....................... . 18 5.6 How doe s EPA Ensure the Obje c tivity , Utility , a nd In te g rity of information thatis not c ove r e d by the s e G ui de l i ne s ?

..................................... 18 6 G uide line s for Ensuring a nd Ma x imizing In formation Qua lity

.................... 19 6.1 How doe s EPA Ensure a nd Ma x imize the Qua lity of Disse minat e d In formation?

.............................................................. . 19 6.2 H ow D oe s E P A D e f i ne Inf l ue nt i a l Inf orm at i on f or the s e G ui de l i ne s ?

....... . 19 6.3 How Doe s EPA Ensure a nd Ma x imize the Qua lity of “ Influe n tia l” In formation?

.............................................................. . 20 6.4 How Doe s EPA Ensure a nd Ma x imize the Qua lity of “ Influe n tia l” Sc ie ntific Risk A s s e s s m e nt Inf orm at i on?

........................................... 21 6.5 Doe s EPA Ensure a nd Ma x imize the Qua lity of In formation from Ex te rna l S our c e s ?

........................................................ 28

T able of Contents 1**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  7 A dm i ni s tr at i ve M e c ha ni s m f or P r e - di s s e m i nat i on R e vi e w ....................... 29 7.1 W hat ar e the A dm i ni s tr at i ve M e c ha ni s m s f or P r e - di s s e m i nat i on R e vi e w s ?

.... 29 8 A dm i ni s tr at i ve M e c ha ni s m s f or C orr e c t i on of Inf orm at i on ..................... . 30 8.1 W h atare EP A' s Adm i ni s tr at i v e Mechani s m s for Affect ed P e rsons t o S eek and O bt a i n C orr e c t i on of Inf orm at i on?

................................... 30 8.2 W h at Should be In cluded in a Request for Correction of I n formation?

....... 30 8.3 W h en Does EPA In tend to Consider a Request for Correction of In formation?

.............................................................. . 31 8.4 How Does EPA In tend to Respond to a Request for Correction of In formation?

.............................................................. . 31 8.5 How Does EPA Ex pect to Process Requests for Correction of I n formation on W hi c h E P A ha s S oug ht P ubl i c C om m e nt ?

............................ . 32 8.6 W h at Should be In cluded in a Request Asking EPA to Reconsider its Decision on a R e que s t f or the C or r e c t i on of Inf orm at i on?

.......................... . 34 8.7 How Does EPA In tend to Process Requests for Reconsideration of EPA D e c i s i ons ?

..................................................... . 34 A p p e n d i x A : I Q G D e ve l op m e n t P r oc e s s an d D i s c u s s i on of P u b l i c C om m e n t s ......... . 36 A.1 Introduction

.......................................................... . 36 A .2 G e ne r a l S um m ary of C om m e nt s .......................................... . 37 A.3 R esponse to Comments by Guidelines Topic Area

............................ . 38 A.3.1 Ex isting Policy

.................................................. . 38 A.3.2 Scope and Applicability

........................................... . 39 A.3.3 Sources of In formation

............................................ . 42 A.3.4 In fluential In formation

............................................ . 43 A.3.5 Reproducibility

................................................... 47 A.3.6 Influential Risk Assessment

......................................... 48 A.3.7 Complaint Resolution

............................................ . 53 A .4 N e x t S t e ps ........................................................... . 56

T able of Contents 2**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  1 I nt r o duc t i on The Environmental Protection Ag ency (EPA) is committed to providing public access to environmental information. This commitment is integ r al to our mission to protect human health a nd the e nvironme n t. One of our g oa l s is tha ta ll parts of soc i e t y - inc l uding c o mmunitie s, i ndi vi dual s , busi n esses, S t at e and l o cal g overnm e nt s, Tri b al g overnm e nt s - have access t o accurat e i n form at i on suffi ci ent t o effect i v el y part i c i p at e i n m a nag i ng hum an heal th and environmental risks. To fulfill this and other important g o als, EPA must rely upon information of appropriate quality for each decision we make. Developed in response to g u idelines issued by the Office of Manag e ment and B udg et (OMB ) 1 under Section 515(a) of the Treasury and General Government Appropriations Act for F i scal Year 2001 (Public L a w 106-554; H.R. 5658), the Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Prot ect i on Agency (the Guidelines) contain EPA’s policy and procedural g u idance for ensuring a nd ma x imizing the qua lity of information we disse minat e . The Guide line s a l so outline administrative mechanisms for EPA pre-dissemination review of information products and describe some new mechanisms to enable affected persons to seek and obtain corrections from EPA reg arding disseminated information that they believe does not comply with EPA or OMB g u idelines. B e y ond policies and procedures these Guidelines also incorporate the following perform ance g o al s:

• D isseminated information should adhere to a basic standard of quality , including obje c tivity , utility , a nd inte g rity

.
• T he principles of information quality should be integ r ated into each step of EPA’s development of information, including creation, collection, maintenance, and disse mination.

• A dministrative mechanisms for correction should be flex ible, appropriate to the nature and timeliness of the disseminated information, and incorporated into EPA’s information re sourc e s ma na g e me nt a nd a d ministrative pra c tic e s . OMB encourag es ag enci es t o i n corporat e st andards and procedures i nt o ex i s t i ng i n form at i on resources manag e ment practices rather than create new, potentially duplicative processes. EPA ha s take n this a dvic e a nd re lie s on nume r ous e x isting qua lity -re late d polic ie s in the s e Guide line s . EPA will work to e n sure se a mle ss imple me ntation into e x isting pra c tic e s . Itis e x pe c t e dtha t EPA ma na g e rs a nd sta ff will fa miliarize the m se lve s with the s e Guide line s , a nd will c ar e f ully review ex isting prog ram policies and procedures in order to accommodate the principles outlined in this doc ume n t.

1 Gu id elin es f o rEn su r i n g an d Max i mi zin g th e Qu ality , Ob j ectiv ity , Utility , an d I n teg r ityo f I n f o r m atio n D i s s e mi n a t e d byF e deral A g en ci es , O M B, 2002. (67 F R8452) H e rei naf t e r “ O MB g u i del i n es ”. h ttp : / /www. wh i t e h o u s e . g o v/ o m b /f e d r e g / r e p r o d u c i b l e 2

. p d f Intr oduc tion 3**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  EPA' s Guide line s ar e inte nde d to c arry out OMB ' s g ove rnme nt-wide polic y re g arding information we disseminate to the public. Our Guidelines reflect EPA' s best effort to present our g oa l s a nd c o mmitme nts for e n suring a nd ma x imizing the qua lity of information we disse minat e . As suc h, the y ar e not a re g u lation a nd do not c ha ng e or substitute for a ny le g a l re quire me nts. They provide non-binding policy and procedural g u idance, and are therefore not intended to create leg a l rig h ts, impose leg a lly binding requirements or oblig ations on EPA or the public when applied in particular situations, or chang e or impact the status of information we disseminate, nor to contravene any other leg a l requirements that may apply to particular ag ency de te rminations or othe r a c tions. EPA' s inte ntion is to fully imple me nt the s e Guide line s in orde r to achieve the purposes of Section 515. These Guidelines are the product of an open, collaborative process between EPA and numerous EPA stakeholders. The Guidelines development process is described in the Appendix to this document. EP A received many public comments and has addressed most comments in these Guidelines. A discussion of public comments is also provided in the Appendix and is g r ouped by overarchi ng th em es and com m e nt s by Gui d el i n es t opi c areas. EP A vi ews th ese Gui d el i n es as a living doc ume n t, a nd a n tic ipat e s the i r re vision a s we work to furthe r e n sure a nd ma x imize information qua lity

.

Intr oduc tion 4**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  2 EPA Mission and Com m i tm ent to Quality 2.1 E PA’s Mi ssi on an d Com m i tm en t to P u b l i cAccess The mission of the EPA is to protect human health and safeg u ard the natural environment upon whic h life de pe nds. EPA is c o mmitte d to making Ame ric a ' s a i r c l e a ne r , wate r pure r , a nd la nd better protected and to work closely with its F e deral, State, Tribal, and local g overnment partners; with citiz ens; and with the reg ulated community to accomplish its mission. In addition, the United States play s a leadership role in working with other nations to protect the g l obal environment. EPA' s commitment to ex panding and enhancing access to environmental information is articulated in our Strateg i c Plan. EPA works every day to ex pand the public' s rig h t to know about and understand their environment by providing and facilitating access to a wealth of information about public health and local environmental issues and conditions. This enhances citizen understanding and involvement and provides people with tools to protect their families a nd the i r c o mmunitie s. EPA statutory responsibilities to protect human health and safeg u ard the natural environment are described in the statutes that mandate and g overn our prog rams. EPA manag e s those prog rams in concertwi th num erous other g overnm e ntand pri v at e sect or part ners. As C ong ress i nt e nded, each statute provide s re g u latory e x pe c t ations inc l uding information qua lity c onside r ations a nd pri n ci pl es. S om e st atut es are m o re speci fi c th an others, butoveral l , each di rect s EP A and other ag encies in how we reg u late to protect human health and the environment. F o r ex ample, the Safe Drinking Water Act (SDWA) Amendments of 1996 set forth certain quality principles for how EPA should conduct human health risk assessments and characteriz e the potential risks to humans from drinking water contaminants. In formation quality is a key component of every statute that g overns our mission. 2.2 Inf o rm ation Managem e nt in EPA The collection, use, and dissemination of information of known and appropriate quality are integ r al to ensuring that EPA achieves its mission. In formation about human health and the environment -- environmental characteristics; phy sical, chemical, and biolog ical processes; and chemical and other pollutants -- underlies all environmental manag e ment and health protection decisions. The availability of, and access to, information and the analytical tools to understand it are essential for assessing environmental and human health risks, desig n ing appropriate and cost-effective policies and response strateg i es, and measuring environmental improvements. EPA works e ve ry da y to e n sure information qua lity , but we do not wa it until the point of disse mination to c onside r importa nt qua lity princ i ple s . While the fina l re vie w of a doc ume nt before it is published is very important to ensuring a product of hig h quality , we know that in orde r to ma x imize qua lity , we must start muc h e ar lie r. Whe n y ou re a d a n EPA re port aty our local library or view EPA information on our web site, that information is the result of processes

EPA Mis s i on and Com m i tm ent to Q uality 5**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  undertaken by EPA and our partners that assured quality along each step of the way

. To better de sc ribe this inte rre late d information qua lity proc e ss, the following pre s e n ts some of the ma jor role s thatEPA pla y s in its e ffort to e n sure a nd ma x imize the qua lity of the information:

• EPA is a collector and generator of inf o rm ation: W h ile most of our prog rams rely on States, Tribes, or the private sector to collect and report information to EPA, there are some prog rams in which EPA collects its own information. One ex ample is the Ag ency ' s enforcement and compliance prog ram, under which EPA collects samples in the field or conducts onsite inspections. W e also conduct orig inal, scientific research at headquarters, in Reg i onal Offices, and at our research laboratories to investig ate and better understand how our environment works, how hum ans react t o chem i cal pol l ut a nt s and other envi ronm ent a l contaminants, and how to model our natural environment to assess the potential impa c tof e nvironme n ta l ma na g e me nt a c tivitie s. Ensuring the qua lity of c o lle c t e d information is central to our mission.

• E PA i s a reci p i en t of i n f o rm ati on : EP A receives a larg e amount of information that ex ternal parties volunteer or provide under statutory and other mandates. Muc h of the e nvironme n ta l information submitte d to EPA is proc e sse d a nd store d in Ag ency information manag e ment sy stems. W h ile, we work to ensure and ma x imize the inte g rity of thatinformation throug h a varie t y of me c ha n isms a nd polic ie s, we ha ve varying le ve ls of qua lity c ontrols ove r information de ve lope d or collected by outside parties. This information g e nerally falls into one of four cat eg ori e s:

 In f o rm ati on col l ected th rou gh con t racts w i th E PA . Ex amples of this information include studies and collection and analy s is of data by parties that are under a contractual oblig ation with EPA. Since EPA is responsible for manag i ng the work assig n ed to contractors, EPA has a relatively hig h deg r ee of control over the quality of this information.

 In f o rm ati on col l ected th rou gh gran ts an d coop erati ve agreem en ts wi t h EPA . Ex amples of this information include scientific studies that are performed under research g r ants and data collected by State ag encies or other g r antees to assess reg u latory compliance or environmental trends. Althoug h EPA has less control over g r antees than contractors, EPA can and does include conditions in g r ants and cooperative ag reements requi ri ng reci pi ent s t o m eetcert a i n cri t e ri a.

 Infor m at ion subm it t e d t o EPA as par t of a r e quir e m e n t unde ra statute, regulation, perm it, order or other m a ndate . Ex amples of this information include required test data for pesticides or chemicals, Tox i cs Release Inventory (TRI ) submissions and compliance information submitte d to EPA by Stat e s a nd the re g u late d c o mmunity . EPA e n sure s
EPA Mis s i on and Com m i tm ent to Q uality 6**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  qua lity c ontrol of suc h information throug h re g ulatory re quire me nts, suc h as requi ri ng sampl e s t o be anal y z ed by speci fi c anal yt i cal procedures and by cert i f i e d l aborat ori e s. However, each EP A prog ramhas speci fi c statutory au thoritie s whic h ma y a ffe c tits ab ility to impose c e rta in qua lity pract i ces.

 The final categ ory of information that is not included in any of the above th ree cat eg ori e s i n cl udes inf o rm ation that is either voluntarily subm itted to EPA in hopes of inf l uencing a decision or that EPA obtains f o r use in developing a policy, regulatory, or other decision

. Ex amples of this information include scientific studies published in journal articles and test data obtained from other F e deral ag encies, industry , and others. EPA may not have any financial ties or reg u latory re quire me nts to c ontrol the qua lity of this ty pe of information. While the quality of information submitted to EPA is the responsibility of the orig inal collector of the information, we nevertheless maintain a robust quality sy stem, that addresses information related to the first three bullets above by inc l uding re g u latory re quire me nts for qua lity a ssura nc e for EPA c ontra c t s, g r a n ts, and assistance ag reements. F o r the fourth categ ory , we intend to develop and publish fa c t ors thatEPA would use in the future to a sse ss the qua lity of voluntary submissions or information that the Ag ency g athers for its own use.

• EPA is a user of inf o rm ation: Upon placement in our information manag e ment sy stems, information becomes available for use by many people and sy stems. EPA users may include Prog ram manag e rs, information product developers, or automated financial tracking sy stems. Depending on the ex tent of public release, users m a y al so i n cl ude ci t y pl anners, hom eowners, t eachers, eng i neers, or c o mmunity a c tivists, to na me a fe w. To satisfy this broa d spe c trum of use r s, it is critical that we present information in an unbiased contex t with thoroug h documentation. EPA is moving bey ond routine administration of reg u latory information and working in concert with States and other stakeholders to provide new information products that are responsive to identified users. In creasing l y , information products are derived from information orig inally collected to support State or F e de ra l re g u latory prog ra ms or ma na g e me nt a c tivitie s. Assuring the suitability of this information for new applications is of paramount importance.

• EPA is a c o nduit forinfor m at ion: Anothe r ma jor role thatEPA pla y s in the m a nag e m e ntof i n form at i on i s as a provi der of publ i c access. S u ch access enabl e s public involvement in how EP A achieves it mission. W e provide access to a variety of information holding s. Some information distributed by EPA includes i n form at i on col l ect ed th roug h contract s; i n form at i on col l ect ed th roug h g r ant s and

EPA Mis s i on and Com m i tm ent to Q uality 7**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  cooperative ag reements; information submitted to EPA as part of a requirement under a statute, reg u lation, permit, order, or other mandate; and information that is either voluntarily submitted to EPA in hopes of influencing a decision or that EPA obtains for use in developing a policy , reg u latory , or other decision. In some cases, EPA serves as an important conduit for information g e nerated by ex ternal parties; however, the quality of that information is the responsibility of the ex ternal information developer, unless EPA endorses or adopts it. 2.3 EPA's Relationship w i th State, Tribal, and Local G overnm e nts As mentioned in the previous section, EPA works with a variety of partners to achieve its mission. Our key g overnment partners not only provide information, they also work with EPA to manag e and implement prog rams and communicate with the public about issues of concern. In addition to implementing national prog rams throug h EPA Headquarters Prog ram Offices, a vast network of EPA Reg i ons and other F e deral, State, Tribal and local g overnments implement both mandated and voluntary prog rams. This same network collects, uses, and distributes a wide rang e of information. EPA plans to coordinate with these partners to ensure the Guidelines are appropriate and effective. One ma jor me c ha n ism to e n sure a nd ma x imize information inte g rity is the Nationa l Environmental In formation Ex chang e Network (NEIEN, or Network). The result of an important partnership between EPA, States and Tribal g overnments, the Network seeks to enhance the Ag e nc y ' s information ar c hite c ture to e nsure time ly a nd one -stop re porting from ma ny of EPA’s information partners. Key components include the establishment of the Central Data Ex chang e (CDX) portal and a Sy stem of Access for internal and ex ternal users. W h en fully implemented, the Network and its many components will enhance EPA and the public’s ability to access, use, a nd inte g r at e information a nd the ab ility of e x te rna l provide rs to re port to EPA.

EPA Mis s i on and Com m i tm ent to Q uality 8**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  3 O M B G u ide line s In Section 515(a) of the Treasury and General Government Appropriations Act for F i scal Year 2001 (Public L a w 106-554; H.R. 5658), Cong ress directed OMB to issue g overnment-wide g ui d el i n es th at“provi de pol i c y and procedural g ui d ance t o Federal ag enci es for ensuri ng and ma x imizing the qua lity , obje c tivity , utility , a nd inte g rity of information (inc luding statistic a l information) disseminated by F e deral ag encies....” The OMB g u idelines direct ag encies subject to the Paperwork Reduction Act (44 U.S.C. 3502(1)) to:

• I ssue the i r own information qua lity g u ide line s to e n sure a nd ma x imize the quality , objectivity , utility , and integ rity of information, including statistical information, by no later than one y ear after the date of issuance of the OMB g u ide line s ;

• E st abl i s h adm i ni s tr at i v e m echani s m s al l o wi ng affect ed persons t o seek and obt ai n correction of information maintained and disseminated by the ag ency that does not c o mply with the OMB or a g e nc y g u ide line s ; a nd

• R eport t o th e Di rect or of OMB th e number and natu re of compl a i nt s recei ved by th e ag ency reg ardi ng ag ency compl i a nce wi th OMB g ui d el i n es concerni ng the qua lity , obje c tivity , utility , a nd inte g rity of information a nd how suc h c o mpla ints were resolved. The OMB g u idelines provide some basic principles for ag encies to consider when developing their own g u idelines including :

• G uidelines should be flex ible enoug h to address all communication media and variety of scope and importance of information products.

• S om e ag ency i n form at i on m a y need t o m eethi g h er or m o re speci fi c ex pect at i ons for objectivity , utility , and integ rity . In formation of g r eater importance should be he ld to a hig he r qua lity sta ndard.

• E nsuring a nd ma x imizing qua lity , obje c tivity , utility , a nd inte g rity c o me s ata cost, so ag encies should use an approach that weig hs the costs and benefits of hig he r information qua lity

.
• A g e ncies should adopt a common sense approach that builds on ex isting processes and procedures. Itis important that ag ency g u idelines do not impose unnecessary administrative burdens or inhibit ag encies from disseminating quality information to the public.

O M B G u idelines 9**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  4 E xisting P o licies and P r ocedures that Ensure and Maxim i ze Inf o rm ation Quality EPA is dedicated to the collection, g e neration, and dissemination of hig h quality information. W e disseminate a wide variety of information products, rang ing from comprehensive scientific a sse ssme n ts of pote n tia l he a lth risks, 2 to web-based applications that provide compliance information a nd map the loc ation of re g u la te d e n titie s, 3 to simple fact sheets for school children. 4 As a result of this diversity of information-related products and practices, different EPA prog ra ms ha ve e volve d spe c i a lize d approa c he s to information qua lity a ssura nc e . The OMB g ui d el i n es encourag e ag enci es t o avoi d th e creat i on of “new and pot ent i a l l y dupl i cat i v e or contradictory processes.” F u rther, OMB stresses that its g u idelines are not intended to “impose unnecessary administrative burdens that would inhibit ag encies from continuing to take advantag e of the In ternet and other technolog ies to disseminate information that can be of g r eat benefit and value to the public.” In this spirit, EPA seeks to foster the continuous improvement of e x isting information qua lity a c tivitie s a nd prog ra ms. In imple me nting the s e g u ide line s , we note thate n suring the qua lity of information is a ke y obje c tive a l ong side othe r EPA obje c tive s , such as ensuring the success of Ag ency missions, observing budg et and resource priorities and restraints, and providing useful information to the public. EPA intends to implement these Guide line s in a wa y thatwill a c h ie ve a ll the s e obje c tive s in a harmonious wa y in c onjunc tion with our e x isting g u ide line s a nd polic ie s, some of whic h ar e outline dbe low. The s e e x a mple s illustrate some of the numerous sy stems and practices in place that address the quality , objectivity , utility , and integ rity of information. 4.1 Quality System The EPA Ag e nc y -wide Qua lity Sy ste mhe lps e n sure thatEPA org a nizations ma x imize the quality of environmental information, including information disseminated by the Ag ency . A g r a de d approa c h is use dto e s tablish qua lity c rite ria thatar e appropriate for the inte nde d use of the information a nd the re sourc e s a va ilable

. The Qua lity Sy ste mis doc ume n te d in EPA Orde r 5360.1 A2, “Policy and Prog ram Requirements for the Mandatory Ag ency -wide Quality Sy ste m ” a nd the “ E PA Qua lity Ma nua l.” 5 To imple me nt the Qua lity Sy ste m , EPA org a nizations (1) assig n a quality assurance manag e r, or person assig n ed to an equivalent position, who has sufficient technical and manag e ment ex pertise and authority to conduct independent oversig ht of the imple me ntation of the org a nization' s qua lity sy ste m; (2) de ve lop a Qua lity Ma na g e me nt Plan, which documents the org a nization' s quality sy stem; (3) conduct an annual assessment of the org a niz ation' s quality sy stem; (4) use a sy stematic planning process to develop acceptance or performance criteria prior to the initiation of all projects that involve environmental information

2 h ttp ://cf p u b .ep a .g o v/n cea/cf m

/ p artm a tt.cf m 3 h ttp : / /www. e p a . g o v/ e n v i r o /w me /
4 h ttp : / /www . e p a . g o v/ ki d s 5 EP AQu alityMan u a l f o r En v i ron men tal P r og ram s 5360 A 1 . May2000. h ttp : / /www . e p a . g o v/ q u a lit y /q s - doc s / 5360.pdf Ex is ting Polic ies and Pr oc edur es that Ens u r e and Max i m i z e Inf o r m ation Q uality 10**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA** 
c o lle c tion a nd/or use ; (5) de ve lop Qua lity Assura nc e Proje c tPla n (s), or e quiva le nt doc ume n t(s) for all applicable projects and tasks involving environmental data; (6) conduct an assessment of ex isting data, when used to support Ag ency decisions or other secondary purposes, to verify that the y ar e of suffic i e n t qua ntity a nd a de quate qua lity for the i r inte nde d use ; (7) imple me nt a l l Ag ency -wide Quality Sy stem components in all applicable EPA-funded ex tramural ag reements; and (8) provide appropriate training , for all levels of manag e ment and staff. The EPA Quality Sy stem may also apply to non-EPA org a nizations, with key principles incorporated in the applicable reg u lations g overning contracts, g r ants, and cooperative ag reements. EPA Quality Sy stem provisions may also be invoked as part of neg o tiated ag reements such as memoranda of understanding . Non-EPA org a niz ations that may be subject to EPA Quality Sy stem requirements include (a) any org a nization or individual under direct contract to EPA to furnish services or items or perform work (i.e., a contractor) under the authority of 48 CF R part 46, (including applicable work assig n ments, delivery orders, and task orders); and (b) other g overnm e ntag enci es recei vi ng assi st ance fromEP A th roug h i nt e rag e ncy a g re e m e n ts. Se parate qua lity a ssura nc e re quire me nts for a ssista nc e re c i pie n ts ar e se t forth in 40 CF R part 30 (g overning assistance ag reements with institutions of hig h er education, hospitals, and other non-profit recipients of financial assistance) and 40 CF R parts 31 and 35 (g overnment assi st ance ag reem ent s wi thS t at e, Tri b al , and l o cal g overnm e nt s). 4.2 P eer Revi ew P ol i c y In a ddition to the Qua lity Sy ste m , EPA' s Pe e r Re vie w Polic y provide s thatma jor sc ie ntific a lly and technically based work products (including scientific, eng i neering , economic, or statistical documents) related to Ag ency decisions should be peer-reviewed. Ag ency manag e rs within Headquarters, Reg i ons, laboratories, and field offices determine and are accountable for the decision whether to employ peer review in particular instances and, if so, its character, scope, a nd timing

. The s e de c i sions ar e ma de c onsiste nt with prog ra m g oa l s a nd prioritie s, re sourc e constraints, and statutory or court-ordered deadlines. F o r those work products that are intended to support the most important decisions or that have special importance in their own rig h t, ex ternal peer review is the procedure of choice. F o r other work products, internal peer review is an acceptable alternative to ex ternal peer review. Peer review is not restricted to the penultimate version of work products; in fact, peer review at the planning stag e can often be ex tremely beneficial. The basis for EPA peer review policy is articulated in Peer Review and Peer Involvement at the U.S. Environmental Protection Agency

. 6 The P eer R e vi ew P ol i c y was fi rst issued in J a nuary , 1993, and was updated in J une, 1994. In addition to the policy , EPA has published a Peer Review Handbook, 7 whic h provide s de ta ile d g u ida nc e for imple me nting the policy

. The handbook was last revised December, 2000.

6 Peer R e v i ewan d Peer In v o l v emen t atth e U

. S

. EPA

. Ju n e 7, 1994. h ttp : / /www. e p a . g o v/ o s p / s p c / p e r e v me m . h t m 7 P eer R e v i ewHan dbook , 2n d Edition , U.S. EP A , Scien ce P o licyC o u n c il, Decem ber 2000, EP A 100- B- 00- 001

. h ttp : / /www. e p a . g o v/ o s p / s p c / p rh a n d b k

. p d f Ex is ting Polic ies and Pr oc edur es that Ens u r e and Max i m i z e Inf o r m ation Q uality 11**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  4.3 Action Developm ent P r ocess The Ag e nc y ’s Ac tion De ve lopme n t Proc e ss a l so se rve s to e n sure a nd ma x imize the qua lity of EPA disseminated information. Top Ag ency actions and Economically Sig n ificant actions as desig n ated under Ex ecutive Order 12866 are developed as part of the Ag ency ' s Action De ve lopme n t Proc e ss. The Ac tion De ve lopme n t Proc e ss e n sure s the e ar ly a nd time ly involvement of senior manag e ment at key decision milestones to facilitate the consideration of a broad rang e of reg u latory and non-reg u latory options and analytic approaches. Of particular importance to the Action Development Process is ensuring that our scientists, economists, and others with technical ex pertise are appropriately involved in determining needed analy s es and research, identifyi ng alternatives, and selecting options. Prog ram Offices and Reg i onal Offices are invited to participate to provide their unique perspectives and ex pertise. Effective consultation with policy advisors (e.g ., Senior Policy Council, Science PolicyCouncil), coreg u lators (e.g ., States, Tribes, and local g overnments), and stakeholders is also part of the process. Fi nal Ag ency R e vi ew (FAR ) g e neral l y t akes pl ace before th e rel ease of subst a nt i ve i n form at i on associ at ed wi thth ese act i ons. The FARprocess ensures th e consi s t e ncy of any policy determinations, as well as the quality of the information underlyi ng each policy de te rmination a nd its pre s e n tation. 4.4 In tegrated E rror Correcti on P r ocess The Ag ency ’s Int e g r at ed Error C o rrect i on P r ocess 8 (IECP) is a process by which members of the public can notify EPA of a potential data error in information EPA distributes or disseminates. This process builds on ex isting data processes throug h which discrete, numerical errors in our data sy stems are reported to EPA. The IE CP has made these tools more prominent and easier to use. Individuals who identify potential data errors on the EPA web site can contact us throug h the IE CP by using the " R eport Error" button or error correction hy pertex t found on major data bases throug hout EPA' s web site. EPA reviews the error notification and assists in bring i ng the notification to resolution with those who are responsible for the data within or outside the Ag ency , as appropriate. The IE CP tracks this entire process from notification throug h final resolution.

8 I n teg r ated Err o rCo rr ectio nP r o cess f o rEn v i r o n men tal Data. h ttp : / /www. e p a . g o v/ c d x/i e c p . h t ml Ex is ting Polic ies and Pr oc edur es that Ens u r e and Max i m i z e Inf o r m ation Q uality 12**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  4.5 In f o rm ati on Resou rces Man agem e n t Man u al The EPA In formation Resources Manag e ment (I RM) Manual 9 articulates and describes many of our information development and manag e ment procedures and policies, including information security , data standards, records manag e ment, information collection, and library services. Especially important in the contex t of the Guidelines provided in this document, the I R M Manual describes how we maintain and ensure information integ rity . W e believe that ma inta ining information inte g rity re fe rs to ke e p ing information " una lte re d," i.e ., fre e from unauthoriz ed or accidental modification or destruction. These integ rity principles apply to all information. In appropriately chang e d or modified data or software impacts information integ rity and comp rom i s es th e val u e of th e i n form at i on sy st em . Because of th e i mport a nce of EP A' s information to the decisions made by the Ag ency , its partners, and the public, it is our responsibility to ensure that the information is, and remains, accurate and credible. B e y ond addressing integ rity concerns, the I R M Manual also includes Ag ency policy on public access and records m a nag e m e nt . These are key chapt e rs th atenabl e EP A t o ensure tr ansparency and the reproducibility of information. 4.6 Risk Characterization P o licy and Handbook The EPA Risk Characteriz ation Policy and Handbook 10 provide g u idance for risk charact eri z at i on th at i s desi g n ed t o ensure th atcri t i cal i n form at i on fromeach st ag e of a ri sk assessment is used in forming conclusions about risk. The Policy calls for a transparent process and products that are clear, consistent and reasonable. The Handbook is desig n ed to provide risk assessors, risk manag e rs, and other decision-makers an understanding of the g o als and principles of ri sk charact eri z at i on. 4.7 P r ogram -Specif i c P o licies We me ntione d just a fe w of the Ag e nc y ' s ma jor polic ie s thate n sure a nd ma x imize the qua lity of information we disseminate. I n addition to these Ag ency -wide sy stems and procedures, Prog ram Offices and Reg i ons implement many Office-level and prog ram-specific procedures to ensure a nd ma x imize information qua lity . The purpose of the s e Guide line s is to se rve a s a c o mmon thread that ties all these policies tog e ther under the topics provided by OMB : objectivity , integ rity and utility . EPA' s approach to ensuring and max imiz ing quality is necessarily distributed across all levels of EPA’s org a niz ational hierarchy , including Offices, Reg i ons, divisions, projects, and even products. Oftentimes, there are different quality considerations for different types of products. F o r ex ample, the quality principles associated with a risk assessment

9 EPAD i rect i v e 2100 In f o rm at i o n R e s o u r ces Man a g e me n t Pol i c yMan u a l

. h ttp : / /www. e p a . g o v/i r mp o l i 8 / p o l ma n/
10 R i s k C h aracterizationHan dbook , U.S

. EP A , S c ien ce P o licyC o u n c il, Decem ber 2000. h ttp : / /www. e p a . g o v/ o s p / s p c / 2 r i s k c h r . h t m Ex is ting Polic ies and Pr oc edur es that Ens u r e and Max i m i z e Inf o r m ation Q uality 13**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  differ from those associated with developing a new model. The Ag ency currently has a comprehensive but distributed sy ste mof polic ie s to a ddre ss suc h unique qua lity c onside r ations. The s e Guide line s provide us with a me c ha n ism to he lp c oordinate a nd synthe size our qua lity policies and procedures. 4.8 EPA C om m i t m e n tt o C o nt inuous Im pr ove m e nt As sug g e sted above, we will continue to work to ensure that our many policies and procedures are appropriately implemented, synthesiz e d, and revised as needed. One way to build on achievements and learn from mistakes is to document lessons learned about specific activities or products. F o r ex ample, the documents that present g u idance and tools for implementing the Qua lity Sy ste mar e routine l y subje c t e dto e x te rna l pe e r re vie w during the i r de ve lopme n t; comments from the reviewers are addressed and responses reviewed by manag e ment before the document is issued. Each document is formally reviewed every five y ears and is either reissued, revised as needed, or rescinded. I f important new information or approaches evolve between reviews, the document may be reviewed and revised more frequently

. 4.9 Sum m a ry of New Activities and Initiatives In response to OMB ' s g u idelines, EPA recog n izes that it will be incorporating new policies and a d ministra tive me c ha n isms. As we re a ffirm our c o mmitme nt to our e x isting polic ie s a nd proc e dure s thate n sure a nd ma x imize qua lity , we a l so pla n to a ddre ss the following ne w ar e a s of foc u s a nd c o mmitme nt:

• W orking with the public to de ve lop a sse ssme n t fa c t ors thatwe will use to a sse ss the qua lity of information de ve lope d by e x te rna l partie s, prior to EPA’s use of thatinformation.

• A ffirming a ne w c o mmitme nt to information qua lity , e s pe c i a lly the tra n spar e nc y of information products.

• E stablishing Ag ency -wide correction process and request for reconsideration panel t o provi de a centr al i z ed poi ntof access for al l affect ed part i e s t o seek and obtain the correction of disseminated information that they believe does not c onform to the s e Guide line s or the OMB g u ide line s .

Ex is ting Polic ies and Pr oc edur es that Ens u r e and Max i m i z e Inf o r m ation Q uality 14**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  5 G uide line s Sc ope and Applic ability 5.1 What is “Quality” According to the G u idelines? Consiste nt with the OMB g u ide line s , EPA is issuing the s e Guide line s to e n sure a nd ma x imize the quality , including objectivity , utility and integ rity , of disseminated information. Objectivity , inte g rity , a nd utility ar e de fine d he re , c onsiste nt with the OMB g u ide line s . “ O bje c tivity ” foc u se s on whether th e di ssem i n at ed i n form at i on i s bei ng present e d i n an accurat e , cl ear, compl e t e , and unbiased manner, and as a matter of substance, is accurate, reliable, and unbiased. “Integ rity ” refers to security , such as the protection of information from unauthoriz ed access or revision, to ensure that the information is not compromised throug h corruption or falsification. “Utility ” refers to the usefulness of the information to the intended users. 5.2 What is the P u rpose of these G u idelines? The collection, use, and dissemination of information of known and appropriate quality is integ r al to ensuring that EPA achieves its mission. In formation about the environment and human health underlies all environmental manag e ment decisions. In formation and the analytical tools to understand it are essential for assessing environmental and human health risks, desig n ing appropriate and cost-effective policies and response strateg i es, and measuring environmental improvements. These Guidelines describe EPA’s policy and procedures for reviewing and substantiating the qua lity of information be fore EPA disse minat e s it. The y de sc ribe our a d ministrative me c ha n isms for enabling affected persons to seek and obtain, where appropriate, correction of information disseminated by EPA that they believe does not comply with EPA or OMB g u idelines. 5.3 When Do these G u idelines Apply? These Guidelines apply to “information” EPA disseminates to the public. “Information,” for purposes of these Guidelines, g e nerally includes any communication or representation of knowledg e such as facts or data, in any medium or form. Preliminary information EPA disseminates to the public is also considered “information” for the purposes of the Guidelines. In formation g e nerally includes material that EPA disseminates from a web pag e . However not all web content is considered " i nformation" under these Guidelines (e.g ., certain information from outside sources that is not adopted, endorsed, or used by EPA to support an Ag ency decision or position). F o r purposes of these Guidelines, EPA disseminates information to the public when EPA initiates or sponsors the distribution of information to the public.

• E PA initiate s a distribution of information if EPA pre par e s the information a nd distributes it to support or represent EPA’s viewpoint, or to formulate or support a reg u lation, g u idance, or other Ag ency decision or position.

G u idelines Sc ope and Applic ability 15**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA** 

• E PA initia te s a distribution of information if EPA distribute s information pre par e dor submitte d by a n outside party in a ma nne r thatre a s onably sug g e sts that EPA endorses or ag rees with it; if EPA indicates in its distribution that the information supports or represents EPA’s viewpoint; or if EPA in its distribution proposes to use or uses the information to formulate or support a reg u lation, g u idance, policy , or other Ag ency decision or position.

• A g e ncy - sponsored distribution includes instances where EPA reviews and comments on information distributed by an outside party in a manner that indicates EPA is endorsing it, directs the outside party to disseminate it on EPA’s behalf, or otherwise adopts or endorses it. EPA intends to use notices to ex plain the status of information, so that users will be aware of whether the information is being distributed to support or represent EPA’s viewpoint. 5.4 Wh at i s Not Covered b y th ese Gu i d el i n es? I f

an item is not considered “information,” these Guidelines do not apply

. Ex amples of items that are not considered information include In ternet hyperlinks and other references to information distributed by others, and opinions, where EPA’s presentation makes it clear that what is being offered is someone’s opinion rather than fact or EPA’s views. “Dissemination” for the purposes of these Guidelines does not include distributions of information that EPA does not initiate or sponsor. B e low is a sample of various types of information that would not g e nerally be considered disseminated by EPA to the public:

• D istribution of information intended only for g overnment employ ees (including intra- or interag e ncy use or sharing ) or recipients of g overnment contracts, g r ants, or cooperative ag reements. In tra-ag ency use of information includes use of information pertaining to basic ag ency operations, such as manag e ment, personnel, and org a niz ational information.

• E PA’s response to requests for ag ency records under the Fr eedom of In formation Ac t (F OIA), the Priva c y Ac t, the F e de ra l Advisory Committe e Ac t (F ACA), or othe r similar la ws.

• D istribution of information in correspondence directed to individuals or persons (i.e., any individual, g r oup, or entity , including any g overnment or political subdivision thereof, or F e deral g overnmental component/unit).

• Information of an ephemeral nature, such as press releases, fact sheets, press conferences, and similar communications, in any medium that advises the public of an event or activity or announces information EPA has disseminated

G u idelines Sc ope and Applic ability 16**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  elsewhere; interviews, speeches, and similar communications that EPA does not disseminate to the public bey ond their orig inal contex t, such as by placing them on the I n ternet. I f a speech, press release, or other “ephemeral” communication is about an information product disseminated elsewhere by EPA, the product itself will be c ove re d by the s e Guide line s .
• Information presented to Cong ress as part of the leg i slative or oversig ht processes, such as testimony of officials, information, or drafting assistance provided to Cong ress in connection with pending or proposed leg i slation, unless EPA simulta ne ously disse minat e s this information to the public

.
• B ackg r ound information such as published articles distributed by libraries or by other distribution methods that do not imply that EPA has adopted or endorsed the materials. This includes outdated or superseded EPA information that is provided as backg r ound information but no long er reflects EPA policy or influences EPA decisions, where the outdated or superseded nature of such material is reasonably apparent from its form of presentation or date of issuance, or where EPA indicates that the materials are provided as backg r ound materials and do not represent EPA’s current view.

• T hese Guidelines do not apply to information distributed by recipients of EPA contracts, g r ants, or cooperative ag reements, unless the information is disseminated on EPA’s behalf, as when EPA specifically directs or approves the dissemination. These Guidelines do not apply to the distribution of any type of research by F e deral employ ees and recipients of EPA funds, where the researcher (not EPA) decides whether and how to communicate and publish the research, does so i n th e sam e m a nner as hi s or her academ i c col l eag ues, and di stri but es the research i n a m a nner th at i ndi cat es i tdoes notnecessari l y representEP A’s offi ci al position (for ex ample, by including an appropriate disclaimer). The Guidelines do not apply even if EPA retains ownership or other intellectual property rig h ts because th e Federal g overnm e ntpai dfor th e research.

• D istribution of information in public filing s to EPA, inc l uding information submitted to EPA by any individual or person (as discussed above), either voluntarily or under mandates or requirements (such as filing s required by statute s , re g u lations, orde rs, pe rmits, or lic e n se s). The Guide line s do not apply where EPA distributes this information simply to provide the public with quicker and easier access to materials submitted to EPA that are publicly available. This will g e nerally be the case so long as EPA is not the author, and is not endorsing , adopting , using , or proposing to use the information to support an Ag ency decision or position.

• D istribution of information in documents filed in or prepared specifically for a judic i a l c a s e or a n a d ministrative a d judic ation a nd inte nde d to be limite d to suc h

G u idelines Sc ope and Applic ability 17**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  actions, including information developed during the conduct of any criminal or civil action or administrative enforcement action, investig ation, or audit involving an ag ency ag ai nstspeci fi c part i e s. 5.5 Wh at H ap p e n s i f In f o rm ati on i s In i t i al l y Not Covered b y th ese Gu i d el i n es, b u t E PA Subse que nt ly D i sse m i nat e s it t o t h e P ublic ? I f

a particular distribution of information is not covered by these Guidelines, the Guidelines may still apply to a subsequent dissemination of the information in which EPA adopts, endorses, or uses the information to formulate or support a reg u lation, g u idance, or other Ag ency decision or position. F o r ex ample, if EPA simply makes a public filing (such as facility data required by reg u lation) available to the public, these Guidelines would not apply to that distribution of information. However, if EPA later includes the information in a backg r ound document in support of a rulemaking , these Guidelines would apply to that later dissemination of the information in that document. 5.6 How doe s EPA Ensur e the Obje c t ivity, Utility, and Inte gr ity of infor m ation that is n o t covered b y th ese Gu i d el i n es? The s e Guide line s apply only to information EPA disse minat e s to the public , outline din se c tion 5.3, above. Other information distributed by EPA that is not covered by these Guidelines is still subje c tto a ll applic ab le EPA polic ie s, qua lity re vie w proc e sse s, a nd c o rre c tion proc e dure s . The s e inc l ude qua lity ma na g e me nt pla n s for prog ra ms thatc o lle c t , ma na g e , a nd use environmental information, peer review, and other procedures that are specific to individual prog rams and, therefore, not described in these Guidelines. Itis EPA’s policy that all of the information it distribute s me e t s a ba sic sta ndard of information qua lity , a nd thatits utility , obje c tivity , a nd inte g rity be sc a l e da nd appropriate to the nature a nd time line ss of the pla nne d and anticipated uses. Ensuring the quality of EPA information is not necessarily dependent on any plans to disseminate the information. EPA continues to produce, collect, and use information thatis of the appropriate qua lity , irre spe c tive of the s e Guide line s or the prospe c t s for dissemination of the information.

G u idelines Sc ope and Applic ability 18**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  6 G uide line s forEnsur i ng and M axim i z i ng Infor m at ion Qualit y 6.1 How does EPA Ensure and Maxim i ze the Quality of Dissem i nated Inf o rm ation? EPA e n sure s a nd ma x imize s the qua lity of the information we disse minat e by imple me nting we ll e s tablishe d policies and procedures within the Ag ency as appropriate to the information product. The r e ar e ma ny tools thatthe Ag e nc y use s suc h a s the Qua lity Sy ste m , 11 review by senior manag e ment, peer review process, 12 communications product review process, 13 the we b g uide , 14 and th e error correct i on process. 15 B e y ond our internal quality manag e ment sy stem, EPA also ensures the quality of information we disseminate by seeking input from ex perts and the g e neral public. EPA consults with g r oups such as the Science Advisory B o ard and the Science Advisory Panel, in addition to seeking public input throug h public comment periods and by hosting public me e ting s . For th e purposes of th e Gui d el i n es, EP A recog ni z es th at i f dat a and anal yt i c resul t s are subj ect ed to formal, independent, ex ternal peer review, the information may g e nerally be presumed to be of accept abl e obj ect i vi t y . However, thi s presumpt i on of obj ect i vi t y i s rebut t abl e . The Ag ency use s a g r a de dapproa c h a nd use s the s e tools to e s tablish the appropriate qua lity , obje c tivity , utility , and integ rity of information products based on the intended use of the information and the resources available. As part of this g r aded approach, EPA recog n iz es that some of the information it disse minat e s inc l ude s influe ntia l sc ie ntific , fina nc ia l, or statistic a l information, and that this categ ory should meet a hig h er standard of quality

. 6.2 How Doe s EPA De fine Influe ntial Infor m ation forthe s e G u ide line s ?

“ Influe n tia l,” whe n use din the phra s e “ i nflue n tia l sc ie ntific , fina nc ia l, or statistic a l information,” means that the Ag ency can reasonably determine that dissemination of the information will ha ve or doe s ha ve a c l e ar a nd substa ntia l impa c t(i.e ., pote n tia l c ha ng e or e ffe c t ) on important public policies or private sector decisions. 16 F o r the purposes of the EPA' s

11 EP AQu alityMan u a l f o r En v i ron men tal P r og ram s 5360 A 1 . May2000. h ttp : / /www. e p a . g o v/ q u a lity /qs

- docs / 5360.pdf
12 P eer R e v i ewHan dbook , 2n d Edition , U.S. EP A , Scien ce P o licyC o u n c il, Decem ber 2000, EP A 100- B- 00- 001

. h ttp : / /www. e p a . g o v/ o s p / s p c / p rh a n d b k

. p d f
13 EP A ' s P r in t an d W e b C o mm u n i cation s P r odu ct R e v i ewGu ide

. h ttp : / /www. e p a . g o v/ d c e d/ p d f / r e v i e w. p d f
14 W e b Gu id e. U.S. EP A

. h ttp : / /www. e p a . g o v/we b g u i d e / r e s o u r c e s /w e b s e r v . h t ml
15 In teg r ated Error C o rrection P r oces s . h ttp : / /www. e p a . g o v/ c d x/i e c p . h t ml
16 T h e te r m " c le ar a n d sub s ta ntia l im p a c t " is use d a s p ar t o f ad e finitio n to d i stinguish d i ffe r e nt c a t e g o r ie s o f in f o r m atio nf o rp u rp o s es o f th ese Gu id elin es. EP Ad o e s n o t in ten d th e classif i catio no f in f o r m atio nu n d erth is d e f i n itio nto ch an g e o rim p act th e statu s o f th e in f o r m atio ninan yo th e rsettin g , su chas f o rp u rp o s es o f d e ter min in g w h eth e rth e d i ssemi n a tio no f th e in f o r m atio nis a f i n a l A g en cyactio n . G u idelines f o rEns u r i ng and Max i m i z i ng Inf o r m ation Q uality 19**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  In formation Qua lity Guide line s , EPA will g e ne ra lly c onside r the following c l a sse s of information to be influential, and, to the ex tent that they contain scientific, financial, or statistical information, that information should adhere to a rig o rous standard of quality :

• Information disseminated in support of top Ag ency actions (i.e., rules, substantive notices, policy documents, studies, g u idance) that demand the ong oing involvement of the Administrator' s Office and ex tensive cross-Ag ency involvement; issues that have the potential to result in major cross-Ag ency or cross-media policies, are hig h ly controversial, or provide a sig n ificant opportunity to a dva nc e the Administrat or' s prioritie s. Top Ag e nc y a c tions usua lly ha ve potentially g r eat or widespread impacts on the private sector, the public or state, local or tribal g overnments. This categ ory may also include precedent-setting or controversial scientific or economic issues.

• Information disseminated in support of Economically Sig n ificant actions as defined in Ex ecutive Order 12866, entitled Regulatory Planning and Review (58 F R51735, October 4, 1993), Ag ency actions that are likely to have an annual effect on the economy of $100 million or more or adversely affect in a material way the economy , a sector of the economy , productivity , competition, jobs, the environment, public health or safety , or State, Tribal, or local g overnments or c o mmunitie s.

• M ajor work products underg oing peer review as called for under the Ag ency ’s Peer Review Policy

. Described in the Science Policy Council Peer Review Handbook, the EPA Pe e r Re vie w Polic y re g ards ma jor sc ie ntific a nd te c hnic a l work products as those that have a major impact, involve precedential, novel, and/or controversial issues, or the Ag ency has a leg a l and/or statutory oblig ation to conduct a peer review. These Major work products are typically subjected to ex ternal peer review. Some products that may not be considered “major” under the EPA Peer Review Policy may be subjected to ex ternal peer review but EPA does not consider such products influential for purposes of these Guidelines.

• C a s e - by -c a s e : The Ag e nc y ma y make de te rminations of whatc onstitute s “influential information” bey ond those classes of information already identified on a case-by -case basis for other types of disseminated information that may have a clear and substantial impact on important public policies or private sector decisions. 6.3 How Does EPA Ensure and Maxim i ze the Quality of “Inf luential” Inf o rm ation? EPA recog n iz es that influential scientific, financial, or statistical information should be subject to a hig h er deg r ee of quality (for ex ample, transparency about data and methods) than information that may not have a clear and substantial impact on important public policies or private sector decisions. A hig h er deg r ee of transparency about data and methods will facilitate

G u idelines f o rEns u r i ng and Max i m i z i ng Inf o r m ation Q uality 20**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  the reproducibility of such information by qualified third parties, to an acceptable deg r ee of imprecision. F o r disseminated influential orig inal and supporting data, EPA intends to ensure reproducibility according to commonly accepted scientific, financial, or statistical standards. Itis important that analytic results for influential information have a hig h er deg r ee of transparency reg arding (1) the source of the data used, (2) the various assumptions employ ed, (3) the analytic methods applied, and (4) the statistical procedures employ ed. Itis also important that the deg r ee of ri g o r wi th whi c h each of th ese fact ors i s present e d and di scussed be scal ed as appropri at e , and that all factors be presented and discussed. In addition, if access to data and methods cannot oc c u r due to c o mpe lling inte re sts suc h a s priva c y , tra de se c r e t s, inte lle c tua l prope rty , a nd othe r confidentiality protections, EPA should, to the ex tent practicable, apply especially rig o rous robustness checks to analytic results and carefully document all checks that were undertaken. Orig inal and supporting data may not be subject to the hig h and specific deg r ee of transparency provided for analytic results; however, EPA should apply , to the ex tent practicable, relevant Ag ency policies and procedures to achieve reproducibility , g i ven ethical, feasibility , and c onfide n tia lity c onstra i nts. Several Ag ency -wide and Prog ram- and Reg i on-specific policies and processes that EPA uses to e n sure a nd ma x imize the qua lity of e nvironme n ta l data , inc l uding disse minat e dinformation products, would also apply to information considered “influential” under these Guidelines. Ag e nc y -wide proc e sse s of partic ular importa nc e to e n sure the qua lity , obje c tivity , a nd tra n spar e nc y of “ i nflue n tia l” information inc l ude the Ag e nc y ' s Qua lity Sy ste m , Ac tion Development Process, Peer Review Policy , and related procedures. Many “influential” information products may be subject to more than one of these processes.

6.4 How Does EPA Ensure and Maxim i ze the Quality of “Inf luential” Scientif ic Risk Assessm ent Inf o rm ation? EPA conducts and disseminates a variety of risk assessments. W h en evaluating environmental problems or establishing standards, EPA must comply with statutory requirements and mandates set by Cong ress based on media (air, water, solid, and haz ardous waste) or other environmental interests (pesticides and chemicals). Consistent with EPA' s current practices, application of these principles involves a “weig h t-of-evidence” approach that considers all relevant information and its qua lity , c onsiste nt with the le ve l of e ffort a nd c o mple x ity of de ta il appropriate to a partic ular risk a sse ssme n t. In our disse mination of influe ntia l sc ie ntific information re g arding huma n health, safety 17 or environmental 18 risk a sse ssme n ts, EPA will e n sure , to the e x te nt pra c tic ab le

17 “ S af etyris k as s e s s men t” des c ribes a v arietyof an aly s es , in v e s tig ation s , or cas e s tu d ies con d u c ted byEP A to res pon d to en v i ron men tal emerg e n c ies

. For ex am ple, w e w o rk to en s u re th at th e ch em ical in du s tryan d s t ate an d lo c a l e n titie s take a c tio n to p r e v e n t, p l a na n d p r e p ar e fo r , a n d r e sp o n d to c h e mic a l e me r ge nc ie s thr o ugh the dev e lopmen t an d s h arin g of in f o rm ation , tools , an d g u i dan ce f o r h azards an aly s es an d ris k as s e s s men t.
18 B ecause the assessment o f “ e nvir o nmental r i sk” is b e ing d i stinguished f r o m “ hum an health r i sk, ” the ter m " e n v i ron men tal ris k " as u s ed inth es e Gu idelin es does n o t directlyin v o lv e h u m anh ealthcon cern s . Inoth e r w o rds , an “ e n v i ron men tal ris k as s e s s men t” is inth is cas e th e equ i v a len t to w h at EP Acom m o n l ycalls an“ ecolog i cal ris k G u idelines f o rEns u r i ng and Max i m i z i ng Inf o r m ation Q uality 21**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA** 
a nd c onsiste nt with Ag e nc y statute s a nd e x isting le g i slative re g u lations, the obje c tivity 19 of such information disse minat e dby the Ag e nc y by applyi ng the following a dap tation of the qua lity principles found in the Safe Drinking W ater Act 20 (SDW A) Amendments of 1996 21 : (A) The substance of the information is accurate, reliable and unbiased. This involves the use of: (i) the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including , when available, peer reviewed science and supporting studies; and (ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies the use of the data). (B ) The presentation of information on human health, safety , or environmental risks, consistent with the purpose of the information, is comprehensive, informative, and understandable. In a document made available to the public, EPA specifies: (i) each population addressed by any estimate of applicable human health risk or each risk assessment endpoint, including populations if applicable, addressed by a ny e s timate of applic ab le e c o log i c a l risk 22 ; (ii) the e x pe c t e drisk or c e n tra l e s timate of huma n he a lth risk for the spe c i fic

a sse ssme n t”

. 19 OMB stated inits g u i d e lin es th at ind i ssemi n a tin g in f o r m atio nag en cies sh all d e v e lo p a p r o cess f o r r e v i ew in g th e q u alityo f th e in f o r m atio n . “ Q u a lity ” in clu d es o b j ectiv ity , u tility , an d in teg r ity . “ O b j ectiv ity ” in v o lv es tw o dis tin ct elemen ts , pres en tationan d s u bs tan ce. Gu id elin es f o rEn su r i n g an d Max i m i zin g th e Qu ality , Ob j ectiv ity , Utility , an d In teg r ityof In f o rm ationDis s e mi n a ted byFederal A g en cies , OMB , 2002. (67 FR8452) h ttp : / /www. wh i t e h o u s e . g o v/ o m b /f e d r e g / r e p r o d u c i b l e 2

. p d f
20 S a f e D r i n ki n g Wat e r A c tA men dmen t s of 1996, 42 U

. S

. C

. 300g

- 1 (b)(3)(A ) & (B)
21 T h e ex cep tio nis r i sk assessmen ts co n d u c ted u n d erSDW Aw h ichw ill ad h e r e to th e SDW Ap r in cip l es as amen ded i n1996.
22 A g en cyas s e s s men t s of h u m an h eal thri s k s n eces s ari l yf o cu s onpopu l a t i o n s . A g en cyas s e s s men t s of ecolog i cal ris k s addres s a v arietyof en tities , s o me of w h ichcanbe des c ribed as popu lation s an d oth e rs (s u c has ecos y s t em s ) w h i c hcan n o t
. T h e ph ras e " a s s e s s men t en dpoi n t " i s i n t e n d ed t o ref l ectth e broader ran g e of i n t e res t s inhe r e nt in e c o lo gic a l r i sk a sse ssme n ts. A s d i sc usse d in the EPA Guidelines for Ecological Ris k As s e s s m ent ( fo und at h ttp://cf p u b .epa.g ov/n cea/cf m

/ recordis play .cf m ? d eid=12460 ), as s e s s men t en dpoi n t s are ex pl i c i t ex pres s i on s of th e actu a l en v i r o n men tal v a lu e th at is to b e p r o t ected , o p e r a tio n a llyd e f i n e d b yaneco lo g i cal en tityan d its attr ib u t es. Fu r th e r m o r e, th o s e Gu id elin es ex p l ainth at aneco lo g i cal en titycanb e a sp ecies ( e .g ., eelg r ass, p i p i n g p l o v er ) , a co mm u n ity( e .g ., b e n th i c in v e r t eb r a tes) , aneco sy stem ( e .g ., w e tlan d ) , o ro th e ren tityo f co n cer n . A nattr ib u t e o f an as s e s s men t en dpoin t is th e ch aracteris tic abou t th e en tityof con cernth at is im portan t to protect an d poten tiallyat ri s k

. Ex am pl es of attr i b u t es i n cl u d e abu n d an ce (of a popu l a t i o n ) , s p eci es ri ch n e s s (of a comm u n i t y ), or f u n c t i o n(of anecos y s t em ). A s s e s s men t en dpoi n t s an d ecol o g i cal ri s k as s e s s men t s are di s c u s s e d m o re f u l l yi n th os e G u i d el i n es as w e ll as oth e r EP As o u r ces s u chas Ecol ogi cal Ri s k As s e s s m ent G u i dance f o r

Super f

und: Proces s f o r

D e s i gni ng and Conduct i ng Ecol ogi cal Ri s k As s e s s m ent s - Int e r i m Fi nalfo unda t h ttp : / /www. e p a . g o v/ o e rrp a g e / s u p e r f u n d/ p r o g r a m s / r i s k / e c o r i s k / e c o r i s k

. h t m G u idelines f o rEns u r i ng and Max i m i z i ng Inf o r m ation Q uality 22**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  populations affected or the ecolog i cal assessment endpoints 23 , including populations if applicable; (iii) each appropriate upper-bound or lower-bound estimate of risk; (i v) each si g ni f i cantuncert a i nt y i d ent i f i e d i n th e process of th e assessm entof ri sk and studie s thatwould a ssist in re solving the unc e rta inty ; a nd (v) peer-reviewed studies known to the Administrator that support, are directly relevant to, or fail to support any estimate of risk and the methodolog y used to re c onc ile inc onsiste nc ie s in the sc ie ntific data
. In applyi ng the s e princ i ple s , “ be s t a va ilable ” usua lly re fe rs to the a va ilability atthe time a n assessment is made. However, EPA also recog n iz es that scientific knowledg e about risk is rapidly chang i ng and that risk information may need to be updated over time. When deciding which influential risk assessment should be updated and when to update it, the Ag ency will take into account its statutes and the ex tent to which the updated risk assessment will have a clear and substantial impact on important public policies or private sector decisions. I n some situations, th e Ag ency m a y need t o wei g h th e resources needed and th e pot ent i a l del a y associ at ed wi th inc o rporating a dditiona l information in c o mparison to the va lue of the ne w information in te rms of its potential to improve the substance and presentation of the assessment. Adaptation clarif ications In order to provide more clarity on how EPA adapted the SDW A principles in this g u idance in lig ht of our numerous statutes, reg u lations, g u idance and policies that address how to conduct a risk assessment and characteriz e risk we discuss four adaptations EPA has made to the SDW A qua lity princ i ple s la ng ua g e . EPA adapted the SDW A principles by adding the phrase “consistent with Ag ency statutes and e x isting le g i slative re g u lations, the obje c tivity of suc h information disse minat e dby the Ag e nc y ” in the introductory parag r aph, therefore applyi ng to both parag r aphs (A) and (B ). This was done to e x pla i n EPA' s inte nt re g arding the s e qua lity princ i ple s a nd the i r imple me ntation c onsiste nt with our statutes and ex isting leg i slative reg u lations. Also, as noted earlier, EPA intends to imple me nt the s e qua lity princ i ple s in c onjunc tion with our g u ide line s a nd polic ie s. The procedures set forth in other EPA g u idelines set out in more detail EPA' s policies for conducting risk assessments, including Ag ency -wide g u idance on various types of risk assessments and prog ram-specific g u idance. EPA recog n iz es that the wide array of prog rams within EPA have resulted not only in Ag ency -wide g u idance, but in specific protocols that reflect the re quire me nts, inc l uding limitations, thatar e ma ndate d by the various statute s a d ministe re d by the Ag ency . F o r ex ample, the Ag ency developed several pesticide science policy papers that ex plained to the public in detail how EPA would implement specific statutory requirements in the F ood Quality Protection Act (F QPA) that addressed how we perform risk assessments. We also recog n iz e that emerg i ng issues such endocrine disruption, bioeng ineered org a nisms, and g e nomics may involve some modifications to the ex isting paradig mfor assessing human health

23 Ibi d
. G u idelines f o rEns u r i ng and Max i m i z i ng Inf o r m ation Q uality 23**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  and ecolog i cal risks. This does not mean a radical departure from ex isting g u idance or the SDWA princ i ple s , but rathe r indic at e s thatfle x ibility ma y be warra nte d a s ne w information a nd approaches develop. EP A i ntr oduced th e fol l o wi ng t w o adapt at i ons i n order t o accom m odat e th e rang e of real -worl d situations that we confront in the implementation of our diverse prog rams. EPA adapted the SDWA qua lity princ i ple s by moving the phra s e " t o the e x te nt pra c tic ab le " from para g r aph (B ) to the introductory parag r aph in this Guidelines section to cover both parts (A) and (B ) of the S D W A adapt at i on. 24 The phrase refers to situations under (A) where EPA may be called upon to conduct " i nfluential" scientific risk assessments based on limited information or in novel situations, and under (B ) in recog n ition that all such “presentation” information may not be available in every instance. The level of effort and complex ity of a risk assessment should also balance the information needs for decision making with the effort needed to develop such information. F o r ex ample, under the F e deral In secticide, F ung icide and Rodenticide Act 25 (F IF RA) and the Tox i c Substances and Control Act 26 (TSCA), re g u late d e n titie s ar e oblig at e dto provide information to EPA concerning incidents/test data that may reveal a problem with a pest i c i d e or chem i cal . W e al so recei ve such i n form at i on vol unt ari l y fromother sources. EP A carefully reviews incident reports and factors them as appropriate into risk assessments and decision-making , even thoug h these may not be considered information collected by acceptable methods or best available method as stated in A(ii). In cident information play ed an important role in the Ag e nc y ' s c onc lusion thatuse of c h lorda ne/he pta c h lor te rmitic ide s c ould re sult in ex posures to persons living in treated homes, and that the reg i strations needed to be modified according l y . Similarly , incident reports concerning birdkills and fishkills were important components of the risk assessments for the rereg i stration of the pesticides phorate and terbufos, respectively

. In addition, this adaptation recog n iz es that while many of the studies incorporated into risk assessments have been peer reviewed, data from other sources may not be peer reviewed. EPA takes many actions based on studies and supporting data provided by outside sources, including confidential or proprietary information that has not been peer reviewed. F or ex ample, industry can be required by reg u lation to submit data for pesticides under F IF R A or for chemicals under TSCA. The data are developed using test g u idelines and Good L aboratory P r act i ces (GLP s) i n accordance wi th EP A reg ul at i ons. W hi l e th ere i s nota requi rem e nt t o have studi es peer revi ewed, such studi es are revi ewed by Ag ency sci e nt i s t s t o ensure th at th ey were conducted according to the appropriate test g u idelines and GLP s and that the data are valid. The fle x ibility provide d by applyi ng “ t o the e x te nt pra c tic ab le ” to para g r aph (A) is appropriate in many circumstances to conserve Ag ency resources and those of the reg u lated community who othe rwise mig ht ha ve to g e ne ra te sig n ific a n t a dditiona l data . This fle x ibility is a l re a dy provide d

24 T h e dis c u s s i oninth is an d f o llow i n g parag r aph s g i v e s s o me ex am ples of th e ty pes of as s e s s men ts th at m a yu n d er s o me circu m s t an ces be con s idered in f l u e n tial. T h es e ex am ples are repres en tativ e of as s e s s men ts perf orme d u n d er oth e r EPAprog ram s , s u chas C E R C L A
25 7 U.S.C. 136 e ts e q . 26 15 U.S.C. 2601 e ts e q . G u idelines f o rEns u r i ng and Max i m i z i ng Inf o r m ation Q uality 24**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  for para g r aph (B ) in the SDWA qua lity princ i ple s . Pe stic ide a nd c he mic a l risk a sse ssme n ts ar e fre que ntly pe rforme dite rative l y , with the first ite ration e mploying prote c tive (c onse r vative ) a ssumptions to ide n tify possible risks. Only if pote n tia l risks ar e ide n tifie d in a sc re e n ing le ve l assessment, is it necessary to pursue a more refined, data-intensive risk assessment. This is e x hibite d, f or

e x a mple , in g u ida nc e de ve lope d f or

use in CERCLA a nd RCRA on tie r e d approaches. In other cases, reliance on “structure activity relationship” or “bridg ing data” allows the Ag e nc y to re ly on data from similar c he mic a l s rathe r tha n re quire the g e ne ration of ne w, chemical-specific data. W h ile such assessments may or may not be considered influential under the Guide line s , this a dap tation of the SDWA princ i ple s re fle c t s EPA' s re lia nc e on le ss-re fine d ri sk assessm ent s where further refi nem e ntcoul d si g ni f i cant l y i n crease th e costof th e ri sk assessment without sig n ificantly enhancing the assessment or chang i ng the reg u latory outcome. In e m e r g e nc y a nd othe r time c ritic a l c i rc umsta nc e s , risk a sse ssme n ts ma y ha ve to re ly on information at hand or that can be made readily available rather than data such as described in (A). One suc h sc e nario is risk a sse ssme n ts a ddre ssing Eme r g e nc y Ex e mption re que sts submitte d under Section 18 of F IF R A 27 whi c h, because of th e em erg e ncy natu re of th e request , m u stbe c o mple te d within a short time fra me . As a n e x a mple , EPA g r a n te d a n e m e r g e nc y e x e mption under Section 18 to allow use of an unreg istered pesticide to decontaminate anthrax in a Senate office building

. The scientific review and risk assessment to support this action were necessarily constrained by the urg e ncy of the action. Other time-sensitive actions include the reviews of new chemicals under TSCA. Under Section 5 of TSCA 28 , EPA must review a larg e number of pre-manufacture notifications (more than 1,000) every y ear, not all of which necessarily include
" i nfluential" risk assessments, and each review must be completed within a short time frame (g eneral l y 90 day s ). The natu re of th e revi ews and ri sk assessm entassoci at ed wi thth ese pre - ma nufa c ture notific ations ar e a ffe c t e dby the limite d time a va ilable a nd the larg e volume of notific ations submitte d. The fle x ibility provide d by applyi ng “ t o the e x te nt pra c tic ab le ” to para g r aph (A) is appropriate to account for safety risk assessment practices. This flex ibility is already provided for parag r aph (B ) in the SDWA qua lity princ i ple s . We applie d the sa me SDWA a dap ta tion for use with huma n he a lth risk a sse ssme n ts to sa fe ty risk a sse ssme n ts with the ne e de dfle x ibility to apply the pri n ci pl es t o th e ex t e ntpract i cabl e . “S afet y ri sk assessm ent s ” i n cl ude a vari et y of anal y s es, investig ations, or case studies conducted by EPA concerning safety issues. EPA works to ensure that the chemical industry and state and local entities take action to prevent, plan and prepare for, and respond to environmental emerg e ncies and site specific response actions throug h the development and sharing of information, tools and g u idance for haz ard analy s es and risk assessment. F o r ex ample, althoug h the chemical industry shoulders most of the responsibility for safety risk assessment and manag e ment, EPA may also conduct chemical haz ard analy s es, i nvest i g at e th e rootcauses and m echani s m s associ at ed wi th acci dent al chem i cal rel eases, and assess the probability and consequences of accidental releases in support of ag ency risk

27 S ect i o n18 of F I F R A , 7 U

. S

. C

. 136p
28 S ect i o n5 of T S C A , 15 U

. S

. C

. 2604 G u idelines f o rEns u r i ng and Max i m i z i ng Inf o r m ation Q uality 25**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  assessments. Althoug h safety risk assessments can be different from traditional human health ri sk assessm ent s because th ey m a y combi n e a vari et y of avai l abl e i n form at i on and m a y use ex pert judg ement based on that information, these assessments provide useful information that is sufficient for the intended purpose. Ne x t , EPA a dap te d the SDWA qua lity princ i ple s by a dding the c l au se “ i nc luding , whe n available, peer reviewed science and supporting studies” to parag r aph (A)(i). Itnow reads: “the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including , when available, peer reviewed science and supporting studies.” In the Ag e nc y ’s de ve lopme n t of " i nflue n tia l" sc ie ntific risk a sse ssme n ts, we inte nd to use a l l relevant information, including peer reviewed studies, studies that have not been peer reviewed, and incident information; evaluate that information based on sound scientific practices as described in our risk assessment g u idelines and policies; and reach a position based on careful consideration of all such information ( i.e ., a process typically referred to as the “weig h t-ofevi d ence” approach 29 ). In this approach, a well-developed, peer-reviewed study would g e nerally be accorded g r eat er wei g ht th an i n form at i on froma l e ss wel l

- devel oped study th athad notbeen peer-reviewed, but both studies would be considered. Thus the Ag ency uses a “weig h t-ofevi d ence” process when eval uat i ng peer-revi ewed studi es al ong wi th al l other i n form at i on. Oftentimes under various EPA-manag e d prog rams, EPA receives information that has not been peer-reviewed and we have to make decisions based on the information available. W h ile many of the studies incorporated in risk assessments have been peer reviewed, data from other sources, such as studies submitted to the Ag ency for pesticides under F IF R A 30 and for chemicals under TSCA, may not alway s be peer reviewed. Rather, such data, developed under approved g u idelines and the application of Good L aboratory Practices (GLP s), are routinely used in the de ve lopme n t of risk a sse ssme n ts. Risk a sse ssme n ts ma y a l so inc l ude more limite d data se ts suc h as monitoring data used to support the ex posure element of a risk assessment. In cases where the s e data ma y not the m se lve s ha ve be e n pe e r re vie w e dthe i r qua lity a nd appropriate use would be addressed as part of the peer review of the overall risk assessment as called for under the Ag ency ' s peer revi ew g ui d el i n es. L a st l y , EP A adapt e d th e S D W A pri n ci pl es for i n fl uent i a l envi ronm ent a l (“ecol og i cal ”) ri sk assessm ent s th atare di ssem i n at ed i n order t o use t e rm s th atare m o stsui t e d for such ri sk a sse ssme n ts. Spe c i fic a lly , EPA a sse ssme n ts of e c o log i c a l risks a ddre ss a varie t y of e n titie s,

29 T h e w e ig h t

- o f

- e v i d e n ce ap p r o achg e n e r a llyco n s id er s all r e lev a n t in f o r m atio ninanin teg r ativ e assessmen t th at tak e s in to acco u n t th e ki n d s o f ev id en ce av ailab l e, th e q u alityan d q u an tityo f th e ev id en ce, th e s tren g th s an d limitation s as s o ciated of eachty pe of ev iden ce, an d ex plain s h o wth e v ariou s ty pes of ev iden ce f i t tog e th er. See, e.g ., EP A ' s Propos ed G u i del i n es f o r

C

a rc i nogen Ri s k As s e s s m ent (Federal R e g i s t er 61(79): 17960- 18011; Ap ri l 23, 1996) an d EPA ' s G u i del i n es f o r C a rc i nogen Ri s k As s e s s m ent (F ederal R e g i s t er 51(185): 33992- 34003; S e pt em ber 24, 1986), av ai l abl e f r om : www. e p a . g o v/n cea/raf / , an d EP A ' s R i s k C h aracterization H a n dbook ( Sci e nce Pol i cy Counci lH andbook: Ri s k C

har act e r i z a t i o n , EPA100- B- 00- 002, Was h i n g t on , D C : U

. S

. EP A , Decem ber 2000).
30 40 C F Rpart 158 G u idelines f o rEns u r i ng and Max i m i z i ng Inf o r m ation Q uality 26**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  some of which can be described as populations and others (such as ecosy stems) which cannot. Therefore, a specific modification was made to include " a ssessment endpoints, including populations if applicable" in place of the term “population” for ecolog i cal risk assessments and EPA added a footnote directing the reader to various EPA risk policies for further discussion of th ese concept s i n g r eat er det a i l

.

G u idelines f o rEns u r i ng and Max i m i z i ng Inf o r m ation Q uality 27**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  6.5 Does E PA E n su re an d Maxi m i ze th e Q u al i t y of In f o rm ati on f r omE x tern al S ou rces? Ensuring a nd ma x imizing the qua lity of information from Stat e s , othe r g ove rnme nts, a nd third parties is a complex undertaking , involving thoug htful collaboration with States, Tribes, the sc ie ntific a nd te c hnic a l c o mmunity , a nd othe r e x te rna l information provide rs. EPA will c ontinue to take ste p s to e n sure thatthe qua lity a nd tra n spar e nc y of information provide d by e x te rna l sources are sufficient for the intended use. F o r instance, since 1998, the use of environmental data collected by others or for other purposes, including literature, industry survey s, compilations from computeriz ed data bases and information sy stems, and results from computeriz ed or mathematical models of environmental processes and conditions has been within the sc ope of the Ag e nc y ' s Qua lity Sy ste m 31

. F o r information that is either voluntarily submitted to EPA in hopes of influencing a decision or that EPA obtains for use in developing a policy , reg u latory , or other decision, EPA will continue to work with States and other g overnments, the scientific and technical community , and other interested information providers to develop and publish factors that EPA would use to assess the qua lity of this type of information. F o r all proposed collections of information that will be disseminated to the public, EPA intends to demonstrate in our Paperwork Reduction Act 32 clearance submissions that the proposed c o lle c tion of information will re sult in information thatwill be c o lle c t e d, ma inta ine da nd use din wa y s c onsiste nt with the OMB g u ide line s a nd the s e EPA Guide line s . The s e Guide line s apply to al l i n form at i on EP A di ssem i n at es t o th e publ i c ; accordi ng l y , i f EP A l at e r i d ent i f i e s a new use for the information that was collected, such use would not be precluded and the Guidelines would apply to the disse mination of the information to the public

.

31 EP AQu alityMan u a l f o r En v i ron men tal P r og ram s 5360 A 1 . May2000, Section1.3.1. h ttp : / /www. e p a . g o v/ q u a lity /qs

- docs / 5360.pdf
32 44 U

. S

. C

. 3501 ets e q. G u idelines f o rEns u r i ng and Max i m i z i ng Inf o r m ation Q uality 28**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  7 A d m i n i s trati v e Mech an i s mf o r P re-d i ssem i n a ti on Revi ew 7.1 Wh at are th e Ad m i n i strati ve Mech an i s m s f o r P re-d i ssem i n a ti on Revi ew s? Ea c h EPA Prog ra m Offic e a nd Re g i on will inc o rporat e the information qua lity princ i ple s outline din se c tion 6 of the s e Guide line s into the i r e x isting pre - disse mination re vie w proc e dure s as appropriate. Offices and Reg i ons may develop unique and new procedures, as needed, to provide additional assurance that the information disseminated by or on behalf of their org a nizations is c onsiste nt with the s e Guide line s . EPA inte nds to fa c ilitate imple me ntation of consistent cross-Ag ency pre-dissemination reviews by establishing a model of minimum review sta ndards ba se d on e x isting polic ie s. Suc h a mode l for pre

- disse mination re vie w would still provide that responsibility for the reviews remains in the appropriate EPA Office or Reg i on. F o r the purposes of the Guidelines, EPA recog n iz es that pre-dissemination review procedures ma y inc l ude pe e r re vie w s a nd qua lity re vie w s thatma y oc c u r atma ny ste p s in de ve lopme n t of information, not only at the point immediately prior to the dissemination of the information.

Adm i nis t r a tive Mec hanis m f o rPr e- dis s e m i nation Review 29**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  8 A d m i n i s trati v e Mech an i s m s f o r Correcti on of In f o rm ati on 8.1 Wh at are E PA' s Ad m i n i strati ve Mech an i s m s f o r Af f

ected P erson s to S eek an d O b tai n Correcti on of In f o rm ati on ? EPA’s Offic e of Environme n ta l In formation (OEI ) ma na g e s the a d ministrative me c ha n isms that enable affected persons to seek and obtain, where appropriate, correction of information disse minat e dby the Ag e nc y thatdoe s not c o mply with EPA or OMB In formation Qua lity Guidelines. W o rking with the Prog ram Offices, Reg i ons, laboratories, and field offices, OEI will recei ve compl a i nt s (or copi es) and di stri but e th emt o th e appropri at e EP A i n form at i on owners. “Information owners” are the responsible persons desig n ated by manag e ment in the applicable EPA Prog ram Office, or those who have responsibility for the quality , objectivity , utility , and integ rity of the information product or data disseminated by EPA. I f a person believes that information disse minat e dby EPA ma y not c o mply with the Guide line s , we e nc oura g e the pe rson to consult informally with the contact person listed in the information product before submitting a request for correction of information. An informal contact can result in a quick and efficient resolution of questions about information quality

. 8.2 Wh at S h ou l d b e In cl u d e d i n a Req u e st f o r Correcti on of In f o rm ati on ? Persons requesting a correction of information should include the following information in their R e quest for C o rrection (R FC ):

• N a m e a nd c onta c tinformation for the individua l or org a nization submitting a c o mpla int; ide n tific ation of a n individua l to se rve a s a c onta c t
.
• A description of the information the person believes does not comply with EPA or OMB g u ide line s , inc l uding spe c i fic c itations to the information a nd to the EPA or OMB g u ide line s , if applic ab le

.
• A n ex planation of how the information does not comply with EPA or OMB g u idelines and a recommendation of corrective action. EPA considers that the complainant has the burden of demonstrating that the information does not c o mply with EPA or OMB g u ide line s a nd thata partic ular c o rre c tive a c tion would be appropriate.

• A n ex planation of how the alleg e d error affects or how a correction would benefit the requestor.

• A n affected person may submit an RF C via any one of methods listed here: • I n t ern et athttp:/ / www.epa.g ov/oei/quality g u idelines • E- m a il at qua lity .g uide line s @e pa .g ov • Fa x at (202) 566-0255

Adm i nis t r a tive Mec hanis m s f o rCor r ec tion of Inf o r m ation 30**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA** 
• Mai l to Information Quality Guide line s Staff , Mail Code 28221T, U.S. EPA, 1200 Pennsy l vania Ave., N.W ., W a shing t on, DC, 20460 • B y cou r i er or i n p erson to Information Quality Guide line s Staff , OEI Docket Center, Room B 128, EPA West B u ilding , 1301 Constitution Ave., N.W ., W a shing t on, DC

8.3 Wh en Does E PA In ten d to Con s i d er a Req u e st f o r Correcti on of In f o rm ati on ? EPA seeks public and stakeholder input on a wide variety of issues, including the identification and resolution of discrepancies in EPA data and information. EPA may decline to review an RF C under these Guidelines and consider it for correction if:

• T he request does not address information disseminated to the public covered by these Guidelines (see section 5.3 or OMB ’ s g u idelines). In many cases, EPA provides other correction processes for information not covered by these Guide line s .
• T he request omits one or more of the elements recommended in section 8.2 and there is insufficient information for EPA to provide a satisfactory response.

• T he request itself is “frivolous,” including those made in bad faith, made without justification or trivial, and for which a response would be duplicative. More information on this subject may be found in the OMB g u idelines. 8.4 H ow Does E PA In ten d to Resp on d to a Req u e st f o r Correcti on of In f o rm ati on ? EPA intends to use the following process:

• E a c h RF C will be tra c ke din a n OEI sy ste m .
• I f an RF C is deemed appropriate for consideration, the information owner office or reg i on makes a decision on the request on the basis of the information in question, including a request submitted under section 8.2. Rejections of a request for correction should be decided at the hig h est level of the information owner office or reg i on. EP A’s g o al is to respond to requests within 90 day s of receipt, by 1) providing either a decision on the request, or 2) if the request requires more tha n 90 c a l e ndar da y s to re solve , informing the c o mpla ina n t thatmore time is re quire d a nd indic at e the re a s on why a nd a n e s timate d de c i sion date
.
• I f a request is approved, EPA determines what corrective action is appropriate. Considerations relevant to the determination of appropriate corrective action include the nature and timeliness of the information involved and such factors as the sig n ificance of the error on the use of the information and the mag n itude of

Adm i nis t r a tive Mec hanis m s f o rCor r ec tion of Inf o r m ation 31**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  the error. F o r requests involving information from outside sources, considerations ma y inc l ude c oordinating with the sourc e a nd othe r pra c tic a l limitations on EPA’s ab ility to take c o rre c tive a c tion.

• W hether or not EPA determines that corrective action is appropriate, EPA provides notice of its decision to the requester.

• F or approved requests, EPA assig n s a steward for the correction who marks the information as desig n ated for corrections as appropriate, establishes a schedule for correction, and reports correction resolution to both the tracking sy stem and to the requestor. OEI will provide reports on behalf of EPA to OMB on an annual basis beg i nning J a nuary 1, 2004 reg arding the number, nature, and resolution of complaints received by EP A. 8.5 H ow Does E PA E xp ect to P r ocess Req u e sts f o r Correcti on of In f o rm ati on on Wh i c h EPA has SoughtP ublic C om m e nt ? When EPA provides opportunities for public participation by seeking comments on information, the public comment process should address concerns about EPA’s information. F o r ex ample, when EPA issues a notice of proposed rulemaking supported by studies and other information described in the proposal or included in the rulemaking docket, it disseminates this information within the me a n ing of the Guide line s . The public ma y the n ra ise issue s in c o mme nts re g arding the information. I f a g r oup or an individual raises a question reg arding information supporting a proposed rule, EPA g e nerally ex pects to treat it procedurally like a comment to the rulemaking , addressing it in the response to comments rather than throug h a separate response mechanism. This approach would also g e nerally apply to other processes involving a structured opportunity for public comment on a draft or proposed document before a final document is issued, such as a draft report, risk assessment, or g u idance document. EPA believes that the thoroug h consideration provided by the public comment process serves the purposes of the Guidelines, provides an opportunity for correction of any information that does not comply with the Guidelines, and does not duplicate or interfere with the orderly conduct of the action. In cases where th e Ag ency di ssem i n at es a study , anal y s i s , or other i n form at i on pri o r t o th e fi nal Ag ency action or information product, it is EPA policy to consider requests for correction prior to the final Ag ency action or information product in those cases where the Ag ency has determined that an earlier response would not unduly delay issuance of the Ag ency action or information product and the complainant has shown a reasonable likelihood of suffering actual harm from the Ag e nc y ' s disse mination if the Ag e nc y doe s not re solve the c o mpla int prior to the fina l Ag e nc y action or information product. EPA does not ex pect this to be the norm in rulemaking s that it conducts, and thus will usually address information quality issues in connection with the final Ag ency action or information product.

Adm i nis t r a tive Mec hanis m s f o rCor r ec tion of Inf o r m ation 32**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  EPA g e ne ra lly would not c onside r a

c o mpla int thatc ould ha ve be e n submitte d a s a time ly comment in the rulemaking or other action but was submitted after the comment period. I f EPA cannot respond to a complaint in the response to comments for the action (for ex ample, because the c o mpla int is submitte d too late to be c onside r e da nd c ould not ha ve be e n time ly submitte d, or because the complaint is not g e rmane to the action), EPA will consider whether a separate response to the complaint is appropriate.

Adm i nis t r a tive Mec hanis m s f o rCor r ec tion of Inf o r m ation 33**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  8.6 What Should be Inc l ude d in a R e que st A s king EPA t o R e c o nside rit s D e c i sion on a Req u e st f o r th e Correcti on of In f o rm ati on ? I f

requesters are dissatisfied with an EPA decision, they may file a Request for Reconsideration (RF R ). The RF R should contain the following information:

• A n indication that the person is seeking an appeal of an EPA decision on a previously submitted request for a correction of information, including the date of the orig inal submission and date of EPA decision. A copy of EPA’s orig inal decision would help ex pedite the process.

• N ame and contact information. Org a nizations submitting an RF R should identify an individual as a contact.

• A n ex planation of why the person disag r ees with the EPA decision and a specific recom m e ndat i on for correct i v e act i on.

• A copy of the orig inal RF C of information.

• A n a ffe c t e dpe rson ma y submit a Re que st for Re c onside r ation (RF R ) via a ny one of the methods listed here: • I n t ern et athttp:/ / www.epa.g ov/oei/quality g u idelines • E- m a il at qua lity .g uide line s @e pa .g ov • Fa x at (202) 566-0255 • Mai l to Information Quality Guide line s Staff , Mail Code 28221T, U.S. EPA, 1200 Pennsy l vania Ave., N.W ., W a shing t on, DC, 20460 • B y cou r i er or i n p erson to Information Quality Guide line s Staff , OEI Docket Center, Room B 128, EPA West B u ilding , 1301 Constitution Ave., N.W ., W a shing t on, DC EPA recommends that requesters submit their RF R within 90 day s of the EPA decision. I f the RF R is se nt a f te r thattime , EPA re c o mme nds tha tthe re que ste r inc l ude a n e x pla nation of why the request should be considered at this time. 8.7 H ow Does E PA In ten d to P r ocess Req u e sts f o r Recon s i d erati on of E PA Deci si on s? EPA intends to consider RF R using the following process:

• E a c h RF R will be tra c ke din a n OEI sy ste m .
• O EI sends the RF R to the appropriate EPA Prog ram Office or Reg i on that has responsibility for the information in question.

Adm i nis t r a tive Mec hanis m s f o rCor r ec tion of Inf o r m ation 34**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA** 
• T he Assista n t Administrat or (AA) or Re g i ona l Administrat or (RA) information owner presents to an ex ecutive panel. The ex ecutive panel would be comprised of th e S c i e nce Advi sor/AA for th e Offi ce of R e search and Devel opm ent(OR D ), C h ief In formation Officer/AA for OEI , and the Economics Advisor/AA for the Office of Policy , Economics and Innovation (OPEI

. ). The 3-member ex ecutive panel would be chaired by the Chief In formation Officer/AA for OEI

. W h en the subject of the RF R orig inated from a member office, that panel member would be repl aced by an al t e rnat e AA or R A . W hi l e th e ex ecut i v e panel i s consi d eri ng an RF R, the de c i sion ma de on the initia l c o mpla int by the information owne r offic e or reg i on remains in effect.

• T he ex ecutive panel makes the final decision on the RF R.

• E P A ’s g o al is to respond to each RFR within 90 day s of receipt, by 1) providing either a decision on the request or 2) if the request requires more than 90 calendar da y s to re solve , informing the c o mpla ina n t thatmore time is re quire d a nd indic at e the re a s on why a nd a n e s timate d de c i sion da te
.
• I f a request is approved, EPA determines what type of corrective action is appropriate. Considerations relevant to the determination of appropriate corrective action include the nature and timeliness of the information involved and such fact ors as th e si g ni f i cance of th e error on th e use of th e i n form at i on and the mag n itude of the error. F o r requests involving information from outside sources, considerations may include coordinating with the source, and other pra c tic a l limita tions on EPA' s ab ility to take c o rre c tive a c tion.

• W hether or not EPA determines that corrective action is appropriate, EPA provides notice of its decision to the requester.

• F or approved requests, EPA assig n s a steward for the correction who marks the information as desig n ated for corrections as appropriate, establishes a schedule for correction, and reports correction resolution to both the tracking sy stem and to the requestor.

Adm i nis t r a tive Mec hanis m s f o rCor r ec tion of Inf o r m ation 35**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  A ppe ndix A IQG Developm ent P r ocess and Discussion of P ublic Com m e nts A .1 I nt r o duc t i on EPA' s Guidelines are a living document and may be revised as we learn more about how best to a ddre ss, e n sure , a nd ma x imize information qua lity . In the proc e ss of de ve loping the s e Guidelines, we actively solicited public input at many stag es. W h ile the public was free to comment on any aspect of the Guidelines, EPA ex plicitly requested input on key topics such as influential information, reproducibility , influential risk assessment, information sources, and error correction. Public input was soug ht in the following way s :

• A n online Public Comment Session was held March 19-22, 2002, as the first draft of the Guidelines was being developed. EP A received approx imately 100 comments.

• A Public Meeting was held on May 15, 2002, after the draft Guidelines were issued. There were 99 participants, 13 of whom made presentations or commented on one or more issue s .
• A 52 day Public Comment period lasted from May 1 to J une 21, 2002, where comments coul d be m a i l e d, fax e d, or e-m a i l e d t o EP A. EP A recei ved 55 com m e nt s duri ng thi s period.

• A meeting with State representatives, sponsored and supported by the Environmental Council of the States (ECOS), was held on May 29, 2002.

• A conference call between EPA and Tribal representatives was held on J une 27, 2002. More detailed information on the public comments is available throug h an OEI web site, serving a s the home pa g e for the EPA In formation Qua lity Guide line s throug h the de ve lopme n t a nd imple me ntation proc e ss. Ple a s e visit this site at http://www.epa.g ov/oei/quality g u idelines

. We have established a public docket for the EPA In formation Quality Guidelines under Docket ID No. OEI

- 10014. The docket is the collection of materials available for public viewing In formation Quality Guidelines Staff, OEI Docket Center, Room B 128, EPA West B u ilding , 1301 Constitution Ave., N.W., Washing t on, DC, phone number 202-566-0284. This docket consi s t s of a copy of th e Gui d el i n es, publ i c com m e nt s recei ved, and other i n form at i on rel at e d t o the Guidelines. The docket is open from 12:00 PM to 4:00 PM, Monday throug h Friday , e x c l uding le g a l holida y s. An inde x of doc ke t c onte n ts will be a va ilable at http://www.epa.g ov/oei/quality g u idelines

.

Appendix 36**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  A.2 G en eral S u m m a ry of Com m e n t s During the various public comment opportunities, EPA received input from a diverse set of org a nizations a nd privat e c itize n s. Comme nts c a m e from ma ny of EPA' s stake holde rs - the reg u lated community and many interest g r oups who we hear from frequently during the manag e ment of EPA' s Prog rams to protect the nation' s land, air, water, and public health. Governm e ntag enci es at th e Federal , S t at e, Tri b al , and l o cal l e vel al so com m e nt ed on the Gui d el i n es. OMB sentcom m e nt s t o every Federal ag ency and EP A recei ved com m e nt s fromt w o members of Cong ress. B e y ond our g overnment colleag ues, the private sector voiced many concerns and helpful recommendations for these Guidelines. W e would like to take this opportunity to thank all commenters for providing their input on these Guidelines. Due to the tig ht time frame for this project, this discussion of public comments g e nerally describes the major categ ories of comments and hig h lig hts some sig n ificant comments, but does not contain an individual response to each public comment. Comments received by EP A during the public comment period reflect a diversity of views reg arding EPA' s approach to developing draft Guidelines as well as the g e neral concept of information qua lity . Some c o mme nte r s inc l ude d de ta ile d re vie w of a ll Guide line s se c tions, while others chose to address only specific topics. In some cases, commenters provided ex amples to de monstrat e how c u rre nt EPA proc e dure s ma y not e n sure a de quate information qua lity for a specific application. Commenters provided g e neral observations such as stating that these Guide line s did not suffic i e n tly a ddre ss EPA' s informa tion qua lity proble m s. Some c o mme nte r s offe re d thatthe Guide line s re lie d too muc h on e x isting polic ie s. In te rpre tations of the inte nt of the Data Qua lity Ac t we re offe re d by some c o mme nte r s. One c o mme nt note dthatimprove me nt of data quality is not necessarily an end in and of itself. Another comment was that the g o al of Guidelines should be more to improve quality , not end uncertainty . Public interest and environmental g r oups voiced concern over what they believed was an attempt by various g r oups to undermine EPA' s ability to act in a timely fashion to protect the environment and public health. Some commenters stated that the directives of the Data Quality Act and OMB cannot override EPA' s mission to protect human health and the environment per the statutory mandates under which it operates. EPA was cong ratulated for the effort and, in some cases, encourag ed to g o even further in a ddre ssing information qua lity . Some c o mme nte r s e nc oura g e dEPA to provide a dditiona l process details, provide more detailed definitions, aug m ent ex isting policies that promote transparency , and share more information about the limitations of EPA disseminated information. In one case, EPA was encourag ed to develop a rating scheme for its disseminated information. This section discusses public comments and our responses to many of the important questions and issues raised in the comments. F i rst, we provide responses to some overarching comments we received from many commenters, then we provide a discussion of public comments that were recei ved on speci fi c t opi cs addressed i n th e draftGui d el i n es.

Appendix 37**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA** 
• Tone : Comme nte r s c ritic ize dthe " de f e n sive tone " , " l e g a listic tone " , a nd the la c k of de ta il a fforde din the Guide line s . Some c o mme nte r s sa id thatit wa s not c l e ar what the Guidelines were ex plaining , or how they mig ht apply to various types of information. We understand and ag ree with many of these criticisms and have ma de atte mpts to be tte r c o mmunic at e the purpose , applic ab ility , a nd c onte n t of the s e Guide line s .
• P l an f o r i m p l em en tati on : Commenters sug g e sted that the Guidelines should de sc ribe EPA' s pla n s for imple me nting the Guide line s . The s e Guide line s provide policy g u idance, and as such, do not outline EPA' s plan for implementation. That i s , th ey do notdescri be i n g r eatdet a i l how each P r og ramand R e g i onal Offi ce wi l l i mpl e m e nt th ese pri n ci pl es. W e do not i nt e nd t o i mpl y th ateach Offi ce wi l l i mpl e m e nt th emi n confl i c twi th one another, butrath er assum e th atbecause each Prog ra m imple me nts a diffe re nt statutory ma ndate or ma ndate s, the r e will be some inherent differences in approach. B e y ond internal implementation, we ag ree that there is more work and communication to be conducted with information provide rs a nd use r s to optimize the provisions se t forth in the s e Guide line s .
• Com m i tm en t to p u b lic access: One com m e nt er sug g e st ed th atwe "rem ove outdated information" from our web site. Other commenters sug g e sted that when a complaint has been filed that the information should be removed from public vi ew whi l e a compl a i nt i s bei ng revi ewed. Thi s i s g e neral l y unaccept abl e t o EP A in lig ht of our commitment to providing the public with access to information; howe ve r , in c e rta in c a s e s EPA ma y c onside r imme diat e re mova l of information (for ex ampl e, when i t i s cl ear t o us th at th e i n form at i on i s g r ossl y i n correctand misleading and its status cannot be adequately clarified throug h a notice or other ex planation). With respect to outdated information, sometimes it serves a historical purpose, and should continue to be disseminated for that purpose. A.3 R esp on s e to Com m e n t s b y Gu i d el i n es T op i c Area A.3.1 Existing P o licy Ma ny c o mme nte r s told us thatwe re ly e x c e ssive ly on e x isting EPA information qua lity polic ie s. Commenters provided specific ex amples of areas they believed were demonstrative of our lack of c o mmitme nt to or une ve n imple me ntation of our e x isting polic ie s. Some c o mme nte r s a l so pointed out that there are key areas in which we lack policies to address quality and, as a result, the Guidelines should address such issues in more detail. Some commenters also noted that EPA itself has hig h lig hted lessons learned with ex isting approaches to information product development. Ong o ing improve me nt in imple me nting e x isting proc e sse s is a ke y princ i ple of qua lity manag e ment. W e view these Guidelines as an opportunity to enhance ex isting policies and redouble our commitment to quality information.
Appendix 38**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  The concept of peer review is considered in three Guidelines sections. (1) Application of the Ag ency ' s Peer Review Policy lang uag e for " m ajor scientific and technical work products and economic analy s is used in decision making " as a class of information that can be considered
" i nfluential" for purposes of the Guidelines; ( 2) Use of " p eer-reviewed science" as a component of some risk assessments; and (3) Use of the Ag ency ' s Peer Review Policy as one of the Ag e nc y -wide proc e sse s to e n sure the qua lity , obje c tivity , a nd tra n spar e nc y of " i nflue n tia l" scientific, financial, and statistical information under the Guidelines. Some commenters ex pressed concerns reg arding application of peer review in EPA. Commenters sug g e st that current peer reviews are not sufficiently standardiz ed, independent, or c onsiste ntly imple me nte d. Pe e r re vie w is a c o rne r stone to EPA’s c r e d ibility a nd we must e n sure that the process alway s works as desig n ed. F o r this reason, we conduct routine assessments to evaluate and improve the peer review process. Commenters also questioned whether peer review is an adequate means to establish "obj ect i vi t y ." W e not e th atOMB g ui d el i n es speci fi cal l y al l o w for th e use of form al , ex t e rnal , independent peer review to establish a presumption of objectivity . OMB g u idelines also state that the presumption of objectivity is rebuttable, althoug h the burden of proof lies with the complainant. Some commenters asked for additional definitions for peer review terms. Our current peer review policy is articulated in Peer Review and Peer I nvolvement at the U.S. Environmental Protection Ag ency . 33 Additional discussion reg arding the application of peer-reviewed science is provided in the discussion of comments on risk assessment. A.3.2 Scope and Applicability W e received a number of comments on section 1.1 (W hat is the P u rpose of these Guidelines? ) of the draft Guidelines. Some commenters arg u ed that the Guidelines should be binding on EPA, thatthe y ar e le g i slative rule s rathe r tha n g u ida nc e , or tha tthe Guide line s must be followe d unless we make a specific determination to the contrary . Others arg u ed that the Guidelines should not be binding or that we should include an ex plicit statement that the Guidelines do not a lte r substa ntive a g e nc y ma ndate s. Some sug g e ste dthatour stat e m e n ts re ta ining disc re tion to differ from the Guidelines sent a sig n al that EPA was not serious about information quality

. W ith respect to the nature of these Guidelines, Section 515 specifies that ag encies are to issue “g uidelines.” As directed by OMB ’ s g u idelines, we have issued our own g u idelines containing nonbinding policy and procedural g u idance. W e see no indication in either the lang uag e or g e neral structure of Section 515 that Cong ress intended EPA’s g u idelines to be binding rules. W e revised this section (now section 1 in this revised draft) by adding a fuller ex planation of how EPA inte nds to e n sure the qua lity of information it disse minat e s . This se c tion inc l ude s lang uag e ex plaining the nature of our Guidelines as policy and procedural g u idance. This lang uag e is intended to g i ve clear notice of the nonbinding leg a l effect of the Guidelines. It

33 h ttp ://e p a .g o v/o sp /s p c /p er ev me m . h t m Appendix 39**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  notifie s EPA sta ff a nd the public thatthe doc ume n t is g u ida nc e rathe r tha n a substa ntive rule a nd ex plains how such g u idance should be implemented. Althoug h we believe these Guidelines woul d notbe j udi ci al l y revi ewabl e , we ag ree th ata st at em ent t o thi s effect i s unnecessary and have deleted it. In response to comments that EPA clarify that the Guidelines do not alter ex isting leg a l requirements, we have made that chang e . In lig ht of that chang e , we think it is c l e ar thatde c i sions in partic ular c a s e s will be ma de ba se d on applic ab le statute s , re g u lations, a nd requirements, and have deleted other tex tin the parag r aph that essentially repeated that point. Elsewhere in the document, EPA has made revisions to be consistent with its status as g u idance. Some commenters arg u ed that all EPA disseminated information should be covered by the Guide line s a nd thatwe la c k au thority to “ e x e mpt” information from the Guide line s . Othe rs thoug ht that the coverag e in EPA’s draft was appropriate. EPA does not view its Guidelines as establishing a fix e d definition and then providing “ex emptions.” Rather, our Guidelines ex plain when a distribution of information g e nerally would or would not be considered disseminated to the public for purposes of the Guidelines. As we respond to complaints and g a in ex perience in imple me nting the s e Guide line s , we ma y ide n tify othe r insta nc e s whe r e information is or is not consi d ered di ssem i n at ed for th e purposes of th e Gui d el i n es. Some commenters cited the Paperwork Reduction Act (PRA), 44 U.S.C. 3501 et seq., to support their arg u ment that the Guidelines should cover all information EPA makes public. EPA’s Guidelines are issued under Section 515 of the Treasury and General Government Appropriations Act for F i scal Year 2001, which directs OMB to issue g overnment-wide g u idelines providing policy and procedural g u idance to F e deral ag encies. In turn, the OMB g u idelines provide direction and g u idance to F e deral ag encies in issuing their own g u idelines. EPA’s Guide line s ar e inte nde d to c arry out OMB ’ s polic y on information qua lity

. One c o mme nte r c ite d in partic ular the te rm “ public information” use din the PRA a s e v ide nc e of Cong ress’s intent under Section 515. In EPA’s view, this does not show that Cong ress intended a specific definition for the key terms, “information” and “disseminated,” used in Section 515. I n th e absence of evi d ence of C ong ressi onal i nt e ntreg ardi ng th e m eani ng of th e t e rm s used i n Section 515, EPA does not believe the PRA requires a chang e in EPA’s Guidelines. W e ag ree with commenters who noted that even if a particular distribution of information is not c ove re d by the Guide line s , the Guide line s would still apply to information disse minat e din othe r way s . As stated in section 1.4, if information is not initially covered by the Guidelines, a subsequent distribution of that information will be subject to the Guidelines if EPA adopts, endorses, or uses it. Some commenters made specific recommendations about what should and should not be covered by the Guidelines. In addition to the specific recommendations, some sug g e sted that the “scope and applicability ” section was too long , while others thoug ht it had an appropriate level of detail. B a sed on other ag encies’ g u idelines and public comments, EPA has removed much of the detail from the discussion of Guidelines coverag e . These revisions were intended to shorten and simplify the discussion without chang i ng the g e neral scope of the Guidelines.

Appendix 40**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  W e revised our definition of “information” in section 5.3, in response to a comment requesting that the Ag ency make clear that information from outside sources is covered by the Guidelines if EPA adopts, endorses, or uses it to support an Ag ency decision or position. In section 5.4, we modifie dse ve ra l of the provisions. We a dde d stat e m e n ts of “ i nte n t” or similar la ng ua g e to de fine the scope of several of the provisions. According l y , dissemination would not include distribution of information “intended” for g overnment employ ees or recipients of contracts, g r ants, or cooperative ag reements. Nor would information in correspondence “directed to” individuals or persons be covered. Thi s recog ni z es th at th ere m a y be i n st ances where EP A m a y use a l e t t e r written to an individual in a way that indicates it is directed bey ond the correspondent and represents a more g e nerally applicable Ag ency policy

. The Guidelines would apply in such a case. EPA has created a categ ory for information of an “ephemeral” nature, including press releases, speeches, and the like. The intent was that the Guidelines should not cover communications that merely serve as announcements, or for other reasons are intended to be fleeting or of limited duration. Consistent with other ag ency g u idelines, we have added lang uag e indicating that the Guidelines do not cover information presented to Cong ress, unless EPA simulta ne ously disse minat e s this information to the public

. Some commenters thoug ht all information from outside sources should be covered by the Guidelines, even if EPA does not use, rely on, or endorse it. Others wished to clarify the point at which the Guidelines cover information from outside sources. As noted above, section 1.4 of the Guide line s e x pla i ns how subse que nt distributions of informa tion in public filing s ma y be c o me subje c tto the Guide line s . We c ontinue to think thatEPA’s own public filing s be fore othe r ag encies should not g e nerally be covered by the Guidelines as long as EPA does not simultaneously disseminate them to the public, since use of this information would be subject to the re quire me nts a nd polic ie s of the a g e nc y to whic h the information is submitte d. W e recei ved a number of com m e nt s, i n cl udi ng fromOMB, arg ui ng th at th e provi si on reg ardi ng information related to adjudicative processes was too broad, and that the Guidelines should c ove r some or a ll information re la te d to a d judic ative proc e sse s, partic ular ly a d ministra tive adjudications. In addition to shortening this section, we have limited this provision to information in documents prepared specifically for an administrative adjudication. This would include decisions, orders, finding s, and other documents prepared specifically for the adjudication. As indicated in the Draft Guidelines, our view is that ex isting standards and prote c tions in a d ministra tive a d judic ations would g e ne ra lly be a de quate to a ssure the qua lity of information in administrative adjudications and to provide an adequate opportunity to contest decisions on the quality of information. For ex ample, in permitting proceeding s , parties may submit comments on the quality of information EPA prepares for the permit proceeding , and judicial review is available based on ex isting statutes and reg u lations. Narrowing the provision to information prepared specifically for the adjudication should make clear that the Guidelines would not g e nerally provide parties with additional avenues of challeng e or appeal during adjudications, but would still apply to a separate distribution of information where EPA adopts, endorses, or uses the information, such as when EPA disseminates it, on the In ternet, or in a rulemaking , or g u idance document. W h en we intend to adopt information such as models or risk assessments for use in a class of cases or determinations (e.g ., for use in all determinations under
Appendix 41**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  a particular reg u latory provision), EPA often disseminates this information separately and in many instances requests public comment on it. According l y , it is not clear that there would be many instances where persons who are concerned about information prepared specifically for an adjudication would not have an opportunity to contest the quality of information. We re spe c t fully disa g r e e with a c o mme nte r’s re c o mme ndation thatre g u la tory limits e s tablishe d by EPA should be subject to the Guidelines. The Guidelines apply to information disseminated by EPA, not to reg u latory standards or other Ag ency decisions or policy choices. In response to comments reg arding information disseminated in rulemaking s and other matters subject to public c o mme nt, EPA c onside r s thatthis information would be disse minat e dwithin the me a n ing of the Guidelines, althoug h we would g e nerally treat complaints reg arding that information procedurally like other comments on the rulemaking or other matter. A.3.3 S ou rces of In f o rm ation W e received many comments on how the Guidelines apply to ex ternal parties, the shared quality responsibilities between EPA and ex ternal parties, and specific EPA responsibilities when using or relying on information collected or compiled by ex ternal parties. EPA r o le s: Some c o mme nte r s e mpha size d thate n suring qua lity of information atthe point of dissemination is no substitute for vig o rous efforts by EPA to receive quality information in the first place and therefore for information providers to produce quality information. One commenter stated that EPA cannot be responsible for all aspects of the quality of the information we disseminate. In response to this and other comments, we have provided additional lang uag e in the s e Guide line s on the various role s tha tEPA a ssume s in e ithe r e n suring the qua lity of the information we disseminate or ensuring the integ rity of information EPA distributes. One c o mme nt sug g e ste dthatwe me ntion the role of the Na tiona l Environme n ta l In formation Ex chang e Network in ensuring information integ rity , which we have done in section 2.4 of the Guide line s . Assessm ent f a ctors: Overall, public input was positive and welcoming of our proposal to de ve lop a sse ssme n t fa c t ors to e va l uate the qua lity of information g e ne rate d by third partie s. A few commenters offered their involvement in the development of these factors, their advice on how to develop such factors, and some ex amples of what assessment factors we should consider. EPA staff have provided such comments to the EPA Science Policy Council workg r oup that was charg e d with developing the assessment factors. EPA welcomes stakeholder input in the development of these factors and published draft assessment factors for public comment in September 2002. Coverage of State Inf o rm ation: Some c o mme nte r s sug g e ste dthatour Guide line s must apply to a ll information disse minat e dby EPA, inc l uding information submitte d to us by Stat e s . Whe r e a s some commenters stressed that the quality of information received by EPA is the responsibility of the providers, others ex pressed concern about the potential impact that EPA’s Guidelines could have on States. W e believe it is important to differentiate between information that we
Appendix 42**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  g e nerate and data or information g e nerated by ex ternal parties, including States. State information, when submitted to EPA, may not be covered by these Guidelines, but our subsequent use of the information may in fact be covered. W e note, however, that there may be pra c tic a l limita tions on the ty pe of c o rre c tive a c tion thatma y be take n, sinc e EPA doe s not inte nd to a lte r information submitte d by Stat e s . Howe ve r, EPA doe s inte nd to work c l ose l y with our State counterparts to ensure and max imize the quality of information that EPA disseminates. F u rthe rmore , one c o mme nte r stat e dthatif re g u latory information is submitte d to a n au thorize d or deleg ated State prog ram, then the State is the primary custodian of the information and the Guidelines would not cover that information. W e ag ree with that statement. W e also received comments reg arding the use of labels, or disclaimers, to notify the public whether information is g e nerated by EPA or an ex ternal party

. W e ag ree that disclaimers and other notifications should be used to ex plain the status of information wherever possible, and we are developing appropriate lang uag e and format. A statement reg arding Paperwork Reduction Act clearance submissions has been added in response to comment by OMB

. A.3.4 Inf l uential Inf o rm ation EPA received a rang e of comments on its definition of “influential.” B e low we provide a summary of the comments raised and EPA’s response. Several commenters g e nerally assert that the definition is too narrow. Other commenters indicated that under EPA' s draft definition, only Economically Sig n ificant actions, as defined in Ex ecutive Order 12866, or only Economically Sig n ificant actions and information disseminated in support of top Ag ency actions, are considered " i nfluential." W e disag r ee. To demonstrate the broad rang e of activities covered by our adoption of OMB ’ s definition, we reiterate the definition below and include an ex ample of each type of action, to illustrate the breadth of our definition. “Influential,” when used in the phrase “influential scientific, financial, or statistical information,” means that the Ag ency can reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or importa nt privat e se c t or de c i sions. We will g e ne ra lly c onside r the following c l a sse s of information to be influential: information disseminated in support of top Ag ency actions; information disseminated in support of “economically sig n ificant” actions; major work products underg oing peer review; and other disseminated information that will have or does have a clear and substantial impact (i.e., potential chang e or impact) on important public policies or important private sector decisions as determined by EPA on a case-by -case basis. In g e neral, influential information would be the scientific, financial or statistical information that provides a substantial basis for EPA’s position on key issues in top Ag ency actions and Economically Sig n ificant actions. I f the information provides a substantial basis for EPA’s position, EPA believes it would g e nerally have a clear and substantial impact.

Appendix 43**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  Top Agency actions: An ex ample of a top Ag ency action is the review of the National Ambient Air Quality Standards (NAAQS) for Particulate Matter. Under the Clean Air Act, EPA is to periodically review (1) the latest scientific knowledg e about the effects on public health and public welfare (e.g ., the environment) associated with the presence of such pollutants in the ambient air and (2) the standards, which are based on this science. The Ac t furthe r dire c t s thatthe Administrat or sha ll make a ny re visions to the sta ndards as may be appropriate, based on the latest science, that in her judg ment are requisite to protect the public health with an adequate marg in of safety and to protect the public welfare from any known or anticipated adverse effects. The standards establish allowable l e vel s of th e pol l ut a nt i n th e ambi entai r across th e Uni t e d S t at es, and S t at es m u st development implementation plans to attain the standards. The P M NAAQS were last revised in 1997, and the nex tperiodic review is now being conducted. “Ec o nom i c a lly signific ant” r u le s: An ex ample of a rule found to be economically sig nificant is the Disposal of Poly chlorinated B i pheny ls (PCB s) F i nal Rule. In 1998, EPA amended its rules under the Tox i c Substances Control Act (TSCA), which addresses the manufacture, processing , distribution in commerce, use, cleanup, storag e and disposal of PCB s . This rule provides flex ibility in selecting disposal technolog ies for PCB wastes and ex pands the list of available decontamination procedures; provides less burdensome me c ha n isms for obta i ning EPA approva l for a varie t y of a c tivitie s; c l arifie s a nd/or modifies certain provisions where implementation questions have arisen; modifies the requirements reg arding the use and disposal of PCB equipment; and addresses outstanding issues associated with the notification and manifesting of PCB wastes and c ha ng e s in the ope ration of c o mme rc ia l stora g e fa c ilitie s. EPA would c onside r the informa tion tha tprovide s the princ i pa l ba sis for this rule to be influe ntia l information. P eer revi e w e d w o rk p r od u c ts: An ex ample of a major work product underg oing peer review is the I R I S Documentation: Reference Dose for Methy lmercury

. Methy lmercury contamination is the basis for fish advisories. Itis necessary to determine an intake to humans that is without appreciable risk in order to devise strateg i es for decreasing mercury emissions into the environment. After EPA derived a reference dose (RfD) of 0.0001 mg /kg

- day in 1995, industry arg u ed that it was not based on sound science. Cong ress ordered EPA to fund an National Research Council/National Academy of the Sc ie nc e s pa ne l to de te rmine whe the r our RfD wa s sc ie ntific a lly justifiable

. The pa ne l concluded that the 0.0001 mg /kg

- day was an appropriate RfD, based on newer studies than the 1995 RfD. The information in this document was evaluated, incorporated, and subjected to comment by the Office of W ater, where it contributed in larg e part to Chapter 4 of Drinking W ater Criteria for the Protection of Human Health: Methylmercury (EPA/823/R-01/001) J a nuary 2001. The peer review mechanism was an ex ternal peer review workshop and public comment session held on November 15, 2000, accompanied by a public comment period from October 30 to November 29, 2000. Case-by-base determ ination – P BT Chem icals Rule: An ex ampl e of a case-by -case de te rmination is the Guida nc e Doc u me nt for Re porting Re le a s e s a nd Othe r Wa ste
Appendix 44**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  Manag e ment Activities of Tox i c Chemicals: Diox in and Diox in-like Compounds (December, 2000). In a final rule published October 29, 1999, EP A lowered the reporting th reshol ds for cert a i n persi s t e ntbi oaccum ul at i v e t ox i c (P BT) chem i cal s th atare subj ect t o reporting under Section 313 of the Emerg e ncy Planning and Community Rig h t-to-Know Act of 1986 (EPCRA) and Section 6607 of the Pollution Prevention Act of 1990 (PPA). W e also added a categ ory of diox in and diox in-like compounds to the EPCRA Section 313 list of tox i c chemicals and established a 0.1 g r am reporting threshold for the categ ory . In addition, EPA added certain other PB T chemicals to the EPCRA Section 313 list of tox i c chemicals and e s tablishe d lowe r re porting thre sholds for the s e c he mic a l s. As a re sult of this rule making , we developed a g u idance document on the reporting requirements for the diox in and diox in-like compounds categ ory , as well as a number of other g u idance documents. The diox in g u idance document provides g u idance on how to estimate annual releases and other waste manag e ment quantities of diox in and diox in-like compounds to the environment from certain industries and industrial activities. Due to the hig h interest level of stakeholders, we solicited public comments on the draft g u idance document and formed a workg r oup of interested stakeholders. The workg r oup reviewed all public comments, provided their own comments, and then reviewed and commented on the final draft. Case-by-case determ ination – National Water Quality Inventory Report: A second e x a mple of a c a s e - by -c a s e de te rmination is the Na tiona l Wate r Qua lity Inve ntory Re port to Cong re ss. The Nationa l Wate r Qua lity Inve ntory Re port to Cong re ss is a bie nnia l report to Cong ress and the public about the quality of our nation' s waters. Itis prepared under Section 305 (b) of the Clean W ater Act (CW A ), which requires States and other jurisdic tions to a sse ss the he a lth of the i r wate rs a nd the e x te nt to whic h wate r qua lity supports State water quality standards and the basic g o als of the CWA. States' Section 305 (b) assessments are an important component of their water resource manag e ment prog ra ms. The s e a sse ssme n ts he lp Stat e s : imple me nt the i r wate r qua lity sta ndards by ide n tifyi ng he a lthy wate rs tha tne e dto be ma inta ine d a nd impa i re d wate rs thatne e dto be restored, prepare their Section 303 (d) lists of impaired waters, develop restoration strate g i e s suc h a s tota l ma x imum da ily loa d s a nd sourc e c ontrols, a nd e va l uate the effectiveness of activities undertaken to restore impaired waters and protect healthy waters. A numbe r of c o mme nte r s sa id thatEPA c r e at e d a limite d de finition of whatty pe s of information are to be considered “influential,” and that we have no rational basis to do so. A number of commenters also stated that “all Ag ency information should be considered influential”; that “all data relied upon by the Ag ency should meet a hig h standard of quality reg ardless of the type”; or that “‘influential’ information includes information used to support any EPA action, not just ‘top’ Ag ency actions.” EPA followed OMB ’ s g u idelines in establishing a definition for “influential” information that was not all-encompassing . OMB stated “the more important the information, the hig h er the quality standards to which it should be held, for ex ample, in those situations involving “influential scientific, financial or statistical information...”. OMB narrowed the de finition of “ i nflue n tia l” in the i r fina l g u ida nc e a s follows:

Appendix 45**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  In this narrower definition, “influential”, when used in the phrase “influential scientific, financial, or statistical information”, is amended to mean that “the a g e nc y c a n re a s onably de te rmine thatdisse mination of the information will ha ve or does have a clear and substantial impact on important public policies or important private sector decisions” (67 F R8455). OMB al so am ended th ei r defi ni t i on t o say th at“each ag ency i s authori z ed t o defi ne “i nfl u ent i a l ” in wa y s appropriate for it g i ve n the nature a nd multiplic ity of issue s for whic h the a g e nc y is responsible” (67 F R8455). We adopted OMB ’ s “influential” definition. Once the Ag ency reviewed the wide rang e of information disseminated to the public, such as major rulemaking s, risk assessments, rule related g u idance, health advisories, annual reports, fact sheets, and coloring books, it became apparent that there were reasons to disting u ish between “influential” information and other information. EPA adopted OMB ’ s definition for “influential” and used ty

pes of information the Ag ency disseminates to further ex plain what information is included. Another commenter sug g e sted that EPA should not indicate whether disseminated information is “influential” when it is first disseminated but should wait to desig n ate information as
“ i nflue n tia l” until e ithe r a n information c o rre c tion re que st is ma de or a fina l a g e nc y a c tion is taken. W e intend to consider this point, as well as other comments made about when disse minat e dinformation be c o me s influe ntia l, a s the Ag e nc y imple me nts the Guide line s . One commenter sug g e sts that the definition of the term “influential” should be more narrow. S p eci fi cal l y , th e com m e nt er st at es th e fol l o wi ng : W ithin the relatively narrow sphere of “disseminated” information, an ag ency should reserve the desig n ation of “influential” for information disseminated in support of ag ency actions that are “major” reg u lations under Ex ecutive Order 12866, provide a “sig nificant” opportunity to advance the ag ency ’s mandate by other means, or involve precedent-setting or reasonably controverted issues. This de sig nation re c og n ize s thatproc e dure s to promote the qua lity of information ha ve sig n ific a n t c o sts, a nd thatthe most sig n ific a n t (a nd the r e f ore the most c o stly ) of such procedures should be reserved for information that is the most important in t e rm s of th e ag ency ’s m i ssi on. EP A ag rees wi thth e com m e nt er th at th ere are si g ni f i cantcost s associ at ed wi th ensuri ng th at information disse minat e dby the Ag e nc y is of hig h qua lity . Conse que ntly , EPA c hose a de finition of the te rm “ i nflue n tia l” to c ove r information that , whe n disse minat e d, will re sult in a clear and substantial impact on important public policies and private sector decisions. W e be lie ve thatthis de finition ba la nc e s the c o sts a ssoc i at e dwith imple me nting the Guide line s , the need to ensure hig h quality information, and the Ag ency ’s mission to protect human health and safeg u ard the natural environment. Several commenters indicated that it is inappropriate for EPA to base its definition of “influential” on categ ories of actions. They sug g e st that the definition be based instead on the
Appendix 46**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  content of the information. We consider our definition to be based on information content, g i ven that those categ ories of disseminated information we defined as influential are those that EPA can reasonably determine will or do have a clear and substantial impact on important public policies or private sector decisions. W e note here that, in addition to the specific classes of di ssem i n at ed i n form at i on we have defi ned as “i nfl u ent i a l , ” EP A has rei t e rat e d th e “case-by

case” portion of the OMB “influential” definition. This g e neral provision is intended to capture disseminated information, based on its content, that would not otherwise rise to the level of “influential” under the other parts of our definition (i.e., top Ag ency actions, Economically Sig n ificant actions, major peer reviewed products). Several commenters assert that EPA should categ orically state that certain specific ty pes of disseminated information products are influential, and that we should categ orically state that certain specific types of disseminated information products are not influential. Given the vast array of information disseminated by the Ag ency , and g i ven the fact that certain information may have a clear and substantial impact on important public policies or private sector decisions atone time , but not ha ve suc h a n impa c tlate r on (a nd vic e ve rsa ) , c l a ssifying ty pe s of information as “influential” or otherwise upfront is difficult and could be misleading . W e intend to rely on our definition in determining whether specific types of disseminated information products are to be considered “influential” for purposes of the Guidelines. A.3.5 Reproducibility Some commenters stated that there needs to be more clarity in the definition of “reproducibility ” and related concepts. W e have tried to provide definitions that are consistent with OMB g u idelines. Also, our Guidelines now include that EPA intends to ensure reproducibility for disseminated orig inal and supporting data according to commonly accepted scientific, financial, or statistical standards. Many commenters thoug ht there should be some kind of method to consider reproducibility when proprietary models, methods, desig n s, and data are used in a dissemination. Some commenters discourag e all use of proprietary models; others sug g e st proprie tary mode l use be minimize dwith applic ation limite d to situations in whic h it is absolutely necessary . W e understand this concern, but note that there are other factors that are appropriate ly c onside r e dwhe n de c i ding whe the r to use proprie tary mode ls, inc l uding fe a s ibility and cost considerations (e.g ., it may be more cost-effective for the Ag ency to use a proprietary model in some situations than to develop its own model). In cases where the Ag ency relies on proprie tary mode ls, the s e mode l applic ations ar e still subje c tto our Pe e r Re vie w Polic y . F u rthe r, as recently directed by the Administrator, the Ag ency ' s Council on Reg u latory Environmental Modeling is now revitaliz ing its development of principles for evaluating the use of environmental models with reg ard to model validation and certification issues, building on current g ood modeling practices. In addition, these Guidelines provide for the use of especially rig o rous “robustness checks” and documentation of what checks were undertaken. These steps, along with transparency about the sources of data used, various assumptions employ ed, analytic methods applied, and statistical procedures employ ed should assure that analytic results are “capable of being substantially reproduced.”

Appendix 47**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  Reg arding robustness checks, commenters were concerned that the EPA did not use the term “especially rig o rous robustness checks.” W e have modified our Guidelines to include this term. Some c o mme nte r s spe c u late d on the ab ility of the Ag e nc y ' s Pe e r Re vie w prog ra m to me e tthe intent of the Guidelines and were concerned about the process to rebut a peer review used to support the objectivity demonstration for disseminated information. Our Peer Review prog ram has been subject to ex ternal review and we routinely verify implementation of the prog ram. Affected persons wishing to rebut a formal peer review may do so using the complaint resolution process in these Guidelines, provided that the information being questioned is considered to be “di ssem i n at ed” accordi ng t o th e Gui d el i n es. R e g ardi ng anal yt i c resul t s , som e com m e nt ers i ndi cat ed th at th e tr ansparency fact ors i d ent i f i e d by EPA (section 6.3 of the Guidelines) are not a complete list of the items that would be needed to demonstrate a hig h er deg r ee of quality for influential information. EPA ag reed with the list of four ite ms thatwa s initia lly provide d by the OMB a nd re c og n ize s that , in some c a s e s , a dditiona l information re g arding disse minat e dinformation would fa c ilitate inc r e a s e dqua lity . Howe ve r, g i ven the variety of information disseminated by the Ag ency , we cannot reasonably provide a dditiona l de ta ils for suc h a de monstration atthis time . Also, in re g ards to laborat ory re sults, whi c h were m e nt i oned by several com m e nt ers, th ese Gui d el i n es are not th e appropri at e pl ace t o se t out for the sc ie nc e c o mmunity EPA’s vie w of whatc onstitute s a de quate de monstration of te st method validation or minimum quality assurance and quality control. Those technical considerations should be addressed in the appropriate quality planning documentation or in reg u latory requirements. EPA has developed g e neral lang uag e addressing the concept of reproducibility and may provide more de ta il a f te r appropriate c onsultation with sc ie ntific a nd te c hnic a l c o mmunitie s, a s c a lle d for by OMB in its g u idelines. W e have already beg un to consult relevant scientific and technical ex perts within the Ag ency , and also have planned an ex pedited consultation with EPA' s Science Advisory B o ard (SAB ) on October 1, 2002. B a sed on these initial consultations, EPA may seek additional input from the SAB in 2003. These consultations will allow EPA to constructively and appropriately refine the application of ex isting policies and procedures, to further improve reproducibility . In the interim, EPA intends to base the reproducibility of disseminated orig inal and supporting data on commonly accepted scientific, financial, or statistical standards. A.3.6 Inf l uential Risk Assessm ent G e neral Risk Assessm ent R i sk assessm ent i s a process where i n form at i on i s anal y z ed t o det e rm i n e i f an envi ronm ent a l haz ard mig ht cause harm to ex posed persons and ecosy stems (paraphrased from Risk Assessment in the F e deral Government, National Research Council, 1983). That is: Risk = haz ard x ex posure F o r a chemical or other stressor to be " risky ," it must have both an inherent adverse effect on an
Appendix 48**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  org a nism, population, or other endpoint and it must be present in the environment at concentrations and locations that an org a nism, population, or other endpoint is ex posed to the stressor. Risk assessment is a tool to determine the likelihood of harm or loss of an org a nism, population, or other endpoint because of ex posure to a chemical or other stressor. To assist those who must make risk manag e ment decisions, risk assessments include discussions on uncertainty , variability and the continuum between ex posure and adverse effects. Risk a sse ssme n ts ma y be pe rforme dite rative l y , with the first ite ration e mploy ing prote c tive (c onse r vative ) a ssumptions to ide n tify possible risks. Only if pote n tia l risks ar e ide n tifie d in a screening level assessment is it necessary to pursue a more refined, data-intensive risk a sse ssme n t. The sc re e n ing le ve l a sse ssme n ts ma y not re sult in " c e n tra l e s timate s" of risk or upper and lower-bounds of risks. Nevertheless, such assessments may be useful in making reg ul at o ry deci si ons, as when th e absence of concern froma screeni ng l e vel assessm ent i s used (along with other information) to approve the new use of a pesticide or chemical or to decide whether to remediate very low levels of waste contamination.

Appendix 49**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  OM B G u ide line s In its g u ide line s OMB sta t e dthat , with re spe c tto influe ntia l information re g arding he a lth, sa fe ty or environmental risk assessments, ag encies should either adopt or adapt the quality principles in the Safe Drinking W ater Act (SDW A) Amendments of 1996. 34,35 In the backg r ound section of the OMB g u idelines, OMB ex plains that " the word ‘adapt' is intended to provide ag encies fle x ibility in apply i ng the s e princ i ple s to various type s of risk a sse ssme n t." G u ide line s De ve lopm e n t Conside r a tion EPA c ar e f ully a nd pra c tic a lly de ve lope d the a dap ta tion of the SDWA qua lity princ i ple s using our considerable ex perience conducting human health and ecolog i cal 36 risk assessments as well a s using our e x isting polic ie s a nd g u ida nc e . EPA conducts many risk assessments every y ear. Some of these are screening level assessments based on scientific ex perts' judg ments using conservative assumptions and available data and can involve human health, safety , or environmental risk assessments. Such screening assessments provide useful information that are sufficient for reg u latory purposes in instances where more elaborate, quantitative assessments are unnecessary . For ex ample, such assessments could indicate, even with conservative assumption, the level of risk does not warrant further investig ation. Other risk assessments are more detailed and quantitative and are based on research and supporting data that are g e nerated outside EPA. F o r ex ample, pesticide reviews are based on scientific studies conducted by reg i strants in accordance with our reg u lations and g u idance documents. Our test g u idelines and Good L aboratory Practices (GLP s) 37 describe sound scientific practices for conducting studies needed to assess human and environmental haz ards and ex posures. S u ch studies are not required to be peer-reviewed. R i sk assessments based on these studies can include occupational, dietary , and environmental ex posures.

34 S a f e D r i n ki n g Wat e r A c tA men dmen t s of 1996, 42 U

. S

. C

. 300g

- 1 (b)(3)(A ) & (B).
35 I nsectio nI I I

. 3

. ii.C. o f its g u i d e lin es, OMB states th at: “ W ithr e g ar d to an aly s is o f r i sk s to h u m anh ealth , s a f e t yan d th e en v i ron men tm a i n t a i n ed or di s s e mi n a t e d byth e ag en ci es , ag en ci es s h al l ei th er adopt or adaptth e eq u a lityp r in cip l es ap p lied b yCo n g r ess to r i sk in f o r m atio nu s ed an d d i ssemi n a ted p u r s u a n t to th e Saf e Dr in ki n g Wat e r A c tA men dmen t s of 1996 (42 U

. S

. C

. 300g

- 1 (b)(3)(A ) & (B)). A g en ci es res pon s i bl e f o r di s s e mi n a t i o nof v i t a l h ealthan d me d i cal in f o r m atio nsh all in terp r e t th e r e p r o d u cib ilityan d p eer - r ev iewstan d ar d s ina m a n n e rap p r o p r iate to as s u rin g th e timelyf l owof v ital in f o rm ationf r om ag en cies to me dical prov iders , patien t s , h ealthag en cies , an d th e p u b lic. I n f o r m atio nq u alitystan d ar d s m a yb e w a iv ed tem p o r ar ilyb yag en cies u n d eru r g e n t situ atio n s ( e .g ., im mi n e n t th reats to p u b lic h ealtho rh o me lan d secu rity ) inacco rd an ce w ithth e latitu d e sp ecif i ed inag en cy - s p ecif i c g u i delin es ”.
36 B ecause the assessment o f “ e nvir o nmental r i sk” is b e ing d i stinguished in OMB ’ s ad ap tatio n o f the SDW Aq u alityp r in cip l es f r o m “ h u m anh ealthr i sk ”, th e ter m " e n v i r o n men tal r i sk " as u s ed inth ese Gu id elin es d o e s n o t d i r ectlyin v o lv e h u m anh ealthco n cer n s . I no th e rw o r d s, “ e n v i r o n men tal r i sk assessmen t” is, inth is case, th e equ i v a len t to w h at EP Acomm o n l yref e rs to as “ ecolog i cal ris k as s e s s men t”.
37 40 C F Rpart 160 f o r F I F R Aan d 40 C F Rpart 792 f o r T S C A

. Appendix 50**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  The results of these risk assessments are conducted and presented to policy makers to inform their risk manag e ment decisions. EPA currently has numerous policies that provide g u idance to internal risk assessors on how to conduct a risk assessment and characteriz e risk. TheEPA Risk Characterization Policy 38and associ at ed g ui d el i n es are desi g n ed t o ensure th atcri t i cal information from each stag e of a risk assessment is used in forming conclusions about risk and thatthis information is c o mmunic at e dfrom risk a sse ssors to polic y make rs. EPA Existing P o lic ie s and G u idanc e Curre nt EPA g u ida nc e a nd polic ie s inc o rporat e qua lity princ i ple s . The s e ar e de sig ne dto e n sure th atcri t i cal i n form at i on fromeach st ag e of a ri sk assessm ent i s used i n form i ng concl u si ons about risk and that this information is communicated from risk assessors to policy makers. One ex ampl e i s the EPA Risk Characterization Policy 39which provides a sing le, centraliz ed body of risk c har a c t e rization imple me ntation g u ida nc e to he lp EPA risk a sse ssors a nd risk ma na g e rs make the risk characteriz ation process transparent and risk characteriz ation products clear, c onsiste nt a nd r e a s onable ( T CCR) . The s e pr inc i ple s ha ve be e n inc l ude d in othe r Ag e nc y ri sk assessm ent g ui d ance, such as the Guidelines for Ecological Risk Assessment

. 40 Other ex ampl es of major, overarching g u idelines for risk assessments include: Guidelines For Exposure Assessment 41 , Guide line s For Ne urotox ic ity Risk Asse ssme nt , 42 and Guidelines For Reproductive Toxicity Risk Assessment

. 43 Each of these documents has underg one ex ternal scientific peer review as well as public comment prior to publication. Additionally , individual EPA offices have developed more specific risk assessment policies to meet the particular needs of the prog rams and statutes under which they operate. 44 EPA' s commitment to sound science is evidenced by our ong oing efforts to develop and continually improve Ag ency g u idance for risk assessment.

38 h ttp : / /www . e p a . g o v/ O S P / s p c / r c p o l i c y . h t m 39 Ibi d
. 40 U S EPA ( 1998). G u i d el i n es f o r ecol o g i cal ri s k as s e s s men t (F ederal R e g i s t er 63(93): 26846- 26924). h ttp : / /www. e p a . g o v/n cea/raf
. 41 U S EPA(1992). Gu ide lin e s F o r E x p o s u re A sse ssme n t . F e deral R e g i s t er 57(104): 22888- 22938. h ttp : / /www. e p a . g o v/n cea/raf /

. 42 U S EPA(1998). Gu idelin es Fo r Neu ro to x icity Risk Assessmen t . F e deral R e g i s t er 63(93): 26926- 26954. h ttp : / /www. e p a . g o v/n cea/raf /

. 43 U S EPA(1996). G u i del i n es For Reproduct i ve T o xi ci t y Ri s k As s e s s m ent . F e deral R e g i s t er 61(212): 56274- 56322 . h ttp : / /www. e p a . g o v/n cea/raf
. 44 T h e O f f i ce of S o l i d Was t e an d Emerg e n c yR e s pon s e h a s dev e l oped T ool s f o r Ecol og i cal R i s k A sse ssme n t fo rSup e r f und Risk A sse ssme n t. One e x a m p l e is the Ec o l o g i c a l R i sk A sse ssme n t Gu id a n c e fo r Super f

und: Proces s f o r

D e s i gni ng and Conduct i ng Ecol ogi cal Ri s k As s e s s m ent s - Int e r i m Fi nal

. h ttp : / /www. e p a . g o v/ o e rrp a g e / s u p e r f u n d/ p r o g r a m s / r i s k / e c o r i s k / e c o r i s k

. h t m h ttp : / /www. e p a . g o v/ o e rrp a g e / s u p e r f u n d/ p r o g r a m s / r i s k /t o o l e c o . h t m Appendix 51**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  EPA ' s Expe r i e n c e C o nduc t i ng R i sk A sse ssm e n t s The first EPA huma n he a lth risk a sse ssme n t g u ide line s 45 were issued in 1986. In 1992, the Ag ency produced a Framework for Ecological Risk Assessment 46 whi c h was repl aced by the 1998 Ec ologic a l Risk Asse ssme n t Guide line s . 47 As emphasiz e d elsewhere in this document, the statutes administered by EPA are diverse. Althoug h the majority of risk assessments conducted wi thi n th e Ag ency are for chem i cal stressors, we al so assess ri sks t o bi ol og i cal and phy si cal stressors. In addition to risk assessment g u idelines, both the EPA Science Policy Council and the EPA Risk Assessment F o rum have coordinated efforts to address the complex issues related to data collection and analy s is for haz ard and ex posure assessments. Thus, the Ag ency has considerable ex perience in conducting both screening level and in-depth assessments for a wide array of stressors. Most environmental statutes oblig ate EPA to act to prevent adverse environmental and human health impacts. F o r many of the risks that we must address, data are sparse and consensus about a ssumptions is rare . In the c onte x t of data qua lity , we se e k to strike a ba la nc e a m ong fa irne ss, accuracy , and efficient implementation. Refusing to act until data quality improves can result in substantial harm to human health, safety , and the environment. P ublic C om m e nt s W e received a rang e of public and stakeholder comments on the adaptation of the S D W A principles for " i nfluential" human health, safety , and environmental risk assessments that are disseminated by EPA. Some commenters stated that we should adopt the SDWA quality principles for human health risk, safety and environmental risk assessments. Many commenters soug ht c l arific ation on re a s ons for EPA' s a dap tation of the SDWA qua lity princ i ple s for huma n health risk assessments and additional information on how we plan to address this process. Others urg e d us to adapt the S D W A principles rather than adopt, because of certain elements in th e S D W A pri n ci pl es th at m a y notbe appl i cabl e t o al l ri sk assessm ent s such as a "centr al estimate of human risk for the specific populations affected." Others stated that we should neither adapt nor adopt SDW A principles because the " D ata Quality Act" does not authoriz e importing decisional criteria into statutory provisions where they do not apply

. The decisional criteria set forth in SDW A are ex pressly limited to SDW A . W e also received comments at a le ve l of de ta il thatar e more appropriate for imple me ntation of the Guide line s tha n for the formulation of the Guidelines. These include comments reg arding the use of clinical human test data, and comments reg arding the use of particular ty pes of assumptions in risk assessments. To the ex tent that an affected person believes that our use of data or assumptions in a particular

45 51 F R33992- 34054, 24 S e pt em ber 1986.
46 F r amew ork F o r Ecol og i cal R i s k A s s e s s men t , U

. S

. EPA , R i s k A s s e s s men tF o ru m , 1992, EPA / 630/R

92/ 001.
47 G u i d el i n es F o r Ecol og i cal R i s k A s s e s s men t , U

. S

. EPA , R i s k A s s e s s men tF o ru m , 1998, EPA / 630/R

95/002F. h ttp://c f p u b .epa. g o v/n cea/c f m

/ ecor s k

. c f m Appendix 52**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  disse mination of information is inc onsiste nt with the s e Guide line s , the issue c a n be ra ise d a tthat time

. A fe w c o mme nte r s ra ise d a que stion re g arding a c onflic t be twe e n EPA’s e x isting polic ie s a nd the SDW A principles and asked us to identify the conflicting specific risk assessment standards and make every effort to reconcile the conflicting standards with the SDW A principles. A few commenters stated that EPA should not have two separate standards for risk assessments (i.e., one for influential and one for non-influential), but that all risk assessments should be considered influe ntia l. Anothe r stat e dthatif the r e is a c onflic t be twe e n e x isting polic ie s a nd the SDWA principles, EPA should identify the conflicting specific risk assessment standards and make e ve ry e ffort to re c onc ile the c onflic ting sta ndards with the SDWA princ i ple s . Some c o mme nte r s have questioned why the “best available, peer reviewed science and supporting studies” lang uag e of SDW A was conditioned by terms such as “to the ex tent practicable” or “as appropriate.” Adaptation of SDWA Quality P r inciples P ubl i c com m e nt s recei ved by th e Ag ency on th e draftGui d el i n es were wi del y di verg ent
. As no obvious consensus could be drawn, we carefully considered comments and arg u ments on adoption and adaptation. W e also reviewed our ex perience with the SDW A principles, ex isting polic ie s, a nd the applic ab ility a nd appropriate ne ss of the SDWA la ng ua g e with re g ard to the variety of risk assessments that we conduct and have determined that, to best meet the statutory oblig ations of the ma ny statute s EPA imple me nts, it re ma ins most appropriate to a dap t the SDW A principles to human health, safety , and environmental risk assessments. In response to public comments we have removed “as appropriate” from these Guidelines in our SDW A adaptation. EPA ag rees that the phrase peer reviewed science “as appropriate” was unclear. W e revised this statement in part (A) to "including , when available, peer-reviewed science and supporting studies.” EP A introduced such adaptations in order to accommodate the rang e of real-world situations we address in the implementation of our diverse prog rams. Numerous commenters ex pressed that EPA did not provide adequate clarifications of how we adapted the principles and what our thinking was on each adaptation. In these Guidelines we have provi ded det a i l e d cl ari f i cat i ons reg ardi ng each adapt at i on m a de t o th e ori g i n al S D W A lang uag e and other remarks reg arding our intent during the implementation of the SDW A a dap tation for influe ntia l disse minations by EPA. We dire c tre a de r to the Guide line s te x tfor such cl ari f i cat i ons. A.3.7 Com p laint Resolution A few commenters noted that EPA should outline how an affected person would rebut the presumption of objectivity afforded by peer review. EPA believes this determination would be made on a case-by -case basis considering the circumstances of a particular peer review and has

Appendix 53**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  decided not to provide specific sug g e stions for affected persons on how to rebut the presumption of objectivity afforded by a peer review. OMB and other com m e nt ers not ed th atag enci es’ g ui d el i n es needed t o m ake cl ear th ata request for correction can be filed if an affected person believes that information does not comply with the EPA Guide line s a nd the OMB g u ide line s . EPA ha s a dde d la ng ua g e in the EPA Guide line s to make sure this is more c l e ar to re a de r s. EPA received numerous comments on the EPA definition of affected persons. In the draft Guidelines, EPA had adopted OMB ' s definition. EPA ag rees with comments sug g e sting that, instead of elaborating on the definition of " a ffected person," a more open approach would be to ask compl a i n ant s t o descri be how th ey are an affect ed person wi th respect t o th e i n form at i on th at is the subje c tof the i r c o mpla int. EPA is a s king thatpe rsons submitting re que sts for c o rre c tion provide, among other thing s , such an ex planation. EP A has revised the Guidelines according l y , so thatwe ma y c onside r this information a l ong with othe r information in the c o mpla int in deciding on how to respond. Some c o mme nte r s note dthatthe EPA Guide line s do not sta t e how the proc e ss will work, specifically , for States, municipalities, and EPA. They ex pressed concern of being “caug h t in the middle,” so to speak, on tryi ng to g e t their own information corrected. EPA does not believe that the Guide line s ne e de d g r e at e r de ta ils on how Stat e s will work with EPA to a ddre ss c o mpla ints, but intends to work closely with States to better ensure timely correction. EPA does appreciate the frustration of an information owner in seeing what they deem " i ncorrect" information in a disseminated document or web site. However, EPA notes that this is a very complex issue that cannot be addressed with g e neral lang uag e in the Guidelines for all cases. Several comments indicated that EPA appears to have g i ven itself " carte blanche" authority to
" e lect not to correct" information. The commenters stated that there was no valid reason why EPA would opt out of correcting information and that all errors should be corrected. To the contrary , EPA like every F e deral ag ency wants to correct wrong information. The issue is not as simple as the correction of an improper z i p code or phone number on the EPA web site. Even these simple errors may be very complex if it would involve chang i ng data in an EPA and/or State database. Furthermore, EPA is not certain of the volume of complaints it will receive after October 1 and therefore needed to provide a g e neral provision in the Guidelines to recog n iz e that once EPA approves a request, the corrective action may vary depending on the circumstances. On a case-by -case basis, EPA will determine the appropriate corrective action for each c o mpla int. EPA de te rmine dthatthis wa s the most re a s onable approa c h

. The revision also re c og n ize s pra c tic a l limita tions on c o rre c tive a c tion for information from outside sourc e s . Se ve ra l c o mme nte r s note dthatEPA ne e d s to e s tablish time fra me s for the c o mpla int proc e ss. Commenters stated that EPA should establish time frames for when affected persons can submit a complaint on an information product, when EPA needs to responds to affected persons with a decision on discrete, factual errors, when EPA would respond to affected persons with a decision on more complex or broader interpretive issues, and when an affected person should submit a
Appendix 54**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  request for reconsideration. One commenter sug g e sted that EPA solicit all complaints at one time during a 6-month window or another time frame. EPA notes that commenters provided helpful ex amples and well thoug ht out proposals for such a suite of time frames and appreciates the public input. EPA did not a g re e on the ne e dto de ve lop two se parate time fra me s for c o mpla ints thatar e more fa c tua l in nature ve rsus those thatar e more c o mple x . One c o mme nte r sug g e sts a 15-da y time line for disc re te fa c tua l e rrors a nd a 45-da y time line for a ll othe r c ompla ints. Anothe r commenter recommended 30 day s for factual errors and 60 day s for all other complaints. Another com m e nt er advi sed EP A t o m odel thi s compl a i ntprocess accordi ng t o th e FOIA proc e ss. This c o mme nte r a l so sug g e ste d a 3-we e k time line for more nume ric c o rre c tions a nd 60 day s for " b roader interpretive issues or technical questions." W h ile EPA appreciates the value of these approaches, they mig ht be problematic to implement. However, as EPA learns more about the nature of this complaint process following some period of implementation, these sug g e sted approaches could be revisited. EPA also ag reed with commenters that a window of opportunity for commenters to submit a request for reconsideration made sense. EPA has advised affected persons in these Guidelines to submit a re que st for re c onside r ation within 90-day s of the initia l c o mpla int de c i sion by EPA. Some c o mme nte r s a s ke d thatEPA e s tablish time line s for whe n EPA would take c o rre c tive action. EPA doe s not a n tic ipat e thatthe r e would be a ny va lue in applyi ng a spe c i fic time fra me for this action and prefers to look at each complaint and appropriate corrective action on a case-by -case basis, as discussed above. Comme nte r s sug g e ste dtha t45 da y s wa s a re a s onable time fra me for EPA to g e t ba c k to the affected person with either a decision or a notice that EPA needs more time. One g r oup noted that HHS, SSA, and NRC adopted the 45-day window. EPA disag r eed with this approach and inste a dopte dfor a 90-da y time fra me similar to the DOT Guide line s . EP A received many comments on how EP A should structure its internal processes for the complaint resolution process. Several comments specifically discussed the role that OEI should play in the initial complaint and the requests for reconsideration. EPA does not ag ree that OEI should be the arbiter on all requests for reconsideration, but does view the role of OEI in the process as an important one. Namely , OEI may work to help ensure consistent responses to complaints and requests for reconsideration. Other comments recommending specific internal implementation processes are being considered as EPA desig n s the correction and request for re c onside r ation a d ministrative proc e sse s in g r e at e r de ta il. Many commenters arg u ed that Assistant Administrators and Reg i onal Administrators should not deci de request s for reconsi d erat i on because th ey woul d be bi ased or woul d have a confl i c tof interest when deciding complaints reg arding information disseminated by their own Offices or prog rams, or if they had to reconsider decisions made by their own staffs. EPA does not ag ree. This type of decision making is within the deleg ated decision making authority of EPA' s Appendix 55**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  officials, and these decisions should be presumed to be unbiased absent a specific showing that a decision maker is not impartial in a particular case. EPA does ag ree with commenters who noted thatit is importa nt to make c onsiste nt de c i sions on c r oss-c u tting information qua lity issue s . In order to achieve appropriate consistency of response to affected persons on requests for re c onside r ation a nd to e n sure thatc r oss-c u tting information qua lity issue s ar e c onside r e d a c r oss the Ag ency at a senior level, EPA intends for an ex ecutive panel to make the final decisions on all requests for reconsideration. F u rthermore, we felt it important to add g r eater detail on the time frame within which EPA would respond to a requestor on their request for reconsideration. W e have added that it is EPA’s g o al to respond to requesters reg arding requests for reconsideration within 90 day s . EP A received many recommendations in public comments to include the public in the EP A complaint process. Specifically , commenters requested that EPA notify the public about all pending requests to modify information and one commenter stated that EPA should allow the public to comment on information corrections requests for information that are considered "centr al t o a rul e m aki ng or other Fi nal Ag ency Act i on" before EP A accept s or rej ect s th e request
. As a g e neral matter, EPA does not intend to solicit public comment on how EPA should respond to requests for correction or reconsideration. EPA also does not intend to post requests for correction and requests for reconsideration on the EPA web site, but we plan to revisit this and many other aspects of the Guidelines within one y ear of implementation. EP A al so recei ved m a ny com m e nt s on how i n form at i on th at i s current l y bei ng revi ewed by EP A in response to a complaint appears to the public on the EPA web site or some other medium. Some c o mme nte r s re c o mme nde d the use of fla g s for a ll information thatha s a c o mpla int pe nding with a note thatwhe r e appropria te , c ha lle ng e dinformation will be pulle d from disse mina tion a nd removed from EPA' s web site. Other commenters stated that the information in question should be removed from public access until the resolution process has been completed. Still other commenters requested that EPA not embark on self-censorship. As a g e neral rule, EPA has decided not to flag information that has a complaint pending

. EPA believes that information that is the subject of a pending complaint should not necessarily be removed from public access based solely on the receipt of a request for correction. A.4 N ext S t ep s EPA is a c tive l y de ve loping ne w polic ie s a nd proc e dure s , a s appropriate , to improve the qua lity of information disseminated to the public. Some activities specifically support ensuring and max imizing the quality , objectivity , utility , and integ rity of information. F or instance, we are consulting with the scientific community on the subject of reproducibility . The EPA Science Advisory B o ard (SAB ) is performing an ex pedited consultation on the subject on October 1, 2002. B a sed on this initial consultation, EPA and the SAB may consider a full review of reproducibility and related information quality concepts in 2003. F u rthermore, as noted earlier, the EPA Science Policy Council has commissioned a workg r oup to develop assessment factors for consideration in assessing information that EPA collects or is voluntarily submitted in support of various Ag ency decisions.
Appendix 56**Guidelines for E**  **n s**  **u r**  **i n**  **g and Maxim**  **i zing the Quality**  **, Objectiv**  **ity ,**  **U t**  **ility ,**  **and Integr ity**  **of Infor m**  **a tion D**  **i ssem**  **inated by EPA**  As new processes, policies, and procedures are considered and adopted into Ag ency operations, we will c onside r the i r re la tionship to the Guide line s a nd de te rmine the e x te nt to whic h the Gui d el i n es m a y need t o chang e t o accom m odat e new act i vi t y .

Appendix 57www.epa.gov/oei