EPA/100/B-06/002 

U.S. Environmental Protection Agency
 

Peer Review Handbook
 

3rd Edition
 

Prepared for the U.S. Environmental Protection Agency
 

by Members of the Peer Review Advisory Group,
 

for EPA’s Science Policy Council
 

Science Policy Council
 

U.S. Environmental Protection Agency
 

Washington, DC 20460
 
This report is dedicated to the memory of our friend and colleague, 

Nash Gerald. Nash was a wonderful colleague who thoughtfully 

strived to perfect the use of peer review at EPA. 

Nash joined EPA in 1989. In addition to his duties coordinating 

science/policy and research for the Office of Air Quality Planning 

and Standards, he consistently contributed to considerations of the 

Agency's peer review practices. Nash’s courteous and 

indefatigable approach to resolving details greatly enhanced the 

revision of this Handbook. He will be missed. 
 Peer Review Handbook Page iii 

Disclaimer 

This Handbook was developed by EPA to provide guidance to EPA staff and managers who are 

planning and conducting peer reviews. This Handbook is intended to improve the internal 

management of EPA by providing recommended procedures and approaches for EPA staff and 

managers. This Handbook is a guidance manual and not a rule or regulation. The Handbook 

does not replace existing laws or regulation, does not change or substitute for any legal 

requirement, and is not legally enforceable. This Handbook does not create or confer legal rights 

or impose any legally binding requirements on EPA or any party. The use of non-mandatory 

language such as "may," "can" or "should" in this Handbook does not connote a requirement but 

does indicate EPA’s strongly preferred approach to ensure the quality of peer reviews conducted 

or initiated by EPA. Mention of trade names or commercial products does not constitute 

endorsement or recommendation for use. 
 

 

 

 Peer Review Handbook Page iv 

The Peer Review Advisory Group 

Connie Bosma ORD 

Jennifer Bowen OPEI 

Michele Burgess OSWER (now at USDA) 

Mary E Clark OAR 

 Kerry Dearfield ORD (now at USDA) 

 Patrick Easter OSWER 

Daniel Fort SAB 

Nash Gerald OAR 

Roland Hemmett Region 2 

 Vincia Holloman OEI 

 Joyce Jatko ORD (now at Navy) 

Barbara Klieforth ORD 

Steven Knott OPPTS 

Andrew McIntosh AO (now at GAO) 

John E Miller ORD 

Manisha Patel OGC 

Mary Reiley OW 

Nancy Wentworth OEI 
 Peer Review Handbook Page v 

Table Of Contents 

DISCLAIMER ...............................................................................................................................iii 

FOREWORD ...............................................................................................................................xiii 

LIST OF ACRONYMS ................................................................................................................ xv 

QUICK START GUIDE TO THE PEER REVIEW PROCESS................................................... 1 

Figure 1 – Flowchart for Planning a Peer Review...................................................................... 2 

Figure 2 - Flowchart for Conducting a Peer Review .................................................................. 3 

Figure 3 - Flowchart for Completing a Peer Review.................................................................. 4 

Manager’s Planning Checklist for Peer Review ......................................................................... 5 

Regulatory Action Development Checklist for Workgroups...................................................... 6 

PEER REVIEW GUIDANCE ........................................................................................................ 9 

1. THE NEED FOR PEER REVIEW.......................................................................................... 11 

1.1 Overview Statement...................................................................................................... 11 

1.2 Understanding Peer Review.......................................................................................... 11 

1.2.1 Why Use Peer Review? ........................................................................................ 11 

1.2.2 What is Peer Involvement? ................................................................................... 12 

1.2.3 What is Peer Review? ........................................................................................... 12 

1.2.4 What is Peer Input?............................................................................................... 12 

1.2.5 How is Peer Review Different from Peer Input?.................................................. 13 

1.2.6 Can Someone Who Provided Peer Input Become an Independent Peer Reviewer 

for the Same Work Product Later in the Process? ................................................................ 13 

1.2.7 Can the Same Peer Reviewer be Used More Than Once if a Product Will Be Peer 

Reviewed More Than Once, and Can the Same Peer Reviewer be Used Again and Again for 

Different Products? ............................................................................................................... 13 

1.2.8 How is Peer Review Different from Public Comment?........................................ 14 

1.2.9 How is Peer Review Different from Stakeholder Involvement? .......................... 14 

1.2.10 What Role Does Peer Review Have in Regulatory Development? ...................... 14 

1.2.11 How Does the Tiering Process Affect Peer Review? ........................................... 15 

1.2.12 Should You Discuss Peer Review in the Analytic Blueprint for Your Regulation?
 

15 

1.2.13 What Role does Peer Review Have in Regulatory Negotiations? ........................ 16 
 

1.2.14 Should You Address Peer Review in the Preamble of a Regulation? .................. 16 

1.2.15 How is Peer Review Documented in the Action Memo for Regulations? ........... 16 

1.3. Peer Review and Information Quality .......................................................................... 16 

1.3.1 Overview Statement.............................................................................................. 16 

1.3.2 How Does Peer Review Ensure the Quality of Information EPA Disseminates? 17 

1.3.3 How Do You Ensure the Quality of Data Supporting EPA’s Information 

Products?17 

1.3.4 Are Work Products Undergoing Peer Review Subject to the Guidelines?........... 18 

1.4 Annual Agency Reporting ............................................................................................ 19 

1.4.1 What Are EPA’s Reporting Practices and What Information Is Publicly 

Available? ............................................................................................................................. 19 

1.4.3 How Will the Public Be Allowed to Comment on the Peer Review Plans for 

Influential Scientific Information and Highly Influential Scientific Assessments? ............. 21 

1.4.4 What Is ORD’s Role in Reviewing the Science Inventory and How Will This 

Review Be Conducted?......................................................................................................... 21 

1.4.5 How Will EPA Submit an Annual Report to OMB? ............................................ 21 

1.5 The Roles of People and Organizations in Peer Review .............................................. 22 

1.5.1 Who is Ultimately Accountable for Peer Review?............................................... 22 

1.5.2 Who are the Agency Staff involved in Peer Review?........................................... 22 

1.5.3 Who are the Decision Makers and What are Their Responsibilities? .................. 22 

1.5.4 Who are the Peer Review Coordinators and What are Their Responsibilities? ... 23 

1.5.5 Who are the Peer Review Leaders and What are Their Responsibilities?............ 25 

1.5.6 Who are the Peer Reviewers? ............................................................................... 26 

1.5.7 What are the Responsibilities of Peer Reviewers? ............................................... 26 

1.5.8 What is an Independent Peer Reviewer? .............................................................. 26 

1.5.9 When does an Agency Internal Peer Reviewer Qualify as Independent? ............ 27 

1.5.10 What is a Peer Review Panel? .............................................................................. 27 

1.5.11 What is a Subject Matter Expert? ......................................................................... 27 

1.5.12 What is the Role of the Science Policy Council (SPC)?....................................... 27 

1.5.13 What is the Role of the Peer Review Advisory Group (PRAG)?......................... 28 

1.5.14 What is the Role of the Office of Research and Development (ORD)? ............... 28 

2. PLANNING A PEER REVIEW.............................................................................................. 29 

2.1 Overview Statement...................................................................................................... 29 

 Peer Review Handbook Page vi 
 

2.2 Determining Which Work Products to Peer Review .................................................... 29 

2.2.1 What are Scientific and/or Technical Work Products?......................................... 29 

2.2.2 What Scientific and/or Technical Work Products Need Peer Review?................ 30 

2.2.3 How Does One Determine Whether a Scientific and/or Technical Work Product is 

Influential Scientific Information?........................................................................................ 30 

2.2.4 How Does One Determine Whether Influential Scientific Information is a Highly 

Influential Scientific Assessment?........................................................................................ 32 

2.2.5 What Economic Work Products Need Peer Review?........................................... 34 

2.2.6 Should Economic Work Products Prepared in Support of Regulations that are 

Classified as Major or Economically Significant be Peer Reviewed?.................................. 35 

2.2.7 What Other Economic Work Products Might Benefit from Peer Review? .......... 36 

2.2.8 What Other Social Science Work Products Need Peer Review?.......................... 36 

2.2.9 How Should Peer Review be Handled for Products Developed Under an 

Interagency Agreement (IAG)? ............................................................................................ 37 

2.2.10 Should Products from Contracts, Grants, and Cooperative Agreements Receive 

Peer Review? ........................................................................................................................ 37 

2.2.11 How Does Peer Review Apply to Products Generated Through EPA Grants or 

Cooperative Agreements?..................................................................................................... 38 

2.2.12 Can the Recipient of a Grant or Cooperative Agreement Use Agreement Funds to 

Pay Peer Reviewers of their Work Products?....................................................................... 39 

2.2.13 Do Products Generated Under EPA Grants or Cooperative Agreements Need to 

be Reported in the Science Inventory ................................................................................... 40 

2.2.14 Should Site-Specific Decisions be Subject to Peer Review?................................ 40 

2.2.15 Should NEPA Products be Subject to Peer Review?............................................ 40 

2.2.16 Should Environmental Regulatory Models be Peer Reviewed? ........................... 41 

2.2.17 Should Another Organization’s Work Products That Have Been Submitted to 

EPA for Use in Decision Making be Peer Reviewed?.......................................................... 41 

2.2.18 Can Work Products That are Not Determined to be Influential Scientific 

Information or Highly Influential Scientific Assessments Still be Peer Reviewed? ............ 42 

2.3 Determining Which Work Products Do Not Receive Peer Review ............................. 42 

2.3.1 What Information Is Exempt from the OMB Bulletin Provisions? ...................... 42 

2.3.2 Are There Other Circumstances Where Peer Review Is Not Necessary? ............ 43 

2.3.3 For Information That Is Not Exempt, Are There Circumstances Where the Peer 

Review Provisions of the OMB Bulletin Can Be Waived or Deferred?............................... 43 

2.3.4 Does Publication in a Refereed Scientific Journal Mean That Adequate Peer 

Review Has Been Performed? .............................................................................................. 43 

2.3.5 Do Voluntary Consensus Standards Undergo Peer Review? ............................... 44 

 Peer Review Handbook Page vii
 
 Peer Review Handbook Page viii 

2.4 Choosing a Peer Review Mechanism ........................................................................... 44 

2.4.1 How Do You Determine the Appropriate Peer Review Mechanism? .................. 44 

2.4.2 What are Some Examples of Internal Peer Review? ............................................ 46 

2.4.3 What are Some Examples of External Peer Review? ........................................... 46 

2.4.4 What is the Role of Peer Review by a Refereed Scientific Journal?.................... 47 

2.4.5 Do Agency Work Products Become Candidates for Peer Review when Peer 

Reviewed Journal Articles are Used in Support of that Work Product?............................... 47 

2.4.6 When and How Often Should Peer Review Occur? ............................................. 48 

2.4.7 What Factors are Considered in Setting the Time Frame for Peer Review? ........ 49 

2.4.8 Which Office/Region or Other Agency is Responsible for Conducting the Peer 

Review? 49 

2.5 Creating the Peer Review Record and Peer Review Reports ....................................... 49 

2.5.1 What is the Peer Review Record?......................................................................... 49 

2.5.2 How Can the Peer Review Record Improve the Peer Review Process?............... 50 

2.5.3 What Should Be in the Peer Review Record? ...................................................... 50 

2.5.4 How and When Should You Develop and Post Peer Review Reports and Agency 

Responses for Influential Scientific Information and Highly Influential Scientific 

Assessments? ........................................................................................................................ 51 

2.5.5 What Should You Do with a Peer Review Record That Pertains to a Rulemaking 

Action? 51 

2.5.6 When Should the Peer Review Record Building Process Begin? ........................ 51 

2.5.7 What are the Differences in Record Keeping for a Review by Individuals 

Compared to Review by a Panel? ......................................................................................... 52 

2.5.8 Where Should the Peer Review Record be Kept and For How Long?................. 52 

2.5.9 Are Internal Peer Review Comments Included in the Peer Review Record?....... 52 

2.6 Budget Planning............................................................................................................ 53 

2.6.1 What Budgetary Factors Should You Consider in a Peer Review?...................... 53 

2.6.2 What Input is Needed for the Annual Budget Formulation and Budget Execution 

Process? 53 

2.7 Legal Considerations .................................................................................................... 53 

2.7.1 What Are the Legal Ramifications of the Peer Review Policy?........................... 53 

2.7.2 Is Legal Advice Needed?...................................................................................... 54 

2.8 Federal Advisory Committee Act (FACA) Considerations.......................................... 54 
 Peer Review Handbook Page ix 

2.8.1 When Do FACA Requirements Apply to EPA-Run Peer Reviews?.................... 54 

2.8.2 When Are EPA-Run Peer Reviews Not Subject to FACA? ................................. 55 

2.8.3 How Do You Ensure that a Contractor-Run Peer Review Does Not Become
 

Subject to FACA?................................................................................................................. 55 

3. CONDUCTING A PEER REVIEW........................................................................................ 57 

3.1 Overview Statement...................................................................................................... 57 

3.2 Charge to the Peer Reviewers....................................................................................... 57 

3.2.1 What is a Charge? ................................................................................................. 57 

3.2.2 What are the Essential Elements of a Charge? ..................................................... 58 

3.2.3 Where Can You Get an Example of a Charge? .................................................... 58 

3.2.4 Can a Stakeholder Provide Input to the Charge to the Peer Reviewers? .............. 58 

3.2.5 Who Writes the Charge When You Hire a Contractor to Conduct the Peer 

Review? 59 

3.2.6 Is it Okay to Ask a Contractor to Develop the Charge to the Peer Reviewers? ... 59 

3.3 Time Line...................................................................................................................... 59 

3.3.1 What are the Factors in Scheduling a Peer Review? ............................................ 59 

3.4 Selection of Peer Reviewers ......................................................................................... 60 

3.4.1 What are Considerations for Selecting Peer Reviewers?...................................... 60 

3.4.2 Where Do You Find Peer Reviewers?.................................................................. 60 

3.4.3 Are External or Internal Peer Reviewers Preferred?............................................. 62 

3.4.4 What is Important in the Mix of a Peer Review Panel?........................................ 62 

3.4.5 Ethical Standards .................................................................................................. 63 

3.4.6 What Techniques Help Ensure Disclosure and Appropriate Resolution of 

Conflicts of Interest?............................................................................................................. 68 

3.4.7 Are There Constraints to Selecting Peer Reviewers? ........................................... 71 

3.4.8 If State Employees Are Used as Peer Reviewers, Can EPA Pay Them for This 

Service? 72 

3.4.9 Does the Agency Recognize Peer Reviews That External Parties Conduct and/or 

Pay For? 72 

3.5 Materials for Peer Reviewers........................................................................................ 73 

3.5.1 What Instructions Do You Give Peer Reviewers?................................................ 73 
 

3.5.2 What Materials Should be Sent to Peer Reviewers?............................................. 73 

3.5.3 How Closely can EPA Interact with Peer Reviewers During the Review?.......... 75 

3.6 Peer Review Services.................................................................................................... 75 

3.6.1 What are Gratuitous Services for Peer Review?................................................... 75 

3.6.2 Can You Use a Contract to Obtain Peer Review Services?.................................. 76 

3.6.3 How Do You Write a Statement of Work for Contracts?..................................... 76 

3.6.4 What are Some Management Controls for Contracts? ......................................... 77 

3.6.5 Can You Identify and/or Select Peer Reviewers When Using a Contract? .......... 80 

3.6.6 How is Travel Handled with Contracts or Purchase Orders? ............................... 80 

3.6.7 How is Travel Handled with Special Government Employees?........................... 81 

4. COMPLETING A PEER REVIEW......................................................................................... 82 

4.1 Overview....................................................................................................................... 82 

4.2 Final Work Product....................................................................................................... 82 

4.2.1 How Do You Incorporate Peer Review Comments into the Final Work Product?
 

82 

4.2.2 How Might Peer Review Comments Impact Your Work Product?...................... 83 

4.2.3 What Should the Final Work Product Say About the Peer Review Process?....... 84 

4.2.4 Can the Identity of Peer Reviewers be Kept Anonymous?................................... 84 

4.3 Completing the Peer Review Record............................................................................ 85 

4.3.1 How Do You Complete the Peer Review Record?............................................... 85 

4.3.2 Where Should the Peer Review Records be Kept, and for How Long? ............... 85 

4.3.3 Is Information Regarding a Peer Review Subject to Release under FOIA? ......... 86 

APPENDIX A EPA’S PEER REVIEW POLICY......................................................................... 1 

APPENDIX B OMB INFORMATION QUALITY BULLETIN FOR PEER REVIEW AND 

PREAMBLE ................................................................................................................................... 1 

APPENDIX C SOUND SCIENCE AND PEER REVIEW IN RULEMAKING.......................... 2 

PEER REVIEW (CONDITIONAL TEMPLATE) ..................................................................... 4 

FRAMEWORK FOR ACTION MEMORANDA...................................................................... 5 

 Peer Review Handbook Page x 
 Peer Review Handbook Page xi 

APPENDIX D EXAMPLES OF PEER REVIEW CHARGES..................................................... 1 

CHARGE EXAMPLE 1 - US EPA Science Advisory Board (SAB) Consultation on 

Suspended and Bedded Sediments.............................................................................................. 3 

CHARGE EXAMPLE 3 - Economics - Study of Municipal Government Costs and Financial 

CHARGE EXAMPLE 4 - External Peer Review of the Design of the Targeted National 

CHARGE EXAMPLE 5 - External Peer Review of the Draft National Strategy for the
 

CHARGE EXAMPLE 6 - Clean Air Scientific Advisory Committee Review of the Agency’s 

CHARGE EXAMPLE 7 - The Multimedia, Multipathway, and Multireceptor Risk Assessment 

CHARGE EXAMPLE 2 - Economics - Benefits Transfer from Adults to Children ................ 4 

Impacts from Environmental Regulations .................................................................................. 5 

Sewage Sludge Survey for Pollutants in Sewage Sludge ........................................................... 7 

Development of Water Quality Criteria for Suspended and Bedded Sediments (SABS) .......... 9 

National Ambient Air Monitoring Strategy.............................................................................. 12 

(3MRA) Model ......................................................................................................................... 15 

APPENDIX E GUIDANCE ON REQUESTING A REVIEW BY THE SCIENCE ADVISORY
 

BOARD (SAB) ............................................................................................................................... 1 

TABLE 1 – PROCESS FOR SELECTING SCIENCE ADVISORY BOARD (SAB) 

PROJECTS ................................................................................................................................. 5 

APPENDIX F EXAMPLE STATEMENTS OF WORK FOR CONTRACTS............................. 1 

STATEMENT OF WORK - EXAMPLE 1 - Peer Review of Prioritization Tool Report ......... 2 

STATEMENT OF WORK - EXAMPLE 2 External Peer Review of Protozoa Method 

Development Criteria Document ................................................................................................ 9 

STATEMENT OF WORK - EXAMPLE 3 - Technical Support for Peer Review Services.... 11 
 Peer Review Handbook Page xii 
 Peer Review Handbook Page xiii 

Foreword 

Science is the foundation that supports all of our work here at EPA. Strong, independent science 

is of paramount importance to our environmental policies. The quality of science that underlies 

our regulations is vital to the credibility of EPA’s decisions and ultimately the Agency’s 

effectiveness in protecting human health and the environment. One important way to ensure 

decisions are based on defensible science is to have an open and transparent peer review process. 

Consistent Agency-wide implementation of peer review has been an EPA priority for many 

years. Since issuing the Peer Review Policy in 1993, EPA has taken many steps to support and 

strengthen the policy. In January 1998, EPA issued the 1st edition of the Peer Review Handbook 

to provide guidance about peer review and peer review processes. The Handbook was updated 

in December 2000 to reflect feedback from EPA’s science community on improvements to the 

usability of the Handbook. The science community found that the Handbook contributed greatly 

to the Agency goal of sound science and substantially improved EPA’s peer review process. 

In December 2004, the Office of Management and Budget (OMB) issued its “Final Information 

Quality Bulletin for Peer Review” (See Appendix B). This OMB Bulletin establishes 

government-wide guidance aimed at enhancing the practice of peer review of government 

science documents. The Bulletin includes guidance for all Federal agencies on what information 

should be subject to peer review, for selecting peer reviewers and for providing opportunities for 

public participation. It also defines a peer review planning process to allow the public to 

comment on peer review plans. While incorporating the provisions of the OMB Bulletin, EPA 

updated its Peer Review Policy to reiterate the Agency's commitment to peer review. This 

policy encourages and expects peer review of all scientific and technical information that is 

intended to inform or support Agency decisions and notes that influential scientific information, 

including highly influential scientific assessments, should be peer reviewed in accordance with 

this Handbook. 
 Peer Review Handbook Page xiv 
 

 

 

 

 Peer Review Handbook Page xv 

Acronyms 

AA Assistant Administrator 

AAS Advisory and Assistance Services 

ADP Action Development Process 

ANPR Advance Notice of Proposed Rulemaking 

ASTM American Society of Testing and Materials 

BOSC Board of Scientific Counselors 

CBI Confidential Business Information 

CCMP Comprehensive Conservation and Management Plan 

CENR Committee on the Environment and Natural Resources 

CFR Code of Federal Regulations 

CMM Contracts Management Manual 

CO Contract(ing) Officer 

COI Conflict of Interest 

COR Contracting Officer Representative 

CREM Council for Regulatory Environmental Modeling 

CTGs Control Techniques Guidelines 

DEO Deputy Ethics Official 

DFO Designated Federal Officer 

DRA Deputy Regional Administrator 

EA Economic Analysis (previously known as Regulatory Impact Analysis) 

EC Executive Council of the SAB 

ECOC Ecological Chemicals of Concern 

EEAC Environmental Economics Advisory Committee 

EIMS Environmental Information Management System 

EIS Environmental Impact Statement 

ELG Effluent Limitation Guidelines 

EMAP Environment Monitoring and Assessment Program 

EO Executive Order 

EPA Environmental Protection Agency 

EPAAR EPA Acquisition Regulations 

FACA Federal Advisory Committee Act 

FAR Federal Acquisition Regulations 

FIFRA Federal Insecticide, Fungicide and Rodenticide Act 

FIRMR Federal Information Resource Management Regulation 

FOIA Freedom of Information Act 

FR Federal Register 

FTE Full Time Equivalent 

GSA General Services Administration 

HHS Health and Human Services 
 

 

 

 

 Peer Review Handbook Page xvi 

HISA Highly Influential Scientific Assessment 

IDLH Immediately Dangerous to Life or Health Concentration 

IAG Interagency Agreement 

IGA Inherently Governmental Activities 

IGF Inherently Governmental Function 

IQGs Information Quality Guidelines 

IRIS Integrated Risk Information System 

ISC-COMPDEP Industrial Source Complex - Complex Terrain Deposition 

ISI Influential Scientific Information 

ISO International Organization for Standardization 

LAN Local Area Network 

LOC Level of Concern 

LOE Level of Effort 

NAS National Academy of Sciences 

NEPA National Environmental Policy Act 

NOAA National Oceanic and Atmospheric Administration 

NODA Notice of Data Availability 

NPDES National Pollutant Discharge Elimination System 

NPS National Park Service 

NRC National Research Council 

NRMRL National Risk Management Research Laboratory 

NTTAA National Technology Transfer and Advancement Act of 1995 

OAM Office of Acquisition Management 

OAP Office of Atmospheric Programs 

OAQPS Office of Air Quality Planning and Standards 

OAR Office of Air and Radiation 

OARM Office of Administration and Resources Management 

OCHP Office of Children’s Health Protection 

OEI Office of Environmental Information 

OGC Office of General Counsel 

OGE Office of Government Ethics 

OIA Office of International Affairs 

OIG Office of the Inspector General 

OIRA OMB's Office of Information and Regulatory Affairs 

OMB White House Office of Management and Budget 

ORC Office of Regional Counsel 

ORD Office of Research and Development 

ORIA Office of Radiation and Indoor Air 

ORMI Office of Regulatory Management and Information 

OSA Office of the Science Advisor 

OSP Office of Science Policy 
 

 

 

 Peer Review Handbook Page xvii 

OST Office of Science and Technology 

OSW Office of Solid Waste 

OTAQ Office of Transportation and Air Quality 

PAHs Polynuclear aromatic hydrocarbons 

PCB Polychlorinated biphenyl 

PCDD Polychlorinated dibenzo(p)dioxin 

PCDF Polychlorinated dibenzofuran 

PEP Peconic Estuary Program 

PL Professional Level 

PMCR Preliminary Model Calibration Report 

PO Purchase Order 

PR Procurement Request 

PPA Pollution Prevention Act 

PRA Paperwork Reduction Act 

PRAG Peer Review Advisory Group 

PRPT Peer Review Product Tracking Database 

Q&A Question and Answer 

QRT Quick Response Task 

RA Regional Administrator 

RC Regional Counsel 

RCRA Resource Conservation and Recovery Act 

RFA Regulatory Flexibility Act 

RfC Reference Concentration (Inhalation) 

RFC Request for Correction 

RfD Reference Dose (Oral) 

RFR Request for Reconsideration 

RGE Regular Government Employee 

RIA Regulatory Impact Analysis (now known as Economic Analysis) 

RI/FS Remedial Investigation/Feasibility Study 

ROD Record of Decision 

RSAC Research Strategies Advisory Committee 

SAB Science Advisory Board 

SAME Simplified Acquisition Made Easy 

SAP Scientific Advisory Panel 

SBREFA Small Business Regulatory Fairness Act 

SERA Screening level Ecological Risk Assessment 

SGE Special Government Employee 

SI Science Inventory 

SOPs Standard Operating Procedures 

SOW Statement of Work 

SPC Science Policy Council 
 

 

 

 

 Peer Review Handbook Page xviii 

SPC-SC Science Policy Council-Steering Committee 

SRA Science Review Administrator 

STAR Science To Achieve Results 

TAMS Toxics Air Monitoring System 

TD Technical Directive 

TQM Total Quality Management 

UCL Upper Confidence Limits 

URL Uniform Resource Locator 

US United States 

USC United States Code 

USEPA United States Environmental Protection Agency 

USGS United States Geological Survey 

VSCF Value of a Statistical Cancer Fatality 

VSL Value of Statistical Life 

WAM Work Assignment Manager 

WMB Waste Minimization Branch 

WTI Waste Technologies Industries 
 

 

 

 

 

 

 Peer Review Handbook Page 1 

Quick Start Guide To 

The Peer Review Process
 

This Section provides flowcharts describing the major steps in conducting a peer review and 

checklists to help plan a successful peer review. Cross references to the appropriate sections in 

the Peer Review Handbook are shown in parentheses and bolded. The flowcharts and checklists 

included are: 

$ Figure 1 - Flowchart for Planning a Peer Review 

$ Figure 2 - Flowchart for Conducting a Peer Review 

$ Figure 3 - Flowchart for Completing a Peer Review 

$ Manager’s Planning Checklist for Peer Review 

$ Regulatory Action Development Checklist for Workgroups 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 Peer Review Handbook Page 3 

Figure 2 - Flowchart for Conducting a Peer Review 

1.	 Develop the charge (§3.2) 

– Determine which key issues to address 

– Include in peer review record (§2.5.3) 

2. Select a peer review mechanism (§2.4): 

– Internal (§2.4.2) 

– External (§2.4.3) 

–	 Mail (e.g., Letter) review (§2.4.3) 

– Face to face meeting (§2.4.3) 

– One time or multiple meetings (§2.4.6) 

– Include logistical information in the peer review 

record (§2.5.3)
 

3. Determine the specific timeline(§3.3): 

– When will the review be started 

– What are the intermediate check points 

– What is the deadline for completion 

4. Select peer reviewers (§3.4): 

– Determine expertise needed (§§3.4.1, 3.4.4) 

–	 Determine sources of peer reviewers (§3.4.2) 

–	 If ISI/HISA, consider asking public to nominate peer
 

reviewers (§3.4.2) 

–	 Consider and address the balance of the panel (§3.4.4) 

–	 Consider conflicts of interest (§§3.4.5, 3.4.6) 

–	 Include documentation in peer review record (§2.5) 

5.	 Materials for the peer review (§3.5): 

–	 Obtain materials from Program for review 

– Prepare instructions for peer reviews (§3.5.1) 

– Forward materials to peer reviewers (§3.5.2) 

– Include copy of materials in peer review record (§2.5) 

6. Conduct the peer review: 

–	 If ISI/HISA, ask reviewers to prepare peer review 

report (§2.5.4) 

– If HISA, consider seeking public comment on
 

work product and allowing the public to present to
 

the peer reviewers (§§1.4.2, 1.4.3, 1.5.3, 2.4.7) 

– If HISA, and if seeking public comment, provide peer
 

reviewers with significant public comments
 

–	 Include in peer review record (§2.5) 

Note: Some of these steps may occur concurrently. 

Work Product

(Ready for Peer Review)

Develop the Charge

Select the Peer Review 

Mechanism

Set the Timelines, 

Including Deadlines

Select Peer Reviewers

Send Materials to Peer 

Reviewers & Conduct 

Review

Complete Peer Review 

and Obtain Comments 

of Peer Reviewers

Continue With 

Completing a Peer 

Review

Work Product 

(Ready for Peer Review) 

Develop the Charge 

Select the Peer Review 

Mechanism 

Set the Timelines, 

Including Deadlines

Select Peer Reviewers

Send Materials to Peer 

Reviewers & Conduct 

Review

Complete Peer Review 

and Obtain Comments 

of Peer Reviewers

Continue With 

Completing a Peer 

Review

Set the Timelines, 

Including Deadlines 

Select Peer Reviewers 

Send Materials to Peer 

Reviewers & Conduct 

Review 

Complete Peer Review 

and Obtain Comments 

of Peer Reviewers 

Continue With 

Completing a Peer 

Review 

Add to Peer Review Add to Peer Review 

Record
 

 

______________________________________________________________________________ 

 Peer Review Handbook Page 5 

Manager’s Planning Checklist for Peer Review 

1) Title of Work Product: ___________________________________________________ 

2) What Decision/Rule/Regulation/Action Does this Work Product Support: ________ 

3) Designation of Scientific and Technical Work Products 

G Is the work product scientific or technical __yes __no? 

G Is the work product __influential scientific information (ISI), __highly influential scientific 

 assessment (HISA), or __other? (See Section 2.2.3 and 2.2.4 of the Peer Review Handbook 

 for an explanation of these terms). 

4) Determining What Peer Review is Needed 

G If ISI or HISA, peer review is needed. 

G If not influential, is peer review still needed? 

G What peer review mechanism is needed (internal and/or external)? 

G When does the review need to be done? 

G How much time will be needed to conduct/complete the review? 

G Are there court ordered deadlines or other constraints? 

G Has senior management (AA/RA/others) been informed of progress/problems? 

G What would constitute success for this review? 

5) Determining the Resources for Peer Review 

G What is the priority of this project relative to other projects in the same office? 

G What resources are needed to conduct the review? 

G What are the impacts of the review on personnel? 

G Who will lead the peer review? 

G Who will conduct the peer review? 

G Who will maintain the peer review record? 

G Where will the peer review record be kept? 

G What mechanism will be used for the peer review? 

G Has the charge been developed? 

G Has internal and external coordination been initiated/completed? 

G Have arrangements for interim/final sign-offs (e.g., for the charge, the panel, on any changes 

 to the final work product) been made? 

G How will results of the review be presented and addressed in the final work product (e.g., in a 

 preamble, in an accompanying appendix – as well as changes in the work product itself)? 

G Has the work product been entered into the Science Inventory? 

6) Comments: _____________________________________________________________ 
 

 

 Peer Review Handbook Page 6 

Regulatory Action Development Checklist for Workgroups 

This checklist will help workgroups plan for peer review in the larger context of regulatory 

development. Each numbered section corresponds to a time period in the regulatory 

development process. 

1. Peer Review Prior to Proposal 

Tier 1 or Tier 2 Rule1
 

G Is the peer review schedule incorporated into the Analytic Blueprint? 

G Does this rule rely upon influential scientific information or a highly influential scientific 

assessment? 

G Will the work product be reviewed using external peer review?
 

Tier 3 Rule 

G Is the peer review schedule incorporated in the plans (e.g., the analytic blueprint) for 

producing the action? 

G Does this rule rely upon influential scientific information or a highly influential scientific 

assessment? 

G If an internal mechanism will be used for peer review, is it acceptable according to Section 

2.4 of the Peer Review Handbook? 

2.	 Sending a Proposed Rule Forward for the Administrator’s Signature 

G Has peer review been completed? 

G Does the action memo indicate whether the rule relies upon influential scientific 

information or a highly influential scientific assessment? 

G If the proposed rule relies on influential scientific information or a highly influential 

scientific assessment, is there a discussion of the peer review in the preamble of the rule? 

3. Before the Proposed Rule Publishes 

G Were the peer review report and any relevant materials included in the docket for this 

rulemaking? 

4. Peer Review Prior to Finalization 

G Is a new peer review plan necessary as a result of new regulatory options? 

5.	 Sending a Final Rule Forward for the Administrator’s Signature 

G Has any new peer review been completed? 

1

For further information on tiering and criteria used to determine the appropriate tier for an action, see 

http://intranet.epa.gov/adplibrary/adp/tiering/step2.htm 
 Peer Review Handbook Page 7 

G Does the action memo indicate whether the rule relies upon influential scientific information or 

a highly influential scientific assessment? 

G If the final rule relies on influential scientific information or a highly influential scientific 

assessment, is there a discussion of the peer review in the preamble of the rule?
 

6. Before the Final Rule Publishes 

G Were the peer review report and any relevant materials included in the docket for this 

rulemaking? 
 Peer Review Handbook Page 8 
 Peer Review Handbook Page 9 

U.S. Environmental Protection Agency 

PEER REVIEW GUIDANCE
 
 Peer Review Handbook Page 10 
 

 

 Peer Review Handbook Page 11 

1. The Need For Peer Review 

1.1 Overview Statement 

Peer review at the U.S. Environmental Protection Agency (EPA) takes many different forms 

depending on the nature of the work product, relevant statutory requirements, and office-specific 

policies and practices. In January 1993, responding to recommendations in the report 

Safeguarding the Future: Credible Science, Credible Decisions2, EPA issued an Agency-wide 

policy for peer review. In 1994 EPA reaffirmed the central role of peer review and instituted an 

Agency-wide implementation program. In 1998, a Peer Review Handbook was created as a 

single, centralized form of implementation guidance for Agency staff and managers. The Peer 

Review Handbook was revised and reissued in December 2000. The newly updated Peer 

Review Policy and this 3rd Edition of the Handbook incorporate insights from the use of the 2nd 

Edition as well as the provisions of the OMB Final Information Quality Bulletin for Peer Review 

that was issued to help agencies enhance their peer review transparency and accountability. 

Among the changes introduced by the OMB Bulletin is the use of the terms “influential scientific 

information” and “highly influential scientific assessments” (see Sections 2.2.3 and 2.2.4 for 

explanations of these terms). 

1.2 Understanding Peer Review 

1.2.1 Why Use Peer Review? 

Peer review is not free; 

however, not doing peer 

review can be costly. 

Peer review is intended to uncover any technical problems 

or unresolved issues in a preliminary (or draft) work product through the use of independent 

experts. This information is then used to revise that draft product so that the final work product 

will reflect sound technical information and analyses. Peer review is a process for enhancing a 

scientific or technical work product so that the decision or position taken by the Agency, based 

on that product, has a sound, credible basis. To be most effective, peer review of a scientific 

and/or technical work product should be incorporated into the up-front planning of any action 

based on the work product - this includes obtaining the proper resource commitments (people 

and money) and establishing realistic schedules. 

Peer review of scientific and/or technical work products should not be looked upon as another 

“hurdle” in the Agency decision making processes. Although conducting a peer review means 

that time and resources have to be included in the decision making process, the benefits justify 

the added cost. Peer review enhances the credibility and acceptance of the decision based on the 

work product. By ensuring a sound basis for decisions, greater cost savings are realized since 

2

EPA/600/9-91/050, March 1992 
 Peer Review Handbook Page 12 

decisions will not be challenged as often and extra effort will not be expended to go back and 

redo the work product. So while peer review is not free, the cost of not doing peer review is 

usually much more expensive. Furthermore, not conducting a peer review may potentially place 

the Agency in the position of having to decide whether to try to defend a scientifically 

questionable position – which can be very costly in terms of both resources, and, more 

importantly, credibility. 

1.2.2 What is Peer Involvement? 

Peer involvement is the process whereby Agency staff involve subject-matter experts from 

outside their program in one or more aspects of the development of work products. Peer 

involvement, therefore, constitutes active outreach to and participation by the broad scientific, 

engineering, public health, economics and other social science communities beyond the Agency 

(external), as well as within the Agency (internal). Typically, peer involvement takes two 

general forms: peer input (ongoing discussions during the development of the work product) and 

peer review (an evaluation of a work plan, preliminary draft or the like, or the final objective 

expert evaluation of the work product). 

1.2.3 What is Peer Review? 

Peer review is a documented critical review of a specific Agency scientific and/or technical work 

product. Peer review is conducted by qualified individuals (or organizations) who are 

independent of those who performed the work, and who are collectively equivalent in technical 

expertise (i.e., peers) to those who performed the original work. Peer review is conducted to 

ensure that activities are technically supportable, competently performed, properly documented, 

and consistent with established quality criteria. Peer review is an in-depth assessment of the 

assumptions, calculations, extrapolations, alternate interpretations, methodology, acceptance 

criteria, and conclusions pertaining to the specific major scientific and/or technical work product 

and of the documentation that supports them. Peer review may provide an evaluation of a 

subject where quantitative methods of analysis or measures of success are unavailable or 

undefined such as research and development. Peer review is usually characterized by a one-time 

interaction or a limited number of interactions by independent peer reviewers. Peer review is 

encouraged during the early stages of the project or methods selection, and/or as part of the 

culmination of the work product, as appropriate. Regardless of the timing of peer review, the 

goal is ensuring that the final product is technically sound. 

1.2.4 What is Peer Input? 

Many Agency work products are developed with the input of various scientific and technical 

experts inside and outside the Agency. Like the contribution made by peer reviewers, peer input 

is valuable and enhances the scientific or technical basis of the products. Peer input, sometimes 

referred to as peer consultation, generally connotes an interaction during the development of an 

evolving Agency work product, providing an open exchange of data, insights, and ideas. Peer 
 Peer Review Handbook Page 13 

input may be characterized by a continued and iterative interaction with scientific experts during 

work product development. A common example of peer input is the input received from 

workgroup members during development of a product. Many Agency products are developed 

through the efforts of a workgroup, which may include external experts, such as State and Tribal 

representatives. These workgroup members have an active, ongoing participation in developing 

the work product. Another example of obtaining peer input is that of an Agency Office sending 

a draft work product to a list of stakeholder representatives for general comments (stakeholder 

representatives often include experts who could be considered “peers”). 

1.2.5 How is Peer Review Different from Peer Input? 

The key distinctions between peer input as described above and formal peer review are the 

independence of the peer reviewers and their level of involvement. The goal of peer review is to 

obtain an independent, third-party review of the product from experts who have not substantially 

contributed to its development. When experts have a material stake in the outcome of the peer 

review (such as a regulated party) or have participated substantially in the development of the 

product (such as a workgroup member), those experts’ reviews may not qualify as unbiased, 

independent peer review and may be better characterized as peer input. 

It is clear that peer input provides valuable contributions to the 

development of the work product. However, peer input does 

not substitute for peer review. In other words, one cannot 

argue that a peer review is not necessary if a work product has 

received “enough” peer input. 

Peer Input is not a 

substitute for Peer Review 

1.2.6 Can Someone Who Provided Peer Input Become an Independent Peer 

Reviewer for the Same Work Product Later in the Process? 

Generally, the answer is no as that expert is no longer independent, but rather a contributor to the 

work product. There may be special circumstances where the expertise is so narrow that another 

peer reviewer is not available. The Peer Review Leader (see Section 1.5.5) will normally be 

responsible for making this determination and documenting the decision in the peer review 

record. 

1.2.7 Can the Same Peer Reviewer be Used More Than Once if a Product Will Be 

Peer Reviewed More Than Once, and Can the Same Peer Reviewer be Used 

Again and Again for Different Products? 

There is no prohibition on using the same peer reviewer more than once on the same product or 

for multiple products of the same office. However, it is preferable to use different people each 

time to provide a broader perspective. When using a contractor to provide peer review services, 

you should recognize that contractors may have a “pool” of reviewers that they use regularly. If 

the same peer reviewers are used repeatedly, they may lose their independence (or the 
 Peer Review Handbook Page 14 

appearance of independence) from the work product(s). If a peer reviewer is asked to participate 

in multiple reviews of the same product it should be noted in the peer review record. 

1.2.8 How is Peer Review Different from Public Comment? 

Peer review and public comment are not the same. Public comment solicited from the general 

public through the Federal Register or by other means is often required by the Administrative 

Procedure Act, other relevant statutes or both. Public comment may also be solicited for policy 

purposes. The Agency takes public comment on some strictly scientific products and almost all 

regulatory decisions. Public commenters usually include a broad array of people with an interest 

in the technical analysis or the regulatory decision; some are scientific experts (who may provide 

some peer input), some are experts in other areas, and some are interested non-experts. The 

critical distinction is that public comment does not necessarily draw the kind of independent, 

expert information and in-depth analyses expected from the peer review process. Public 

comment is open to all issues, whereas the peer 

review process is limited to consideration of 

specified technical issues. While it may be an 

important component of EPA’s decision making 

process, public comment does not substitute for peer 

review. 

Public comment does not substitute 

for peer review. 

1.2.9 How is Peer Review Different from Stakeholder Involvement? 

Stakeholder involvement occurs when the Agency works with external interest groups that have 

some stake in or concerns about the outcome of the technical work product or regulatory 

position. This is an interactive process which usually involves other agencies, industry groups, 

regulated-community experts, environmental groups, other interest groups that represent a broad 

spectrum of the regulated community, etc., and usually strives for a consensus approach. The 

goal of peer review, on the other hand, is to obtain an 

independent, third-party review for ensuring scientific 

quality and technical credibility of the work product that 

supports a policy or decision. 

Stakeholder involvement is not 

a peer review mechanism. 

1.2.10 What Role Does Peer Review Have in 

Regulatory Development? 

Peer review of scientific and/or technical work products that support regulations is an important, 

fundamental step in the policy setting and regulatory development processes and affirms the 

credibility of the Agency. Because the work products supporting regulations often are subject to 

intense scrutiny by the general public and the stakeholders involved, you should plan ahead and 

document the peer review process for such work products. 
 

 

 

 Peer Review Handbook Page 15 

In general, peer review should be completed prior to issuance of the proposed regulation. In 

some cases, work products that support final regulations may need an additional peer review if 

those scientific and/or technical work products change significantly after the public comment 

period. 

A regulation itself is not subject to the Peer Review Policy. However, if a regulation is 

supported by influential scientific information or a highly influential scientific assessment, the 

underlying work product should be peer reviewed before EPA issues the proposed regulation. 

See Sections 2.2.3 and 2.2.4 for a more complete explanation of what is considered “influential 

scientific information” or a “highly influential scientific assessment” and for guidance in 

determining whether a work product contains this type of information. Remember, soliciting 

comments from stakeholders and the general public after a rule is proposed does not constitute 

peer review. 

The original data and formal analytic models used in Economic Analyses (EAs) are subject to 

the OMB Bulletin if they constitute influential scientific information or highly influential 

scientific assessments. However, the straightforward application of data, models, and scientific 

and economic analyses used in previously peer-reviewed EAs and economic assessments or 

other peer-reviewed products are not subject to additional formal peer review. 

1.2.11 How Does the Rulemaking Tiering Process Affect Peer Review? 

Tier 1 and Tier 2 rulemakings are, by definition, important Agency rulemakings3

. Therefore, you 

should carefully scrutinize work products supporting Tier 1 and Tier 2 rules to determine 

whether they should undergo peer review. In almost all cases you should use external peer 

review for a work product that is intended to support a Tier 1 or Tier 2 rulemaking. Although 

acceptable in certain circumstances for influential scientific information, the use of an internal 

peer review mechanism for such work products should be avoided. 

Work products supporting Tier 3 rulemakings may also benefit from peer review. For work 

products supporting a Tier 3 rule, internal or external peer review may be appropriate depending 

on the nature of the product and other factors. You can find more information on the differences 

between internal and external peer review in Section 2.4. 

1.2.12 Should You Discuss Peer Review in the Analytic Blueprint for Your 

Regulation? 

Workgroups should specifically address peer review in each analytic blueprint. For peer review 

purposes, development of the analytic blueprint is the process whereby the workgroup identifies 

3 For further information on tiering and criteria used to determine the appropriate tier for an action, see 

http://intranet.epa.gov/adplibrary/adp/tiering/step2.htm 
 

 

 

 

 Peer Review Handbook Page 16 

the supporting scientific and/or technical work products and recommends what kind of peer 

review is needed. The analytic blueprint should show the schedule of the peer review in the 

context of the schedule for the overall rulemaking. 

1.2.13 What Role does Peer Review Have in Regulatory Negotiations? 

Regulatory negotiations are not generally subject to peer review; however, to ensure final 

decisions are based on sound and credible science, where possible, work products that support 

the negotiation should be subjected to peer review before the negotiation takes place. 

1.2.14 Should You Address Peer Review in the Preamble of a Regulation? 

Generally, yes. For proposed and final regulations that rely on influential scientific information 

or a highly influential scientific assessment, you should discuss the peer review report in the 

preamble, as described in the OMB Bulletin. If you are not the rulewriter, take steps to ensure 

the regulatory workgroup is aware of this provision of the OMB Bulletin. See Appendix C – 

Sound Science and Peer Review in Rulemaking, for template information. 

1.2.15 How is Peer Review Documented in the Action Memo for Regulations? 

For all rules requiring the Administrator's signature (proposed and final), indicate in the action 

memo what kind of peer review took place. See Appendix C for more information. 

1.3. Peer Review and Information Quality 

1.3.1 Overview Statement 

EPA is committed to providing quality environmental information to its partners and the public. 

This commitment is integral to its mission to protect human health and the environment. One of 

its goals is that all parts of society have access to accurate information sufficient to effectively 

participate in managing human health and environmental risks. To fulfill this and other 

important goals, EPA strives to rely upon information of appropriate quality for each decision 

made. In early 2002, pursuant to the Information Quality Act4

, the OMB developed 

government-wide guidance for “ensuring and maximizing” the quality of information Federal 

agencies disseminate to the public.5

 In late 2002, EPA developed and issued its Information 

4

The Information Quality Act, also known as the Data Quality Act, was issued by Congress, under Section 

515(a) of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554; 

H.R. 5658) 

5

Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information 

Disseminated by Federal Agencies, OMB, 2002 (67 FR 8452). 
 

 

 

 Peer Review Handbook Page 17 

Quality Guidelines6

 for ensuring the information that supports EPA’s mission is reliable and 

accurate (objectivity); is appropriate for the intended use (utility); and, is protected from 

compromise (integrity). Peer review, when appropriate, enhances the quality of the information 

EPA disseminates. 

1.3.2 How Does Peer Review Ensure the Quality of Information EPA 

Disseminates? 

The Agency recognizes peer review as a component of pre-dissemination review that 

complements and enhances the “objectivity” and “utility” of EPA’s information products. The 

Agency recommends that offices conduct pre-dissemination reviews of information to ensure 

that the information is of appropriate quality before it is disseminated to the public. Predissemination review is especially important for influential scientific information and highly 

influential scientific assessments. 

1.3.3 How Do You Ensure the Quality of Data Supporting EPA’s Information 

Products? 

EPA’s Quality System7

 provides guidance on systematic planning for work products that involve 

the collection of new environmental data or the use of existing environmental data. The 

Agency’s policies for planning for the appropriate level of quality assurance and peer review 

ensure that the information the public receives is reliable and defensible. Documentation of this 

planning occurs at various stages in development of the final product: 

a) A Quality Assurance Project Plan or similar documentation is developed and approved 

prior to the collection of new data, or use of existing data. Systematic planning helps 

ensure that the data and information are of the right type, quality and quantity for the 

conclusions reached in the final work product. 

b) EPA’s Action Development Process (ADP)8

 is another mechanism that assists the 

Agency in achieving the objectivity and transparency of information used in developing 

6

Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity, of Information 

Disseminated by the Environmental Protection Agency, EPA/260R-02-008 (October 2002), 

(http://www.epa.gov/quality/informationguidelines) 

7

EPA’s Quality System provides guidance on systematic planning for the management, collection and use 

of all environmental data to ensure the data are of the right type, quantity and quality for the intended use. EPA 

Quality Manual for Environmental Programs 5360 A1. (May 2000), Section 1.3.1, 

http://www.epa.gov/QUALITY/qs-docs/5360.pdf 

8

EPA's Action Development Process: Guidance for EPA Staff on Developing Quality Actions, EPA June 

2004, http://intranet.epa.gov/adplibrary/index.htm 
 

 

 

 Peer Review Handbook Page 18 

regulations, economic and policy decisions. The stakeholder involvement and 

consultation that are part of the ADP enhance EPA’s efforts to disseminate quality 

information that is useful to the public, and are a valuable component of predissemination review. 

c)	 The guidance entitled General Assessment Factors for Evaluating the Quality of 

Scientific and Technical Information9

 is an additional resource for EPA’s staff as they 

evaluate the quality and utility of information, regardless of the source. The document 

complements EPA’s Information Quality Guidelines by raising the awareness of the 

information-generating public about EPA’s ongoing interest in ensuring and enhancing 

the quality of information available for Agency use. 

d)	 Peer review may be an important assessment tool. For example, peer review may be used 

by researchers to confirm the robustness of a theory or design before proceeding to the 

next level, such as field sampling or fabrication of new equipment. This is particularly 

helpful in "cutting edge" situations where technical standards or other measures of 

validation may not yet exist. 

1.3.4 Are Work Products Undergoing Peer Review Subject to the Guidelines? 

Products that are undergoing peer review are not considered to be disseminated under EPA’s 

Information Quality Guidelines because they are dynamic documents that are subject to change 

and therefore, do not represent EPA’s final decision or position. These products should contain 

the following disclaimer: 

This information is distributed solely for the purpose of pre-dissemination peer 

review under applicable information quality guidelines. It has not been formally 

disseminated by EPA. It does not represent and should not be construed to 

represent any Agency determination or policy. 

In cases where the information is highly relevant to specific policy or regulatory deliberations, 

this disclaimer should appear on each page of the work product. Peer review products that are 

disseminated are subject to EPA’s Information Quality Guidelines. 

9 General Assessment Factors for Evaluating the Quality of Scientific and Technical Information, EPA 

(June 2003), http://www.epa.gov/osa/spc/pdfs/assess2.pdf 
 Peer Review Handbook Page 19 

1.4 Annual Agency Reporting 

1.4.1 What Are EPA’s Reporting Practices and What Information Is Publicly 

Available? 

EPA maintains a database, called the Science Inventory (http://cfpub.epa.gov/si), which is 

designed to track and report science activities across the Agency, including their status and peer 

review plans (See section 1.4.2). Offices are expected to keep this information current; agenda 

entries for the highly influential scientific assessments and influential scientific information 

products should be updated at least every six months. Real-time updates may occasionally be 

necessary, for instance if there is an imminent change in the timing for the peer review of a high 

visibility study or a change in the timing of the public availability of a draft of a highly 

influential assessment. 

EPA will make its peer review plans for highly influential scientific assessments and influential 

scientific information available for public comment through the Science Inventory. Offices are 

to consider any comments that are submitted on the peer review plans for which they are 

responsible. The OMB Bulletin also calls for final peer review reports for the highly influential 

scientific assessments and influential scientific information to be developed and made available 

to the public. EPA will make these reports available on the Science Inventory web site, which 

will directly link to the peer review agenda entry for that item. Additional information on the 

content of these reports is provided in Section 2.5. 

The Deputy Administrator has designated the Office of Research and Development (ORD) to 

conduct an annual review of the peer review plans. As called for in OMB's Peer Review 

Bulletin, ORD expects to submit a report to OMB by December 15 of each year. This report will 

include information concerning the peer reviews conducted on the highly influential scientific 

assessments and influential scientific information during the previous fiscal year. ORD will 

generate this report from the information in the Science Inventory database. 

1.4.2 What Information Is Provided in the Science Inventory? 

The Science Inventory database is designed to track and report peer review and other science 

activities across the Agency. The database is the single repository for product-specific peer 

review reporting and tracking and uses a common reporting form. Peer review work products 

are divided into three categories: highly influential scientific assessments, influential scientific 

information, and other products. 

Each AA/RA will designate a Peer Review Coordinator (See section 1.5.4) to ensure that their 

science activities and peer review products are fully represented, accurate, and complete. 

Specific information is included for each work product, including a summary or abstract of the 

product, what the product will be used for, and whether the product will undergo peer review. If 

the product will undergo peer review, additional information regarding the peer review plans is 
 Peer Review Handbook Page 20 

provided. If the activity or product is considered to be influential scientific information or a 

highly influential scientific assessment, the following information should be provided to address 

the provision of the OMB Bulletin that allows the public to view and comment on the Agency’s 

peer review plans for these activities or products: 

a) A paragraph including the title, subject and purpose of the activity or product; 

b) An Agency contact to whom inquiries may be directed to learn the specifics of the peer 

review plan; 

c) Whether the dissemination is likely to be influential scientific information or a highly 

influential scientific assessment; 

d) The timing of the review (including deferrals); 

e) Whether the review will be conducted through a panel or individual letters (or whether an 

alternative procedure will be employed); 

f) Whether there will be opportunities for the public to comment on the work product to be 

peer reviewed and, if so, how and when these opportunities will be provided; 

g) Whether EPA will provide significant and relevant public comments to the peer 

reviewers before they conduct their review; 

h) The anticipated number of reviewers (3 or fewer; 4-10; or more than 10); 

i) A succinct description of the primary disciplines or expertise needed in the review; 

j) Whether reviewers will be selected by EPA or by a designated outside organization; 

k) Whether the public, including scientific or professional societies, will be asked to 

nominate peer reviewers. 

If the product will not undergo peer review, Offices are asked to provide a statement on why peer 

review is not necessary (see Section 2.3 for further details). 

Each record should be electronically signed by the Peer Review Coordinator. By signing it, the 

Peer Review Coordinator indicates that it is ready for ORD review and, in the case of highly 

influential scientific assessments and influential scientific information, is ready to be made 

available for public comment. 
 

 

 Peer Review Handbook Page 21 

1.4.3 How Will the Public Be Allowed to Comment on the Peer Review Plans for 

Influential Scientific Information and Highly Influential Scientific 

Assessments? 

The OMB Bulletin directs Federal agencies to make the peer review plans for influential 

scientific information and highly influential scientific assessments publicly available for 

comment. EPA will provide for this by making electronic files of the peer review plans available 

on the Science Inventory. The public can then comment on these plans and comments will be 

directed to the peer review contact identified for the product. Offices are to consider the 

comments submitted by the public on the peer review plans for highly influential scientific 

assessments and influential scientific information, in accordance with the OMB Bulletin. 

1.4.4 What Is ORD’s Role in Reviewing the Science Inventory and How Will This 

Review Be Conducted? 

ORD conducts an annual review of the peer review products in the database. The purpose of this 

review is to ensure that the Science Inventory fully represents the Agency’s peer review products 

and to determine if the peer review decisions are consistent with the guidance presented in this 

Handbook. ORD will consult, as necessary, with the appropriate persons in each organization 

(see Section 1.5). ORD will then consolidate the information and findings for the SPC and the 

Deputy Administrator. Any conflicts arising from the review will be resolved by the Deputy 

Administrator. 

1.4.5 How Will EPA Submit an Annual Report to OMB? 

EPA expects to submit an annual report to OMB by December 15 of each year that summarizes 

the peer reviews that were conducted during the previous fiscal year for highly influential 

scientific assessments and influential scientific information. Release of any reviewer 

information retrieved by a personal identifier will be performed in accordance with the Privacy 

Act, 5 USC § 552a as amended, and as interpreted in OMB implementing guidance, 40 FR 

28,948 (July 9, 1975). According to the OMB Bulletin, the annual report should include the 

following: 

a) The number of peer reviews conducted subject to the Bulletin 

b) The number of times alternative procedures were invoked 

c) The number of times waivers or deferrals were invoked (and in the case of deferrals, the 

length of time elapsed between the deferral and the peer review) 

d) Any decision to appoint a peer reviewer pursuant to any exception to the applicable 

independence or conflict of interest standards of the Bulletin 
 Peer Review Handbook Page 22 

e) The number of peer review panels that were conducted in public and the number that 

allowed public comment 

f) The number of public comments provided on the peer review plans, and 

g) The number of peer reviewers used who were recommended by professional societies. 

1.5 The Roles of People and Organizations in Peer Review 

1.5.1 Who is Ultimately Accountable for Peer Review? 

Under the Peer Review Policy, the Administrator has designated the Assistant Administrators 

and Regional Administrators (AAs and RAs) to be accountable for implementing the Policy in 

their respective organizations. The Deputy Administrator is ultimately responsible for peer 

review across the Agency and is the final arbitrator of conflicts and concerns about peer review. 

1.5.2 Who are the Agency Staff involved in Peer Review? 

The principal Agency staff involved are Decision Makers (and their line managers), Peer Review 

Leaders and Peer Review Coordinators. In addition, there are many other critical staff in each 

Office and Region who have responsibility for peer review activities (e.g., Office and Division 

Peer Review Coordinators, technical information managers, and, of course, any Agency staff 

who serve as internal peer reviewers). Finally, ORD has oversight responsibility, as designated 

by the Deputy Administrator, for ensuring the Agency’s Peer Review Policy is implemented. 

1.5.3 Who are the Decision Makers and What are Their Responsibilities? 

The AA/RA is the ultimate Decision Maker for his/her organization and is accountable for the 

decisions regarding the identification of influential scientific information and highly influential 

scientific assessments and the mechanism(s) of peer review utilized for each of the products. 

The AA/RA may designate Office Directors and/or Division Directors as the front-line Decision 

Makers. 

Furthermore, the Decision Makers commit the resources needed to ensure a proper peer review. 

Decision Makers are responsible for ensuring that the peer reviews are properly performed and 

documented. 

In order to ensure greater independence of peer reviews, it is important to strictly separate the 

management of work products from the actual peer review of those work products. Therefore, 

the Decision Maker and the Peer Review Leader for a work product should not be the same 

person. 
 Peer Review Handbook Page 23 

The Decision Maker needs appropriate training on how to manage the peer review process. The 

Peer Review Coordinator for the Office can advise the Decision Maker on how to obtain the 

appropriate training; the training approach varies among the different EPA offices. 

Specific responsibilities of the Decision Maker(s) are these: 

a)	 Determine which work products in their organization call for peer review and which ones 

are influential scientific information or highly influential scientific assessments; 

b) Designate (in conjunction with the Project Manager) a Peer Review Leader to organize 

each peer review; 

c) Provide advice, guidance, and support to the Peer Review Leader in the preparation, 

conduct, and completion of the peer review; 

d) Ensure that sufficient funds are designated in the office’s budget request to conduct the 

peer review; also ensure that adequate resources and/or extramural management support 

are available for the peer review; 

e) Establish a realistic peer review schedule; 

f)	 Designate the stage(s) of product development where peer review is appropriate; 

g) Consider public comments on the peer review plans and consider and decide if the public 

will be asked to nominate potential peer reviewers for influential scientific information 

and for highly influential scientific assessments; 

h) For highly influential scientific assessments, decide whether it is feasible and appropriate 

to make the draft scientific assessment available to the public for comment at the same 

time it is submitted for peer review (or during the peer review process) and whether it is 

feasible and appropriate to sponsor a public meeting where oral presentations on 

scientific issues can be made to the peer reviewers by interested members of the public. 

When public comment on the assessment is sought, provide peer reviewers with 

significant comments raised by the public; 

i)	 Ensure all relevant issues and comments raised by the peer reviewer(s) are adequately 

addressed and documented for the record, and where appropriate, incorporated into the 

work product that is used as basis for decision making. 

1.5.4 Who are the Peer Review Coordinators and What are Their Responsibilities? 

The Peer Review Coordinator is designated by the AA/RA to coordinate and monitor peer 

review activities in his/her respective Office or Region. This person should be of sufficient 
 

 

 

 Peer Review Handbook Page 24 

stature and judgment to have the access to and confidence of all levels of Office or Regional 

management. The Peer Review Coordinator is the main contact for his/her organization and can 

also direct interested parties to other persons/contacts in the Office on specific work products 

(e.g., Peer Review Leader). 

Specific responsibilities of the Peer Review Coordinator are these: 

a) General oversight responsibility for the Office’s or Region’s peer review process; 

b) Report peer review activities to the AA/RA; 

c) Help mediate difficult issues between the organization and others; if an issue cannot be 

resolved, then the Peer Review Coordinator can bring the issue to the attention of the 

appropriate level Decision Makers in each organization for resolution; 

d) Function as the liaison with ORD and the Science Policy Council (SPC) to: 

1) Represent the Office/Region before the SPC on peer review issues; 

2) Advise ORD of any changes in the list of work products and peer review mechanisms 

during the annual reporting, and when necessary, at other times; 

3) Participate in Agency peer review training, workshops, etc., as requested and 

disseminate this information to the organization; coordinate and/or present training 

within the organization. 

e) Submit information on the organization’s peer review plans and activities as needed to 

implement the Peer Review Policy (See Section 1.4). On a product specific basis, this 

data submission responsibility may be re-designated within the organization: 

1) Assure that all Science Inventory records for peer review products are accurate and 

comply with OMB reporting requirements (see Section 1.4.2); 

2) Assure the Science Inventory record addresses all issues and reporting requirements 

for the annual report to OMB and sign the record, making it available to the public 

via the Science Inventory website. 

f) Establish procedures to ensure that the work product peer review documentation (i.e., 

peer review record) is filed and maintained in an appropriate manner (see Section 2.5); 

g) Provide advice, guidance, and support to the various Peer Review Leaders for the 

performance of the peer reviews; 
 Peer Review Handbook Page 25 

h) Distribute Agency-wide peer review guidance and materials to appropriate Office/Region 

personnel, as requested. 

1.5.5 Who are the Peer Review Leaders and What are Their Responsibilities? 

The Peer Review Leader is assigned by the Decision Maker to organize, conduct and complete 

the peer review for a specific individual work product. The Peer Review Leader will obtain the 

assistance and support of the Peer Review Coordinator as well as any others within the Agency 

to help perform the peer review. The Peer Review Leader will be chosen on a case by case basis 

depending on the work product needing peer review. The Peer Review Leader cannot be the 

Decision Maker. The Peer Review Leader could be the Project Manager for the work product. 

The Peer Review Leader should have appropriate training on how to conduct a peer review 

before conducting the peer review. The Peer Review Coordinator for the Office can advise the 

Peer Review Leader on how to obtain the appropriate training; the training approach varies 

among the different EPA offices, however, uniform training modules are available. 

Specific responsibilities of the Peer Review Leader are: 

a) Keep the Decision Maker informed of the status of a given project; provide the Peer 

Review Coordinator with data for the annual report; 

b) Organize, conduct, and complete the peer review following Agency procedures: 

1)	 Establish and maintain the peer review record for the specific individual peer 

review currently being performed (see Section 2.5); this includes providing the 

peer review summary information in the Science Inventory for the Peer Review 

Coordinator to sign when the peer review is completed; 

2) Select the peer reviewers in consultation with others involved with the peer 

review (e.g., Decision Maker) and ensure that conflict of interest issues are 

addressed and documented in the peer review record (see Section 3.4.6); 

3) Advise peer reviewers of their responsibilities, including preparation of a peer 

review report if the product is influential scientific information or a highly 

influential scientific assessment. 

c)	 Provide information to the Decision Maker (including all appropriate managers in the 

Peer Review Leader’s chain of command) on the charge, profile of peer reviewers, the 

peer review comments, and a proposal on how to address the comments. Obtain Decision 

Maker approval on the approach to responding to peer reviewer comments. Clearly 

identify any peer review comments for the Decision Maker that will not be addressed in 
 Peer Review Handbook Page 26 

the agreed upon approach. (See Section 2.5.3 for specific contents of the peer review 

record): 

1) Make the peer review report for influential scientific information and highly 

influential scientific assessment publicly available; 

2) For highly influential scientific assessments, develop the Agency response to the 

peer review and make it publicly available. 

d) Notify the Peer Review Coordinator that the peer review is completed for the annual 

report; 

e)	 Archive the peer review record in a manner consistent with the organization’s archiving 

procedures. 

When a contractor is used to conduct a peer review, some of the above responsibilities are 

assumed by the contractor (see Section 3.6). In addition, when peer reviews are completed 

through a Federal Advisory Committee, some of the above responsibilities are assumed by the 

Designated Federal Officer (DFO). 

1.5.6 Who are the Peer Reviewers? 

Peer reviewers are individuals who have technical expertise in the subject matter of the work 

product undergoing peer review. Peer reviewers can come from EPA, another Federal agency, 

or from outside of the Federal government. 

1.5.7 What are the Responsibilities of Peer Reviewers? 

Peer reviewers need to be willing participants in the peer review process – they should agree to 

read all materials, participate fully, act ethically, and protect confidential information that arises. 

Peer reviewers should maintain the confidentiality of the product, perform the review in a timely 

manner, and be unbiased and objective. Peer Reviewers should not be given any products or 

data that may reveal confidential business information (CBI) unless the peer reviewer is internal 

to EPA and has the appropriate clearance (see Section 3.4.7). 

1.5.8 What is an Independent Peer Reviewer? 

An independent peer reviewer is an expert who was not associated with the generation of the 

specific work product either directly by substantial contribution to its development or indirectly 

by significant consultation during the development of 

the specific product. The independent peer reviewer, 

thus, is expected to be objective (See Sections 1.2.6 

and 1.2.7 for further information). 

The quality of the peer review is 

dependent on the competence and 

independence of the reviewers. 
 Peer Review Handbook Page 27 

Independence is freedom from institutional or ideological bias regarding the issues under review 

and is necessary for objective, fair, and responsible evaluation of the work product. If a selected 

reviewer has a particular scientific or technical perspective, it may be desirable to balance the 

review with peer reviewers with other perspectives. In addition to being independent, ideally, 

peer reviewers should be free of real or perceived conflicts-of-interest. Conflict of interest is 

different from independence and is discussed in Section 3.4.5 and 3.4.6. If there are potential 

conflicts of interest (real or perceived), they should be fully understood and addressed by the 

peer review leader (or other appropriate official) to ensure a credible peer review. 

1.5.9 When does an Agency Internal Peer Reviewer Qualify as Independent? 

An Agency independent internal peer reviewer is one who comes from a different organizational 

unit than the one where the review question or document originates. A different organizational 

unit usually denotes, at minimum, a different Office (i.e., above Division level in programs; 

above Branch level in Regions) within the organization. In particular, a reviewer should not 

come from within the chain of command, either upward or downward. Agency staff may not 

serve as peer reviewers for the final peer review of a highly influential scientific assessment 

unless a specific exception is granted under Section III.3.c of the OMB Bulletin. 

1.5.10 What is a Peer Review Panel? 

A peer review panel can range from a few individuals to ten or more, depending on the issue 

being investigated, the time available and any limitations on resources. Individuals who serve as 

peer reviewers should have appropriate scientific and technical expertise such that the review 

panel covers the broad spectrum of expertise necessary to address the issues/questions presented 

in the charge. 

1.5.11 What is a Subject Matter Expert? 

A subject matter expert is one who has specific scientific and technical expertise in the matter 

under review. The importance of scientific and technical expertise in the subject matter is 

obvious, however, knowledge or just “knowing” about the subject area is not equivalent to 

expertise in the subject matter. For Agency decisions, a multi-disciplinary group of experts 

corresponding to the disciplines that contribute to complex Agency decisions is often necessary 

for a full and complete peer review. For example, a risk assessment that relies on both animal 

and human data usually would call for experts in both areas for a complete review. For 

economic analyses, experts from the economic disciplines are necessary. 

1.5.12 What is the Role of the Science Policy Council (SPC)? 

According to the Peer Review Policy statement: “The Science Policy Council is responsible for 

overseeing Agency-wide implementation of this policy, including: promoting consistent 
 Peer Review Handbook Page 28 

interpretation; assessing Agency-wide progress; developing recommendations for revisions of 

the policy as necessary; and issuing the Peer Review Handbook, which provides additional 

information and procedures on implementing this policy.” The SPC meets its responsibilities 

through coordination with the Peer Review Coordinators, the Peer Review Advisory Group 

(PRAG) and the Office of Research and Development (ORD). 

The SPC, PRAG and ORD are not responsible for identifying specific products for peer review 

or determining the level of review or mechanism for that review; those functions are the 

responsibility of management within each Office or Region. 

1.5.13 What is the Role of the Peer Review Advisory Group (PRAG)? 

The Science Policy Council has created the PRAG to assist in the implementation of the 

Agency's Peer Review Policy. The primary role of the PRAG is to provide interpretation of the 

policy and to assist the SPC and Agency Offices and Regions in preparing updates to the Peer 

Review Handbook. 

1.5.14 What is the Role of the Office of Research and Development (ORD)? 

The Deputy Administrator has designated the Office of Research and Development (ORD) to 

provide oversight to the Program Offices and Regions in the collection and review of 

information that is contained in the Science Inventory (for detailed information, see Section 1.4). 
 

 

 Peer Review Handbook Page 29 

2. Planning A Peer Review 

2.1 Overview Statement 

Planning a peer review is a critical first step to ensuring a successful peer review of a work 

product. The initial step is to determine whether your work product should be peer reviewed. 

Once you have determined that a peer review will be conducted, the Decision Maker and Peer 

Review Leader need to plan an appropriate review. This includes the determination of resources 

(budget and personnel), schedule for the completion of the peer review, mechanism for peer 

review, development of the peer review record, and, finally, the selection of peer reviewers. 

2.2 Determining Which Work Products to Peer Review 

2.2.1 What are Scientific and/or Technical Work Products? 

The first step in determining which work products should be peer reviewed is to identify 

products that are scientific and/or technical in nature. The term scientific and/or technical work 

products is generally consistent with the term “scientific information” in the OMB Bulletin10. 

Scientific and/or technical work products are used to support a research agenda, regulatory 

program, policy position or other Agency position or action. Scientific and/or technical work 

products include economic and social science work products. Categories of work products 

include, for example: risk assessments, technical studies and guidance, analytical methods, 

scientific database designs, technical models, technical protocols, statistical survey/studies, 

technical background materials, technical guidance (except for guidance providing policy 

decisions), research plans, and research strategies (re journal articles, see section 2.3.4). 

Products that would not be considered scientific and/or technical work products can include 

those: that address procedural matters (e.g., planning, reporting, coordination, notification); that 

are primarily policy statements (e.g., relocation policy); that are conference proceedings (unless 

the proceedings are used as the scientific basis for an Agency action or decision); or that are 

decision documents, e.g., Environmental Impact Statement (EIS), Record of Decision (ROD), or 

Economic Analysis (EA). For any of these, the decision document itself is not subject to the 

Peer Review Policy, but its underlying scientific and/or technical models, data, and/or work 

products upon which these documents are based are candidates for peer review. In addition, the 

following Agency documents are not considered scientific and/or technical work products under 

the Peer Review Policy: strategic plans, analytic blueprints, and goals documents. 

10OMB defines “scientific information” as “factual inputs, data, models, analyses, technical information, or 

scientific assessments based on the behavioral and social sciences, public health and medical sciences, life and earth 

sciences, engineering, or physical sciences. . .” (OMB Bulletin, Section I.5.) 
 

 

 

 Peer Review Handbook Page 30 

Making final determinations concerning which work products are scientific and/or technical is a 

responsibility of the Decision Maker (See Section 1.5.3). All scientific and/or technical work 

products should be entered into the peer review component of the Science Inventory. 

2.2.2 What Scientific and/or Technical Work Products Need Peer Review? 

The principle underlying the Peer Review Policy is 

that all influential scientific and technical work 

products used in decision making will be peer 

reviewed. The process for identifying which 

products are “influential” and then determining the 

mechanism of review will take into account various 

criteria and the circumstances surrounding the use of 

the work product. To maintain flexibility, the Decision Maker(s) for peer review should consider 

the full field of possible work products to identify those additional products that might still 

warrant peer review as well as the full spectrum of peer review mechanisms for each product. 

OMB defines influential to mean information that an agency reasonably can determine will have 

or does have a clear and substantial impact on important public policies or private sector 

decisions. EPA has provided further guidance on making determinations with respect to whether 

a document is “influential” in its Information Quality Guidelines.11 Once a product has been 

identified as being influential scientific information, the Decision Maker should determine 

whether it meets OMB’s definition of a “highly influential scientific assessment” (see Section 

2.2.4). The OMB Bulletin calls for additional peer review procedures for highly influential 

scientific assessments. 

When in doubt about whether a 

work product merits peer review, 

decide to peer review it. 

2.2.3 How Does One Determine Whether a Scientific and/or Technical Work 

Product is Influential Scientific Information? 

Generally, determinations whether a scientific and/or technical work product is “influential” will 

occur on a case-by-case basis. The continuum of work products covers the range from the 

obviously influential, which clearly need peer review, to those products which clearly are not 

influential and don’t need peer review. The majority of EPA’s work products fall in-between 

those two ends of the continuum and should be closely evaluated and assessed with respect to 

certain factors (see below). The Decision Maker should make the judgment as to whether a work 

product is influential scientific information. There is no easy, single “yes/no” test that applies to 

the whole continuum of work products for determining whether a work product is influential 

11Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity, of Information 

Disseminated by the Environmental Protection Agency, EPA/260R-02-008 (October 2002), 

(http://www.epa.gov/quality/informationguidelines) 
 Peer Review Handbook Page 31 

scientific information. A useful rule of thumb is that if there is any doubt about whether a work 

product needs peer review, go ahead and subject it to peer review. 

Generally, scientific and/or technical work products that are used to support a regulatory 

program or policy position and that meet one or more of the following factors would be 

considered to be influential scientific information: 

a) Establishes a significant precedent, model, or methodology; 

b) Likely to have an annual effect on the economy of $100 million or more, or adversely 

affect in a material way the economy, a sector of the economy, productivity, competition, 

jobs, the environment, public health or safety, or State, Tribal, or Local governments or 

communities; 

c) Addresses significant controversial issues; 

d) Focuses on significant emerging issues; 

e) Has significant cross-Agency/interagency implications; 

f) Involves a significant investment of Agency resources; 

g) Considers an innovative approach for a previously defined problem/process/methodology; 

h) Satisfies a statutory or other legal mandate for peer review. 

Historically, EPA has defined products subject to peer review in terms of having “major impact.” 

“Major impact” means that the work product may have applicability to a broad spectrum of 

regulated entities and other stakeholders, or that it will have narrower applicability, but with 

significant consequences on a smaller geographic or practical scale. The Agency developed the 

previously listed factors to consider when determining if a product had a “major impact.” The 

Agency has also linked its use of the term "influential" to the term “major” in its Information 

Quality Guidelines. The OMB Peer Review Bulletin provides similar factors to use in 

determining if a scientific assessment is highly influential, thereby creating some overlap 

between the two definitions. Scientific assessments that meet the criteria of both influential and 

highly influential should be considered as highly influential. 

Typically, an influential scientific and/or technical work product that has a reasonable likelihood 

of supporting a regulatory decision or policy/guidance of major impact may also have a clear and 

substantial impact on private sector decisions. The scientific and/or technical work products that 

underlie many of the Agency’s major rulemakings, policy and guidance documents of general 

applicability would be designated “influential” under this construct because of their far-reaching 

impacts. 
 

 

 Peer Review Handbook Page 32 

The novelty or controversy associated with the work product may determine whether it is 

influential scientific information. Influential scientific information may be novel or innovative, 

precedential, controversial, or emerging (“cutting edge”). An application of an existing, 

adequately peer-reviewed methodology or model to a situation that departs significantly from the 

situation it was originally designed to address may make peer review appropriate. Similarly, a 

modification of an existing, adequately peer-reviewed methodology or model that departs 

significantly from its original approach may also make peer review appropriate. Determining 

what constitutes a “significant departure” as used in this Section is the responsibility of the 

Decision Maker. 

In summary, an influential scientific or technical work product has a major impact, involves 

precedential, novel, and/or controversial issues, or the Agency has a legal and/or statutory 

obligation to conduct a peer review. 

2.2.4 How Does One Determine Whether Influential Scientific Information is a 

Highly Influential Scientific Assessment? 

Once a scientific and/or technical work product has been categorized as “influential scientific 

information,” the Decision Maker should determine whether the product meets OMB’s definition 

of a highly influential scientific assessment (see also Figure 4). As with the categorization of a 

work product as influential scientific information, the decision whether or not to elevate a work 

product to the highly influential category occurs on a case-by-case basis and is largely based on 

evaluation of the same factors as identified above. 

Note that “highly influential scientific 

assessments” differ from “influential 

scientific information” both in terms of 

their degree of influence and in 

substance. OMB defines highly 

influential scientific assessments as 

influential scientific information that 

“the agency or the Administrator 

determines to be a scientific assessment 

that: 

a) Could have a potential impact of 

more than $500 million in any 

year, or 

 

b) Is novel, controversial, or 

precedent-setting or has 

significant interagency interest.” 

All work products EPA may 

generate or use 

Influential Scientific Information 

Highly Influential Scientific 

Assessments 

Figure 4-EPA Work Products
 

This diagram depicts the relation between all EPA work 

products and those considered Influential Scientific 

Information or Highly Influential Scientific Assessments 
 

 

 

 

 

 

 

 

 

 

 Peer Review Handbook Page 33 

OMB defines a scientific assessment as “an evaluation of a body of scientific or technical 

knowledge, which typically synthesizes multiple factual inputs, data, models, assumptions, 

and/or applies best professional judgment to bridge uncertainties in the available information”12. 

The OMB Bulletin includes the following examples of assessments that could be considered 

highly influential scientific assessments: 

a) State-of-science reports; 

b) Technology assessments; 

c) Weight-of-evidence analyses; 

d) Meta-analyses; 

e) Health, safety, or ecological risk assessments; 

f) Toxicological characterizations of substances; 

g) Integrated assessment models; 

h) Hazard determinations; 

i) Exposure assessments. 

OMB’s definition of a highly influential scientific assessment overlaps significantly with the 

factors you should consider when assessing whether a work product is influential scientific 

information. The decision to categorize a product as a highly influential scientific assessment 

first depends on determining whether the influential scientific information is a “scientific 

assessment,” and then on assessing the likely degree of the product’s impact. 

The more far-reaching or significant the impacts of a scientific assessment are, the more 

appropriate it is to categorize the product as a highly influential scientific assessment. The 

determination as to whether the degree of novelty, precedent, and controversy associated with a 

particular assessment warrants designation as a highly influential scientific assessment should be 

made by the Decision Maker. If the influential scientific assessment involves significant issues 

that are truly “cutting edge,” it may be appropriate to designate it as a highly influential scientific 

assessment. The greater the “significant departure,” as discussed above, the more appropriate it 

is to include the product as a highly influential scientific assessment. These are subjective 

determinations that should be made by the Decision Maker. 

12OMB Bulletin, Section I.7. 
 Peer Review Handbook Page 34 

OMB also directs agencies to consider the degree of potential economic impact of a scientific 

assessment in deciding whether it should be categorized as a highly influential scientific 

assessment. As explained above, OMB defines highly influential scientific assessments as those 

scientific assessments that may have an impact of more than $500 million in any year. 

2.2.5 What Economic Work Products Need Peer Review? 

Economic analyses (EA) and reports are considered scientific and/or technical work products, 

and as such it may be appropriate to peer review them. If those work products are ISIs, then they 

should be peer reviewed if they have not already been subject to adequate peer review (See 

section 2.2.3). The following economic work products. should generally be peer reviewed. 

a) Internal Agency guidance for conducting economic and financial analysis; 

b)	 Economic and financial methodologies that will serve as a principal method or protocol 

used to conduct economic analyses within a program; 

c) Unique or novel applications of existing economic and financial methodologies, 

particularly those that are recognized to be outside of mainstream economic practices; 

d)	 Broad-scale economic assessments of regulatory programs, such as those required by 

Congressional mandates (e.g., the Clean Air Act reports to Congress on benefits and 

costs); 

e) Stated preference (e.g., contingent valuation) and revealed preference surveys (e.g., 

recreational travel cost surveys) developed to assist in the economic analysis of a 

regulation or program; 

f)	 National surveys of costs and expenditures for environmental protection (e.g., financial 

needs surveys, pollution abatement expenditures surveys); 

g)	 Economic research plans developed to assess and advance the state-of-science in 

economic theory, methodologies or modeling (in particular, the technical feasibility of 

the plan’s components); 

h) Meta-analyses (i.e., re-analyses of existing published literature and supporting data on the 

measurement of economic benefits, costs and impacts); 

i)	 Data and analytical models underlying economic analyses, particularly those supporting 

economically significant rules, if the models and corresponding use of the data have not 

been previously subject to adequate peer review. 
 Peer Review Handbook Page 35 

If these types of economic work products support a highly influential scientific assessment, you 

should conduct an external peer review. External peer reviews can be provided by the Science 

Advisory Board’s Environmental Economics Advisory Committee, other appropriate outside 

organizations, or individuals that have expertise in the technical economic issues raised in the 

economic work product. 

2.2.6 Should Economic Work Products Prepared in Support of Regulations that 

are Classified as "Major" or "Economically Significant" be Peer Reviewed? 

Generally, if an EA applies accepted, previously peer-reviewed methods in a straightforward 

manner, it would not undergo an additional peer review. EAs prepared to support “major” or 

“economically significant” regulations (these terms are defined at the end of this Section) 

typically do not utilize innovative or untried economic methods. It is unnecessary to conduct 

peer reviews of straightforward applications or transfers of accepted, previously peer-reviewed 

economic methods or analyses. The procedures used to transfer or adapt an economic work 

product are generally established by separate economic guidance documents which have been 

peer reviewed. Therefore, EAs that are developed using these procedures do not normally 

undergo an additional peer review, even those prepared in support of “major” and “economically 

significant” rules. To the extent that the underlying data and models are considered “influential 

scientific information,” they are subject to peer review, including questions regarding the 

applicability of the specific data set or model for the use. 

Even when peer review is not needed, additional peer input can be beneficial in the development 

of economic work products for “major” and “economically significant” rules. At present, some 

peer input of these analyses is already likely to be included as part of the regulatory development 

process, including input received from other EPA offices represented on the workgroup for the 

rule, from the Agency’s Regulatory Steering Committee, and from the public as part of the 

public comment process for the rule. But there may be added benefit to employing additional 

peer input procedures, such as actively soliciting input from economists in other Federal 

Agencies on the quality and completeness of the economic analysis. 

Under Section 3(f)(1) of Executive Order 12866, “economically significant” rules are those that 

may have an annual effect on the economy of $100 million or more or adversely affect in a 

material way the economy, a sector of the economy, productivity, competition, jobs, the 

environment, public health or safety, or State, Local, or Tribal governments or communities. 

The term “major,” as defined in the Congressional Review Act, means a rule that has resulted in 

or is likely to result in: an annual effect on the economy of $100 million or more; a major 

increase in costs or prices for consumers, individual industries, Federal, State, or Local 

government agencies, or geographic regions; or significant adverse effects on competition, 

employment, investment, productivity, innovation, or on the ability of United States-based 

enterprises to compete with foreign-based enterprises in domestic and export markets. 
 

 

 Peer Review Handbook Page 36 

2.2.7 What Other Economic Work Products Might Benefit from Peer Review? 

There may be other economic work products not covered in the preceding sections for which 

peer review might be useful. Examples of such work products are presented below; however, by 

listing these examples, there is no intent to establish a presumption of peer review for these work 

products. 

a) Analyses measuring the economic impacts and effectiveness of adopting market-based or 

economic incentives as regulatory management instruments; 

b) The technical analyses supporting economic policies established under other government 

organizations (e.g., economic models used to study transportation, economic 

development, and international trade policies). 

Most of these types of economic work products do not exhibit the degree of complexity, or 

establish an innovative or untried approach, that would warrant a peer review. However, other 

factors, such as the potential significance of the analysis for cross-Agency or interagency 

practices, or the significance of the issues addressed, may make peer review desirable. 

2.2.8 What Other Social Science Work Products Need Peer Review? 

Typically, a social science work product is one that includes empirical, logic-based approaches 

to answer technical questions about human motivation, human behavior, social interactions, and 

social processes, which are relevant to the environmental issues being addressed. The term 

“behavior” includes overt actions; underlying psychological processes such as cognition, 

emotion, temperament and motivation; and bio-behavioral interactions. The term “social” 

includes sociocultural, socioeconomic, and sociodemographic status; biosocial interactions; and 

the various levels of social context from small groups to complex cultural systems. Examples of 

social science work products include analyses and/or evaluations related to such topics as 

pollution prevention, risk communication, environmental information, environmental justice, 

quality of life, decision-making, and public participation. 

The following social science work products should normally undergo peer review: 

a) Internal Agency guidance for conducting social impact assessments and other 

community cultural assessments related to different environmental protection approaches 

such as community-based watershed protection (heretofore referred to as social 

assessments); 

b)	 New social science methodologies that will serve as a principal method or protocol used 

to conduct social assessments; 
 Peer Review Handbook Page 37 

c) Unique or novel applications of existing social science methods such as surveys, focus 

groups, interviews, network analyses, comparative analyses, and content analyses; 

d) New national surveys of values, perceptions and preferences related to environmental 

protection; 

e) Innovative research or analyses that address the human dimensions of environmental 

protection or environmental change in terms of social trends, future predictions and/or 

behavioral generalizations; 

f)	 Social science research plans developed to assess and advance the state-of-science in 

social science theory, methodologies or modeling (in particular, the technical feasibility 

of the plan’s components). 

2.2.9 How Should Peer Review be Handled for Products Developed Under an 

Interagency Agreement (IAG)? 

Under an interagency agreement (IAG), EPA provides funds to another agency for that agency to 

use for a specific purpose. The receiving agency's guidance for peer review may likely be 

different from EPA Peer Review Policy, although the OMB Bulletin establishes some minimal 

common guidance for the Federal government. Regardless, if EPA plans to use any work 

products from that agreement, EPA should decide whether those documents need review under 

EPA Peer Review Policy. 

2.2.10 Should Products from Contracts, Grants, and Cooperative Agreements 

Receive Peer Review? 

If there is a scientific and/or technical work product resulting from a grant, contract, or 

cooperative agreement and it is considered influential scientific information or a highly 

influential scientific assessment and it will likely be used in Agency decision-making, the work 

product needs peer review. Since it would probably result in a perceived, if not real, conflict of 

interest, the group that is generating the work product usually cannot conduct or perform the peer 

review of its own work product. Exceptions may be made in certain instances for organizations 

that have adequate and well established recognized procedures for peer review, such as the 

National Academy of Sciences (NAS). In practice, the Agency may need to peer review the 

product on its own, or arrange with an independent third group (e.g., via another extramural 

vehicle) to conduct the peer review. The Agency should not use scientific and/or technical work 

products from contracts, grants, or cooperative agreements to support decision making unless the 

work products are peer reviewed for both scientific and technical rigor and applicability to the 

specific use to be made of the product. 

Be aware that contracts are very different from grants and cooperative agreements. Please note 

that there are important legal restrictions on the direct use of work products developed under 
 

 

 Peer Review Handbook Page 38 

grants and cooperative agreements in the Agency’s decision-making process (see 

http://intranet.epa.gov/ogd/) for additional information). 

2.2.11 How Does Peer Review Apply to Products Generated Through EPA Grants 

or Cooperative Agreements? 

It may be desirable to peer review scientific and/or technical work products that are generated 

through EPA grants or cooperative agreements, but special considerations apply. 

EPA may legally provide financial assistance for research that is intended to stimulate or support 

development of scientific knowledge that is not primarily for EPA’s direct use or benefit. The 

resulting work products might be widely disseminated either through publication in scientific 

journals or through other means, as opposed to a report tailored to EPA’s specific needs and 

requirements. EPA can consider these work products just as it can review other published 

scientific works when formulating its programs and policies. Further, EPA retains a royalty free, 

nonexclusive and irrevocable right to use the work products for Federal purposes, even if the 

recipient has copyrighted the material (see 40 CFR 30.36(a)). 

EPA may determine that the recipient’s work product is influential scientific information 

because: 1) it will be used to support an EPA program or policy position, and 2) it meets one or 

more of the seven factors outlined in Section 2.2.3. In this situation, the work product should be 

peer reviewed. 

The following are options for peer reviewing the product: 

a) EPA can have the product peer reviewed with the participation of the assistance 

agreement recipient/author. In this case, EPA could arrange for an independent peer 

review of the product itself, or may contract with a third party to conduct the review. 

EPA would also enter into a contract with the author (formerly the recipient), which 

would task the author to prepare a response to the peer reviewers’ comments, and to 

revise or prepare an addendum to the product in response to peer reviewer comments as 

determined appropriate by EPA. 

A caveat to this approach is that it may be difficult to get the recipient/author to agree to 

allow EPA to determine how to revise the product in response to the peer review 

comments. 

b) EPA can have the product peer reviewed without the participation of the recipient/author. 

EPA could arrange for the peer review itself, or could contract with a third party to 

conduct the review. In this case, however, the work product would not be revised to 

incorporate the peer review comments. Instead, EPA would receive the comments and 

prepare a statement that documents EPA’s own response to the comments. The EPA 

Decision Maker who is using the work product to support an EPA program or policy 
 Peer Review Handbook Page 39 

decision should be provided information on both the conclusions of the recipient’s work 

product and EPA’s own conclusions from the peer review. 

c) Recipients can get their products peer reviewed on their own. Recipients may determine 

on their own that peer review would benefit the credibility of their product. Provided 

EPA agrees that a peer review would further the public purpose of the assistance 

agreement, EPA may include funds for the peer review in the agreement (see 2.2.10 for 

additional information). Alternatively, the recipient may make arrangements for, and 

fund, an independent peer review of their product. In either case, EPA would need to 

evaluate whether the peer review process undertaken by the recipient was acceptable for 

the purposes for which EPA was planning to use the work product. EPA may accept the 

peer review if it determines that it is of appropriate quality and that EPA could defend the 

peer review as if it were conducted by EPA itself. 

Under options a) or b), issues may arise over obtaining access for peer reviewers to the raw data 

used by the recipient to generate the work product. Under 40 CFR 30.36(c)(2), EPA has a right 

to obtain raw data produced by a non-profit organization or university under an assistance 

agreement, even where the agreement does not specifically provide for this access. Nevertheless, 

it may be prudent to include a specific term in the assistance agreement clarifying this point to 

avoid misunderstandings. EPA may have to pay for obtaining access to the data if its transmittal 

imposes additional costs on the recipient. Assuming our use of the data is incidental to the 

principal purpose of the agreement, we also have a specific right to authorize peer reviewers to 

use the data for Federal purposes under 40 CFR 30.36(c)(2). 

EPA cannot obtain access to the raw data if EPA specifically bargains away this right in the 

assistance agreement. 

Again, consult the Office of General Counsel (OGC) for help in drafting appropriate language 

for your assistance agreement. 

2.2.12 Can the Recipient of a Grant or Cooperative Agreement Use Agreement 

Funds to Pay Peer Reviewers of their Work Products? 

As noted in Section 2.2.11, provided EPA agrees that a peer review would further the public 

purpose of the assistance agreement, EPA may include funds for the peer review in the 

agreement. A payment to peer reviewers in exchange for their review of a scientific and/or 

technical work product is allowable as a fee for professional services under assistance 

agreements. (To accurately characterize this cost, however, it is important that the payment be 

referred to as a fee, rather than an honorarium.) See OMB Circular A-21, Section J, item 32, 

Professional Services Cost (Educational Institutions), OMB Circular A-122, Attachment B, Item 

39, Professional Services Costs (Non-Profit Organizations), and OMB Circular A-87, 

Attachment B, Item 33, Professional Services Costs (State, Local and Indian Tribal 

Governments). See http://www.whitehouse.gov/OMB/circulars/ for further details. 
 Peer Review Handbook Page 40 

2.2.13 Do Products Generated Under EPA Grants or Cooperative Agreements Need 

to be Reported in the Science Inventory 

As a matter of practice, EPA organizations are encouraged to include scientific work being done 

under grants and cooperative agreements in the Science Inventory so that Agency staff are aware 

of the ongoing work. If a grant or cooperative agreement product is likely to be used in Agency 

decision making (assuming this use is incidental to the principal purpose of the agreement), it 

should generally be considered a candidate for peer review and noted as such in the Science 

Inventory. 

2.2.14 Should Site-Specific Decisions be Subject to Peer Review? 

A site-specific decision itself is not subject to peer review and does not need peer review as 

described in the Peer Review Policy. However, if a site-specific decision is supported by 

influential scientific information or a highly influential scientific assessment, that work product 

should be peer reviewed. Generally speaking, you should closely examine how the underlying 

scientific and/or technical work product is adapted to the site-specific circumstances. 

2.2.15 Should NEPA Products be Subject to Peer Review? 

Although an Environmental Impact Statement (EIS) prepared under the requirements of the 

National Environmental Policy Act (NEPA) receives extensive review through the “scoping” and 

interagency review processes that are part of NEPA, this is not usually considered peer review. 

If the underlying scientific and/or technical data, models, analyses, or work products are 

influential scientific information or a highly influential scientific assessment, then the relevant 

portions should be peer reviewed. 

In general, the Agency’s role in preparing the NEPA document may suggest the extent of review 

the document should get (See sections 2.2.3 and 2.2.4). If EPA is developing the document as 

part of an EPA action/decision (EPA is the Lead Agency under NEPA), and the document is 

influential scientific information or a highly influential scientific assessment, then it should 

receive independent peer review. If not, (the document is not influential, will have little impact, 

is non-controversial, etc.), then peer input might be appropriate. 

On the other hand, if EPA is reviewing an EIS from another agency (EPA is not the Lead 

Agency under NEPA), it is likely that we are reviewing the EIS for conflicts with EPA policy 

and general environmental concerns. In such a case, EPA should ask if the underlying scientific 

and/or technical work product that supports the EIS has been peer reviewed to avoid concerns 

about the full credibility and soundness of the EIS based on the science and technical support. 

EPA should work with the other organization/agency to ensure that scientific and/or technical 

work products receive peer review adequate for EPA purposes. 
 Peer Review Handbook Page 41 

2.2.16 Should Environmental Regulatory Models be Peer Reviewed? 

Generally, yes. Specific guidelines for the peer review of environmental regulatory models have 

been published by the Agency. These can be found on the EPA web site under the Science 

Policy Council home page (http://www.epa.gov/osa/spc/). In 2000, the Science Policy Council 

established the Agency’s Council for Regulatory Environmental Modeling (CREM) which 

promotes consistency and consensus among environmental modelers and users. The CREM is a 

good resource for you to contact for specific questions regarding peer review of environmental 

regulatory models (http://cfpub.epa.gov/crem/). 

2.2.17 Should Another Organization’s Work Products That Have Been Submitted 

to EPA for Use in Decision Making be Peer Reviewed? 

Any scientific and/or technical work product that is used in Agency decision making and is 

considered influential scientific information or a highly influential scientific assessment becomes 

a candidate for peer review regardless of whether the work product is produced by the Agency or 

another organization. Therefore, all work products important to EPA environmental decision 

making that are independently generated by other organizations (e.g., other Federal agencies, 

interagency groups, State and Tribal bodies, environmental groups, industry, educational 

institutions, international bodies) need to be considered as candidates for peer review. 

If possible, when EPA knows that a work product is being generated by another organization and 

may be of interest to EPA for future use, the appropriate EPA office(s) should work with that 

organization, and others, as appropriate (e.g., the states), to promote the use of peer review. For 

example, the Office of International Affairs (OIA), as well as the impacted Program or Regional 

Office(s), should be included when international products are being considered for EPA use. 

It is hoped that if the other organization has the work product independently peer reviewed, the 

peer review will meet the intent of the Agency’s Peer Review Policy and EPA’s proposed use of 

the product (i.e., the peer review is basically equivalent to what EPA would do). Agency staff 

from the appropriate office(s) should examine closely the particulars of the peer review to ensure 

independence and a conscious effort to incorporate the peer reviewers’ comments into the final 

work product. If there are perceived, or real, conflicts of interest, this may preclude the use of 

that peer review and, in those instances, another peer review would be needed. See Section 3.4.9 

for considerations of when an outside party conducts and/or funds peer review of their own work 

product and submits it to the Agency. 

If the outside organization does not have the work product peer reviewed and EPA decides it 

should be peer reviewed, the appropriate EPA office(s) should ensure peer review of that work 

product occurs prior to the Agency’s use of the work product in decision making. Peer review 

can be accomplished by asking the outside organization to do so, or, if it declines, EPA may 

conduct or arrange for the peer review. If EPA is conducting or arranging the peer review, the 

product should be entered in the Science Inventory. 
 Peer Review Handbook Page 42 

2.2.18 Can Work Products That are Not Determined to be Influential Scientific 

Information or Highly Influential Scientific Assessments Still be Peer 

Reviewed? 

Yes. Scientific and/or technical work products that are not categorized as influential scientific 

information or a highly influential scientific assessment may nonetheless be candidates for peer 

review. For example, a project manager may decide to use peer review because of particular 

program needs and goals. Peer review may also be warranted because it adds substantial value 

to the work product. 

2.3 Determining Which Work Products Do Not Receive Peer Review 

2.3.1 What Information Is Exempt from the OMB Bulletin Provisions? 

The following information does not need to be peer reviewed, even if it might be considered a 

highly influential scientific assessment or influential scientific information: 

a)	 Information related to national security, foreign affairs, or negotiations involving 

international trade or treaties where peer review would interfere with the need for secrecy 

or promptness; 

b)	 Information disseminated in the course of an individual adjudication or permit 

proceeding (including a registration, approval, licensing, site-specific determination), 

unless peer review is practical and appropriate and the information is scientifically or 

technically novel or likely to have precedent-setting influence on future adjudications 

and/or permit proceeding; 

c)	 Information involving a health or safety dissemination that is time-sensitive; 

d) A regulatory impact analysis or regulatory flexibility analysis subject to interagency 

review under Executive Order 12866, except for underlying data and analytical models 

used; 

e)	 Routine statistical information (e.g., periodic demographic and economic statistics) and 

analyses of these data to compute standard indicators and trends; 

f)	 Accounting, budget, actuarial, and financial information, and; 

g)	 Information disseminated in connection with routine rules that materially alter 

entitlements, grants, user fees, or loan programs, or the rights and obligations of 

recipients thereof. 
 Peer Review Handbook Page 43 

2.3.2 Are There Other Circumstances Where Peer Review Is Not Necessary? 

There may be circumstances in which it is not necessary for a work product to undergo peer 

review. For example, peer review is not generally conducted: 

a) For work that has been previously reviewed in a manner consistent with OMB’s Bulletin 

and this Handbook. For example, a cancer risk assessment methodology or an exposure 

modeling technique that was the subject of earlier peer review of appropriate technical 

merit would not generally undergo additional peer review, even if the product supported a 

significant Agency decision; 

b) If an application of an adequately peer-reviewed work product does not depart 

significantly from its scientific or technical approach; 

c) When the scientific and/or technical methodologies or information being used are 

commonly accepted in the field of expertise, e.g., many products supporting Control 

Techniques Guidelines (CTGs) and Effluent Limitation Guidelines (ELGs). Such use 

should provide the appropriate documentation to support the commonly held view. 

2.3.3 For Information That Is Not Exempt, Are There Circumstances Where the 

Peer Review Provisions of the OMB Bulletin Can Be Waived or Deferred? 

The Administrator may waive or defer the peer review provisions of the OMB Bulletin for 

influential scientific information or highly influential scientific assessments if there is a 

compelling rationale for the waiver or deferral. The use of waivers is expected to be limited to 

unusual and compelling situations not otherwise covered by the exemptions, such as situations 

where unavoidable legal deadlines prevent full implementation of the Bulletin’s peer review 

provisions. According to the Bulletin, deadlines found in consent decrees will not ordinarily 

warrant waiver of the provisions because those deadlines should be negotiated to permit time for 

conducting a peer review. Deferral of some or all of the peer review provisions may be an 

appropriate way to accommodate immovable deadlines. If any of the OMB Bulletin provisions 

are deferred, peer review should be conducted as soon as practicable thereafter. Deferrals of 

peer review of influential scientific information and highly influential scientific assessments 

should be approved by the Administrator. 

If peer review of a work product is not planned, an explanation should be included in the 

database entry for that work product in the Science Inventory. 

2.3.4 Does Publication in a Refereed Scientific Journal Mean That Adequate Peer 

Review Has Been Performed? 

Scientific papers (articles) that are peer reviewed by a credible refereed scientific journal may 

not generally need to undergo further peer review. However, you should expect that there may 
 Peer Review Handbook Page 44 

be cases where EPA determines that a more rigorous or transparent review process is necessary, 

(e.g., if a particular journal review process did not address questions that EPA determines should 

be addressed before using or disseminating the information, such as the extent of uncertainty 

inherent in the findings, the sensitivity to model choice, or the applicability of the data presented 

for use in the Agency’s models). Also see Section 2.4.4 for additional information. 

2.3.5 Do Voluntary Consensus Standards Undergo Peer Review? 

Generally, no. The National Technology Transfer and Advancement Act of 1995 (NTTAA) 

directs EPA to use available voluntary consensus standards in its regulatory activities unless to 

do so would be inconsistent with applicable law or otherwise impractical. For purposes of the 

NTTAA, voluntary consensus standards are defined as technical standards (e.g., materials 

specifications, test methods, sampling procedures, and business practices) that are developed or 

adopted by voluntary consensus bodies (such as ISO or ASTM). The general purpose of the 

NTTAA is to reduce private and governmental costs by avoiding having the government 

“reinvent the wheel” in the development of technical standards. Voluntary consensus standards 

would normally not undergo peer review because the underlying process used by issuing 

organizations to develop and approve these standards is generally considered adequate for 

purposes of the Agency’s peer review policy. EPA reserves the right to conduct a peer review if 

it determines that the particular standard it wants to use is not a voluntary consensus standard for 

purposes of the NTTAA. 

2.4 Choosing a Peer Review Mechanism 

2.4.1 How Do You Determine the Appropriate 

Peer Review Mechanism? 

The mechanism of the peer 

review should match the 

importance and complexity of 

the work product. 

During the planning of a peer review, the Decision Maker 

and the Peer Review Leader may consider several mechanisms for the peer review of scientific 

and/or technical work products. These options range from consultations with EPA colleagues 

not involved in developing the product to a large and formal panel of outside subject matter 

experts. The peer review effort might be a focused one-time evaluation, or could encompass 

several examinations over the course of a project. In principle, peer review provides the greatest 

credibility for influential scientific information and highly influential scientific assessments 

when it involves well-qualified external reviewers, is intensive in its examination, and operates 

through a more or less formal and transparent process. As a practical matter, however, time and 

resource considerations in many cases impose limitations on what can be reasonably achieved. 

Arranging for the most appropriate and feasible peer review will involve good judgment 

regarding the extent to which the peer review will improve the credibility of the product as well 

as consideration of substance, time, and resources. Developing a peer review plan that provides 

for appropriate depth, timing, and content is an important matter for early consideration by the 

Decision Maker and Peer Review Leader. Note that use of peer input or public or stakeholder 

involvement does not constitute peer review. 
 Peer Review Handbook Page 45 

While the following information is described in more detail elsewhere, these are important 

considerations in determing a peer review mechanism. The approach best suited to a specific 

work product will depend on the nature of the topic and the intended final product. Generally, 

the more novel or complex the science or technology, the greater the cost implications of the 

impending decision, and the more controversial the issue, then the stronger the indication is for a 

more extensive and involved peer review and for external peer review in particular. Certain 

work products may clearly lend themselves to extensive external peer review; generally these 

will be products with large impacts (e.g., those that support Tier 1 and Tier 2 rulemakings). 

Other work products may not need a large scale external peer review and may utilize a less 

involved, less resource intensive review. The peer review of some products may be better served 

with some form of internal peer review or a combination of internal and external peer review. 

It is important to make the choice of peer review mechanism at the time that the work is planned 

(for products supporting rule makings, at the analytic blueprint stage) so that peer review costs 

and time can be budgeted into the work plan. Essentially, the level of peer review should match 

the impact and complexity of the work product. For example, a rule under development carries 

considerable weight and deserves careful handling and attention; therefore, the supporting work 

product deserves similar care and attention for its peer review. Factors that need to be 

considered include: use of internal vs. external peer reviewers; individual letter reviews vs. panel 

reviews; the number of reviewers; the timing of the peer review. No single peer review 

mechanism is likely to work best in all situations. Some useful guidance includes: 

a) Influential scientific information intended to support the most important decisions, or 

work products that have special importance in their own right, external peer review is the 

approach of choice. Generally, the more complex, novel and/or controversial the 

product, or the higher impact it has, the more the Decision Maker should consider 

implementing a peer review involving external experts; 

b) Highly influential scientific assessments are expected to undergo external peer review. 

When time and resources allow, panels are preferable; 

c) Work products that are less complex, novel, or controversial, or have a lower impact 

might be subject to a less extensive, less resource-intensive review processes; 

d) Group discussion with peer reviewers can be very helpful at some point in the peer 

review process as it allows interaction among peer reviewers with different perspectives 

and expertise. Public panels are more transparent than closed discussions. On the other 

hand, simply soliciting individual comments is easier, faster, and less expensive. 

Individual review is probably more appropriate for peer review at the early stages of a 

product’s development or for products with less impact and complexity; 

e) More reviewers are necessary for more complex projects (more disciplines) and for more 

controversial topics (differences in scientific perspective within a discipline). 
 Peer Review Handbook Page 46 

f) Strict time constraints, such as a court-ordered deadline, can make a less involved or 

formal peer review mechanism imperative. Decision Makers and Peer Review Leaders 

should make maximum efforts to ensure that such a process is systematic and objective. 

2.4.2 What are Some Examples of Internal Peer Review Mechanism? 

a) Individual independent experts from within the Agency (e.g., ORD experts on non-cancer 

effects of lead (Pb) review a draft article on benchmark dose completed by a Program 

Office); 

b)	 An ad hoc panel of independent experts from within the Agency (e.g., an independent 

internal workgroup convened to examine the case for the classification of a chemical as a 

carcinogen); 

c) Technical merit review by scientists in an Agency laboratory (e.g., an initial review of the 

risk assessment for a regional incinerator by Agency scientists). 

2.4.3 What are Some Examples of External Peer Review Mechanisms? 

a) Independent experts from outside the Agency (e.g., a letter review by outside scientists); 

b)	 An ad hoc panel of independent experts outside the Agency (e.g., a group is convened to 

develop a consensus on the carcinogenicity of a particular industrial chemical); 

c) Agency-sponsored peer review workshops (e.g., a review of potential indicators of 

ecosystem damage); 

d) Review by an established Federal Advisory Committee Act mechanism such as the 

Science Advisory Board (SAB), FIFRA Scientific Advisory Panel (SAP), ORD’s Board 

of Scientific Counselors (BOSC), or the Clean Air Scientific Advisory Committee 

(CASAC) - e.g., a review of a criteria document for a particular chemical risk; 

e) Agency-appointed special board or commission (e.g., a review of the risk assessment 

methodology prepared by the Clean Air Act Commission on Risk Assessment). Note: 

The Office of General Counsel should be consulted regarding EPA's authority to 

establish and finance the activities of a commission or board. 

f)	 Review by the NAS (e.g., a review of the state of current knowledge about children’s 

health risks from pesticide exposures). 

There are other bodies that may provide external commentary on Agency work products but 

are not considered peer review mechanisms, such as interagency committees (e.g., a review 
 

 

 Peer Review Handbook Page 47 

of prospective research plans by the Committee on the Environment and Natural Resources 

coordinated by the White House), committees convened by another Federal agency or 

government organization (e.g., a review of the Dioxin Reassessment by the Health and 

Human Services Committee to Coordinate Environmentally Related Programs), and/or 

reviews initiated by non-governmental groups (e.g., a Society of Risk Analysis review of 

cancer guidelines). 

2.4.4 What is the Role of Peer Review by a Refereed Scientific Journal? 

Peer review of journal articles (written by EPA or non-EPA authors) performed by a credible, 

refereed scientific journal contributes to the scientific and technical credibility of the reviewed 

product. Generally, EPA considers peer review by such journals as adequate for reviewing the 

scientific credibility and validity of the findings (or data) in that article, and therefore, a 

satisfactory form of peer review13. However, in some cases, peer review of an Agency work 

product that uses these articles may be conducted (see Section 2.4.5). 

EPA-authored journal articles, whether used in an Agency work product or not, are included in 

the Science Inventory to highlight the extensive work EPA produces in the scientific literature. 

Prior to submitting an article to a journal for peer review, EPA employees are encouraged to 

have the article internally peer reviewed (see Section 2.4.2); such internal peer review is already 

common practice in certain parts of EPA. Articles may also need examination in accordance 

with any organizational clearance procedures, especially when the author is presenting him or 

herself as an EPA employee. For EPA employees, conflict of interest law and policy will also 

apply. 

2.4.5 Do Agency Work Products Become Candidates for Peer Review when PeerReviewed Journal Articles are Used in Support of that Work Product? 

In most instances, Agency work products are candidates for peer review even when supported by 

peer-reviewed journal article(s). Although the use of articles that have been peer reviewed by a 

credible journal strengthens the scientific and technical credibility of any work product in which 

the article(s) appears or is referenced, it does not automatically eliminate the need to consider 

whether the work product itself should be peer reviewed. In most cases, journal peer review may 

not cover issues and concerns that the Agency may want peer reviewed to support an Agency 

action. Under these circumstances, the scientific and/or technical work product in which the 

article(s) appears or is referenced becomes a candidate for peer review. A journal article 

authored by EPA employees should be used in the same manner as an article published by 

anyone else in a credible, well-recognized journal. 

13 Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity, of Information 

Disseminated by the Environmental Protection Agency, EPA/260R-02-008 (October 2002), 

(http://www.epa.gov/quality/informationguidelines) 
 Peer Review Handbook Page 48 

If an Agency work product is based solely on a single article that has received peer review by a 

credible journal (e.g., where a model is suggested for a singular use that fits a specific Agency 

need), peer review of the Agency work product may or may not be necessary depending on how 

closely you apply the findings from the article. If an Agency work product is based on two or 

more articles that have received peer review by a credible journal(s), the Agency work product 

generally should be considered for peer review. Decisions to make (or not make) a work product 

a candidate for peer review should be documented in the peer review record. 

One important factor to remember with regard to the use of articles that have received journal 

peer review concerns the availability of documentation from that peer review. Ideally, EPA 

should maintain a clear, easily accessible record of the peer review to ensure the credibility and 

validity of the peer review (see Section 2.5 for details on the peer review record). However, the 

documentation from a journal peer review may not normally be available to the Agency, so such 

documentation is not expected in the peer review record. 

2.4.6 When and How Often Should Peer Review Occur? 

The Decision Maker and Peer Review Leader have significant discretion in deciding on the 

timing and the frequency of peer review. Options abound, each with merits depending on the 

context and specified peer review objectives. In many situations, a single peer review event, 

beginning when the final draft work product becomes available, is the approach taken. However, 

it is increasingly apparent that peer review performed earlier in the work product development 

stages provides a superior approach for some work products. There may be substantial 

incremental benefit to conducting more than one peer review during the whole process of work 

product development, particularly where it involves complex tasks, has decision branching 

points, or could be expected to produce controversial findings. In addition, early review could be 

beneficial at the stage of research design or data collection planning where the product involves 

extensive primary data collection. The Decision Maker and Peer Review Leader should 

determine when the peer review(s) should occur, considering the type of work product under 

development and at what point in its development process a peer review would be most 

beneficial. 

Other types of work products that could benefit from early, up-front peer review in their 

development are scientific and technical planning products. Examples of such products are 

research proposals, plans, and strategies. Also, while not products per se, ongoing research 

programs can be peer reviewed. 

Remember though, that while more than one peer review can be beneficial, the distinction 

between peer input and peer review should be kept in mind. Experts providing input during the 

development or planning stages of the work product generally do not become peer reviewers of 

that product (see Sections 1.2.2 to 1.2.7 for full discussion on this distinction). 
 Peer Review Handbook Page 49 

2.4.7 What Factors are Considered in Setting the Time Frame for Peer Review? 

Several factors impact how quickly a peer review may be desired. These include deadlines for 

completion of a project, research program, or rulemaking, funding availability, availability of 

quality peer reviewers, the possible need to seek public comment on the peer review product, and 

statutory and/or court-ordered deadlines. 

Peer review sometimes leads to new information and analyses. Reviewers may make 

recommendations for new research that would alter the work product and thus modify the 

scientific/technical basis for the action or rule it supports. For this reason, a completed peer 

review is desirable before issuing any proposal for public comment. If that is not logistically 

possible because of court or statutory deadlines, or other appropriate reasons, the Decision 

Maker should make every effort to complete the peer review before the close of the comment 

period. Because peer review comments on such work products could be of sufficient magnitude 

to warrant a revision to the proposed action or rule, Decision Makers should exercise diligence in 

completing the peer review prior to the proposal stage whenever possible. 

Whenever feasible and appropriate, Offices should make a draft scientific assessment that is 

highly influential available to the public for comment at the same time it is submitted for peer 

review (or during the peer review process) and sponsor a public meeting where oral presentations 

on scientific issues can be made to the peer reviewers by interested members of the public. 

When employing a public comment process as part of the peer review, Offices should, whenever 

practical, provide peer reviewers with access to public comments that address significant 

scientific or technical issues. To ensure that public participation does not unduly delay activities, 

Offices should specify time limits for public participation throughout the peer review process. 

2.4.8 Which Office/Region or Other Agency is Responsible for Conducting the 

Peer Review? 

The organization of the Decision Maker is normally responsible for conducting the peer review. 

Responsibility for conducting a peer review can be negotiable when more than one Agency 

Office or Region or other agencies are involved. Often, the degree of involvement by any of the 

organizations and agencies and their ability to fund peer review will determine who has the lead 

for the peer review. 

2.5 Creating the Peer Review Record and Peer Review Reports 

2.5.1 What is the Peer Review Record? 

The peer review record is the formal record (file) of decision on the conduct of the peer review, 

including the type of peer review performed and an explanation of how the peer review 

comments were addressed. It includes sufficient documentation for an uninvolved person to 

understand what actually happened and why. The Peer Review Leader (with the program 
 Peer Review Handbook Page 50 

manager if there is one) creates a separate, clearly marked peer review file within the overall file 

for development of the work. Once the peer review is completed, it is the responsibility of the 

Peer Review Leader to ensure that the peer review record is filed and maintained in accordance 

with the organization’s document retention procedures. 

The Peer Review Record is separate from the entry in the Science Inventory Database. While 

some information from the peer review record appears in the database, the paper peer review 

record is the official record of the peer review. 

2.5.2 How Can the Peer Review Record Improve the Peer Review Process? 

A good peer review record allows future reference to what happened during the peer review, and 

helps Decision Makers make appropriate use of peer reviewer input. In addition, a good record 

helps ensure that EPA’s Peer Review Policy is implemented. The Peer Review Leader is 

responsible for ensuring that the peer review record for individual work products is collected and 

maintained until completion of the peer review effort. 

2.5.3 What Should Be in the Peer Review Record? 

The peer review record should include all materials considered by the individual peer reviewers, 

the peer review report, and other input. Such materials include, at a minimum (see also Section 

4.3.1): 

a) The draft work product submitted for peer review; 

b)	 Materials and information (including the charge) given to the peer reviewers; 

c) The peer review report, which summarizes the peer review findings and contains 

information about the peer reviewers (such as reviewers’ names, affiliations, and a 

statement concerning potential conflicts and their resolution, if applicable); 

d) Logistical information about conduct of the peer review (such as times and locations of 

meetings); 

e)	 A memorandum, or other written record, approved by the Decision Maker, responding to 

the peer review comments specifying acceptance or, where thought appropriate, rebuttal 

and non-acceptance. The Office should prepare a written response to the peer review 

report addressing each comment. 

f)	 The final work product. 

When deciding if particular materials should be included in the record, the Peer Review Leader 

should consider whether the materials would help reconstruct the peer review process and results 

at a later time. If the materials may be helpful, they should be part of the peer review record. 
 Peer Review Handbook Page 51 

2.5.4 How and When Should You Develop and Post Peer Review Reports and 

Agency Responses for Influential Scientific Information and Highly 

Influential Scientific Assessments? 

Offices are expected to make peer review reports of the influential scientific information and 

highly influential scientific assessments publicly available to implement the provisions of the 

OMB Bulletin. Offices should instruct peer reviewers to prepare a report that describes the 

nature of their review and the nature of their findings and conclusions. The peer review report 

should either (a) include a verbatim copy of each reviewer’s comments (either with or without 

specific attributions) or (b) represent the views of the group as a whole, including any disparate 

and dissenting views, although attribution of comments to names is not necessary. The names of 

the reviewers and their organizational affiliations should be included in the report. For highly 

influential scientific assessments, the report should also include the charge to the reviewers and a 

short paragraph on both the credentials and relevant experiences of each peer reviewer. 

EPA will post or provide a link to the peer review reports on the Science Inventory website 

(http://cfpub.epa.gov/si/) along with all materials related to the peer review (charge statement 

and Agency response). The credibility of the final work product is likely to be enhanced if the 

public understands how the Agency addressed the specific concerns raised by the peer reviewers. 

Offices should consider preparing a written response for inclusion in the peer review report. For 

highly influential scientific assessments, the OMB Bulletin explicitly calls for Offices to prepare 

a written response to the peer review report explaining (a) the agency’s agreement or 

disagreement with the views expressed in the report, (b) the actions that have or will be 

undertaken to respond to the report, and (c) the reasons the Office believes those actions satisfy 

any key concerns or recommendations in the report. These responses will also be posted on the 

Science Inventory website. 

2.5.5 What Should You Do with a Peer Review Record That Pertains to a 

Rulemaking Action? 

The Peer Review Leader should coordinate with his/her program’s docket office to see that 

proper docketing procedures are followed for a peer review of a work product supporting a rule. 

If EPA relies on influential scientific information or a highly influential scientific assessment to 

support a regulatory action, the preamble should include a discussion of how EPA implemented 

the provisions of the OMB Bulletin. See Appendix C for a template to use for this purpose. 

2.5.6 When Should the Peer Review Record Building Process Begin? 

An early start at developing and maintaining a peer review record will help ensure the record is 

complete and helpful. Ideally, the record begins when the decision to peer review a work 

product is made. The Peer Review Leader should construct the peer review record from this 

point on – doing so will avoid potentially time-consuming reconstruction at a later point. Note 

that the peer review record is not complete until it contains a copy of the final work product 
 

 

 Peer Review Handbook Page 52 

which addresses the peer review comments. If a product is not peer reviewed, a record should 

also be created which explains why the product was not peer reviewed. 

2.5.7 What are the Differences in Record Keeping for a Review by Individuals 

Compared to Review by a Panel? 

Generally, the content of the peer review record would be the similar. In the case of a review by 

individuals, the peer review record would typically contain each individual's comments, while 

for a panel review, the record typically contains a summary or other synthesis documenting the 

panel's deliberations. 

2.5.8 Where Should the Peer Review Record be Kept and For How Long? 

During the active conduct of the peer review, the Peer Review Leaders maintain the peer review 

record themselves until the peer review is complete. Minimally, the file should be maintained 

until one year after the completed peer review is reported in the next annual reporting cycle. 

After that, the peer review record should be maintained for a reasonable period of time, as 

delineated by the organization's document retention policy. Establishment and maintenance of 

the archive where the peer review records ultimately reside are an organization’s responsibility 

(i.e., not that of an individual program manager or Peer Review Leader). Generally, to allow 

flexibility, individual Offices and Regions should decide the appropriate level of organizational 

responsibility and how to ensure the record is routinely available. The peer review record may 

be kept with other records relating to the overall project, as long as it is easily and separately 

identifiable. 

There are also specific procedures regarding the use of dockets for record-keeping; however, 

these are not covered in this Peer Review Handbook. The documents contained in the peer 

review record should be maintained in accordance with the Agency’s record-keeping retention 

schedule for such records. For details, see EPA’s National Records Management Program at 

http://www.epa.gov/records. One long-term archiving mechanism may be the formal archiving 

at the Federal Records Center. 

2.5.9 Are Internal Peer Review Comments Included in the Peer Review Record? 

To be considered a legitimate peer review, internal EPA peer reviews should be formally 

conducted and documented. Such a process would be consistent with the guidance found in this 

Handbook for planning, conducting, and completing a peer review. When you follow this formal 

process to obtain peer review from EPA peers (see Section 1.5.9), then the whole record of that 

internal peer review should be included in the peer review record. This includes all the materials 

detailed in Section 2.5.3 (also see Section 4.3.1). Conducting a formal internal peer review is not 

the same thing as informal input from your EPA colleagues (i.e., “colleagues down the hall”), 

nor peer input from Agency personnel helping to develop the work product, nor organizational 

review and clearance processes. Such inputs from these informal processes should not be placed 
 Peer Review Handbook Page 53 

in the peer review record. The peer review record should contain only the information obtained 

when you conduct a formal internal peer review. 

In some cases, an internal EPA peer review may be followed by a separate external peer review. 

In such a case, however, the external peer review will stand as the peer review of record since it 

is viewed as more independent in nature, has broader fields of available expertise which can be 

brought to bear on the issues and often includes greater depth for specific disciplines. 

2.6 Budget Planning 

2.6.1 What Budgetary Factors Should You 

Consider in a Peer Review?
 

Peer review is part of the 

normal cost of doing business. 

Resources necessary for the implementation of the Peer Review Policy should be requested 

through the usual Agency budgetary processes. The budget formulation process within the 

Executive Branch is followed, after appropriation bills are passed by Congress, by budget 

execution. These two processes provide opportunities to secure resources for activities carried 

out by Headquarters and Regional offices, including peer review. The work products which will 

undergo peer review should have adequate funding requests for the coming fiscal year. 

Similarly, adequate funding should appear in the actual approved operating plan to ensure the 

peer review can be conducted. For purposes of budget planning, the costs of peer review would 

include the FTE cost of staff, the contract or other costs associated with the use of outside peer 

reviewers, and the administrative costs of conducting a review (copying, travel expenses, etc). 

2.6.2 What Input is Needed for the Annual Budget Formulation and Budget 

Execution Process? 

Senior Management in Program Office and Regions (including Decision Makers and budget 

officers) should ensure that budget requests include anticipated resources for peer review. Peer 

review should be considered as a normal part of doing business. Peer review resource 

considerations should also be addressed in the analytic blueprint for Agency rule-making actions. 

2.7 Legal Considerations 

2.7.1 What Are the Legal Ramifications of the Peer Review Policy? 

The Peer Review Policy does not establish or affect legal rights or obligations. Rather, it 

confirms the importance of peer review where appropriate, outlines relevant principles, and 

identifies factors Agency staff should consider in implementing the policy. Except where 

provided otherwise by law, peer review is not a formal part of or substitute for notice and 

comment rulemaking or adjudicative procedures. EPA’s decision to conduct peer review in any 

particular case is wholly within the Agency’s discretion. Similarly, nothing in the Policy creates 

a legal requirement that EPA respond to peer reviewers. However, to the extent that EPA 
 Peer Review Handbook Page 54 

decisions rely on scientific and/or technical work products that have been subjected to peer 

review, the remarks of peer reviewers should be included in the record for that decision. 

2.7.2 Is Legal Advice Needed? 

AA/RA staff and management should work regularly and closely with individual OGC/Regional 

Counsel (RC) staff assigned to Agency activities. Peer Review Leaders should initially consult 

with their usual OGC/RC advisors for legal advice or referral. Headquarters attorneys have 

specialties in specific areas and can be consulted as needed (e.g., FACA considerations, 

contractual responsibilities; ethics and potential conflicts of interest). 

2.8 Federal Advisory Committee Act (FACA) Considerations 

The Federal Advisory Committee Act, 5 USC App. 2, imposes certain open meeting (public 

announcement in the Federal Register), balanced membership, and chartering requirements (with 

the approval of the General Services Administration (GSA)) before the Agency establishes, 

controls or manages an “advisory committee” for advice or recommendations. Peer review 

carried out by formal and established (chartered) Federal advisory committees, such as the 

Science Advisory Board (SAB) or the FIFRA Scientific Advisory Panel (SAP), is always subject 

to FACA requirements. However, FACA considerations do not apply to every EPA and 

contractor-run peer review. 

In the next Section you will find information on the applicability of FACA to EPA- or 

contractor-run peer reviews. 

2.8.1 When Do FACA Requirements Apply to EPA-Run Peer Reviews? 

In most cases, Federal Advisory Committee Act (FACA) requirements apply to EPA-run peer 

reviews that are conducted by formal and established (chartered) Federal advisory committees, 

such as the Science Advisory Board (SAB) or the FIFRA Scientific Advisory Panel (SAP). 

These groups have the following characteristics: 

a) The group is established, controlled, or managed by EPA; 

b) The group has a fixed membership, established purpose, and a set agenda; and 

c) The group strives to produce collegial, rather than individual, advice to EPA. 

EPA-run peer reviews that are not originally intended to be subject to FACA, but which exhibit 

the above characteristics, may unintentionally become subject to FACA. Questions concerning 

the applicability of FACA to peer review meetings should be addressed to the FACA experts in 

the Cross-Cutting Issues Law Office of OGC, or the appropriate Office of Regional Counsel. 
 Peer Review Handbook Page 55 

2.8.2 When Are EPA-Run Peer Reviews Not Subject to FACA? 

If EPA conducts a peer review with the purpose of obtaining advice from the individual peer 

reviewers and not for the purpose of obtaining a peer review product from the group (as a 

collective or consensus body), the peer review, in most cases, would not be subject to FACA. 

Peer review participants provide only their own views or recommendations and do not vote nor 

do they provide collective or consensus recommendations to EPA. When referring to the 

recommendations of the individual reviewers, EPA should not characterize these 

recommendations using terms such as “collective” or “consensus.” As a general matter, 

workshops and “letter reviews” that seek individual views or comments are usually not subject to 

the requirements of FACA. 

In addition to ensuring that peer reviewers only provide comments as individuals, EPA officials 

may wish to lessen the potential for a challenge under FACA by seeking balanced participation 

at peer review meetings, and allowing interested members of the public to attend, and ensuring 

that they have access to appropriate materials. 

Non-FACA meetings may be announced in the Federal Register (providing that it is clear in the 

notice that such meetings are not subject to FACA) as it provides the public with useful 

information and a point of contact concerning the peer review. In addition, non-FACA (as well 

as FACA) meetings should also be advertised via other avenues (e.g., the Web, local 

newspapers, and mailing lists). 

2.8.3 How Do You Ensure that a Contractor-Run Peer Review Does Not Become 

Subject to FACA? 

Committees (or other peer review groups) established, controlled or managed by an outside 

organization (such as by an EPA contractor) to provide that outside organization with advice and 

recommendations (that will be submitted eventually to EPA as a contractor report) are not 

subject to FACA. Although FACA should not apply to contractor-run peer reviews, there are 

things that you (i.e., EPA) can inadvertently do that may invoke FACA. 

The following are considerations that you should be aware of when a contractor runs a peer 

review (e.g., letter review, panel, workshop, etc.) for EPA: 

a) The outside party’s peer review may be subject to FACA if EPA establishes, manages or 

controls the peer review group (e.g., EPA selects panel members, runs the meeting, etc.); 

b) EPA should not provide contractors with a draft agenda or suggested format for meetings. 

EPA contractors should manage and control the process, including running any meetings; 
 Peer Review Handbook Page 56 

c) At the request of the EPA contractor, EPA can provide a briefing to the peer reviewers 

(e.g., in a conference call with the contractor on the line) on the history or background of 

the development of the document. EPA should only provide technical or background 

information and not use the call to take over the contractor’s peer review group. Not only 

should the contractor be on the line, but it should be very clear to all participants that the 

contractor is in charge of the call. The contractor, not EPA, should invite people to 

participate, make all administrative arrangements, conduct the meeting, and control the 

agenda; 

d)	 EPA employees may attend the peer review panel meetings or workshops. However, 

they may not take over the control of the meeting. The contractor may call on them to 

speak when appropriate, but EPA staff should limit their participation to providing 

technical and/or background information, and not attempt to, or appear to, take over the 

contractor’s meeting; 

e) Since FACA does not apply when a contractor establishes, controls, or manages a peer 

review, the contractor need not avoid terms such as “collective” or “consensus” when 

reporting agreement among its peer reviewers (subcontractors); 

f)	 EPA may provide comments to the contractor on the contractor’s peer review only to the 

extent that the Agency is verifying that the contractor has satisfactorily completed the 

report in accordance with the work assignment. EPA should not attempt to make changes 

in the contractor’s conclusions; this would compromise the independence of the peer 

review conducted by the contractor. 
 

 

 Peer Review Handbook Page 57 

3. Conducting A Peer Review 

3.1 Overview Statement 

The success and usefulness of any peer review depends on the quality of the draft work product 

submitted for peer review, the care given to the statement of the issues or "charge," the match 

between the peer review draft product and the form of peer review, the match between the peer 

review draft product and the scientific/technical expertise of the reviewers, and Agency use of 

peer review comments in the final product. It is not simply enough to conduct a peer review; 

each of the foregoing elements deserves serious attention. 

Figure 2 shows the order of activities for conducting a peer review. The charge should be 

completed before selection of the peer reviewers to ensure that they have the appropriate 

expertise to address the questions raised. 

3.2 Charge to the Peer Reviewers 

3.2.1 What is a Charge?

 As part of each peer review, the Peer Review Leader formulates a clear, focused charge that 

identifies the technical and scientific issues on which the Agency would like feedback and 

invites suggestions for improving the document as a whole. This request signals the Agency's 

receptivity to expert recommendations. The charge to peer reviewers usually makes two general 

requests. First, it focuses the review by presenting 

specific questions and concerns surrounding such 

issues as the comprehensiveness of the literature 

reviewed, the soundness of the method proposed, the 

scientific support for the assumptions employed, and 

the sensitivity of the results to alternative assumptions. 

Secondly, it invites general comments on the entire 

work product. The reviewers should be asked to ensure 

that the potential implications of the uncertainties for the technical conclusions drawn are clearly 

and transparently presented. The specific and general comments should focus on the scientific 

and technical merits of the work product and, where germane, whether the scientific/technical 

studies have been applied in a sound manner. Remember, the peer review is not for the decision 

or action itself, but for the underlying scientific and/or technical work product; reviewers should 

not be asked to provide advice on policy. Additionally, Focused questions greatly simplify the 

task of collating, analyzing and synthesizing peer review comments on a topical basis. The 

questions should be specific enough to encourage helpful comments, including constructive 

alternatives, but not so specific that they preclude creative responses (e.g., simply asking whether 

or not the reviewer agrees with the model choice or conclusion). Understanding the reviewer’s 

Time is well-spent preparing a good 

charge, as the charge is crucial for 

an effective peer review. 
 Peer Review Handbook Page 58 

reasoning is pivotal to the Agency’s ability to address the reviewer’s concern as well as crafting 

specific improvements to the work product. The peer reviewers' written responses to these 

questions help the Agency create a peer review record. As a general rule, time is well-spent 

preparing a good charge, as the charge is crucial for an effective peer review. 

3.2.2 What are the Essential Elements of a Charge? 

a) A brief overview or introduction (describe what the work product is, how it was 

developed, how it will be used) 

b) As needed, a brief description or listing of any background materials provided to the peer 

reviewers 

c) The specific issues or questions to be addressed by the peer reviewer(s), including 

logistical details such as: 

1)	 The due date of reviewers’ comments; 

2)	 The format of reviewer responses (e.g., MS Word, Adobe Acrobat, e-mails, or 

text files); 

3) The point of contact, in case peer reviewers have questions. 

3.2.3 Where Can You Get an Example of a Charge? 

Appendix D – Examples of Peer Review Charges, contains examples of successful charges that 

cover a variety of issues. Appendix E – Guidance on Requesting a Review by the Science 

Advisory Board, provides guidance for obtaining Science Advisory Board (SAB) services. 

3.2.4 Can a Stakeholder Provide Input to the Charge to the Peer Reviewers? 

Yes. EPA may decide to obtain stakeholder input on the charge to the peer reviewers, but EPA 

makes the final determination on what elements to include in the charge to ensure that it meets 

EPA’s needs for the peer review. 

a) If you obtain stakeholder input, include interested parties to the extent feasible based 

upon statutory, regulatory, budgetary and/or time constraints. Do not limit input to one 

stakeholder or one side of a controversial issue (e.g., a responsible party or environmental 

group). 

b)	 If EPA has hired a contractor to perform the peer review, it should still be EPA personnel 

who obtain stakeholder input and provide the list of charge questions to the contractor. 
 Peer Review Handbook Page 59 

c) If you form a committee of stakeholders to help develop the charge, be aware that your 

committee may become subject to the requirements of FACA (see Section 2.8). 

3.2.5 Who Writes the Charge When You Hire a Contractor to Conduct the Peer 

Review? 

In general, if EPA hires a contractor to perform the peer review, EPA should allow the contractor 

independence in conducting it. However, with regard to the charge, EPA should provide the 

contractor with a substantive list of questions that EPA wants included in the charge letter to 

ensure that the peer review meets EPA’s needs. If the EPA project manager has them ready, the 

list of charge questions can be incorporated into the Statement of Work. Based on this list, the 

contractor can then prepare and submit the actual charge letter to the peer reviewers. Prior to 

submitting the charge letter to the reviewers, the contractor should be directed to give EPA an 

opportunity to review the charge letter to ensure that it meets EPA’s needs. EPA cannot submit 

the charge directly to the peer reviewers when the review is being conducted by a contractor. 

3.2.6 Is it Okay to Ask a Contractor to Develop the Charge to the Peer Reviewers? 

No. EPA should provide the charge questions to the contractor as discussed above. The 

contractor, though, may provide assistance and advice in the development of the charge. 

3.3 Time Line 

3.3.1 What are the Factors in Scheduling a Peer Review? 

The peer review schedule is a critical feature of the process. The schedule should take into 

account the availability of a quality draft work product, availability of appropriate experts, time 

available for peer review comments, deadlines for the final work product, and logistical aspects 

of the peer review (e.g., contracting procedures). 

The schedule for peer review should take into account the overall rulemaking (or other decision 

making) schedule. For rules, in particular those in Tier 1 and Tier 2, the scheduling of the peer 

review should be included in the development of the analytic blueprint. Peer review sometimes 

leads to new information and analyses, or recommendations for new research that would alter the 

work product and thus modify the scientific/technical basis for the action. For this reason, it is 

usually advisable to complete the peer review before taking public comment on the rule, or at 

least before the close of the public comment period. Note, however, that whenever feasible and 

appropriate, offices should make a draft highly influential scientific assessment available to the 

public for comment during the peer review process, and, if feasible and appropriate, sponsor a 

public meeting where oral presentations on scientific issues can be made to the peer reviewers by 

interested members of the public. When employing a public comment process as part of the peer 

review, Offices should provide the reviewers access to the public’s comments that address 

scientific or technical issues. 
 

 

 Peer Review Handbook Page 60 

3.4 Selection of Peer Reviewers 

3.4.1 What are Considerations for Selecting Peer Reviewers? 

The choice of peer reviewers should be based primarily upon on the reviewers’ expertise, 

knowledge, skills, and experience and should include specialists from multiple disciplines, as 

appropriate. The group of reviewers should be sufficiently broad and diverse to fairly represent 

the relevant scientific and technical perspectives and fields of knowledge; they should represent 

a balanced range of technically legitimate points of view. Each peer reviewer should have 

recognized expertise that bears on the subject matter under discussion. It may sometimes be 

appropriate, as in the case of FACA committees, to consider "affiliation" (e.g., industrial, 

academic, or environmental community). 

The selection of independent peer reviewers is also critical to an effective peer review. EPA 

should always make every effort to use peer reviewers who do not have any conflict of interest 

or an appearance of a lack of impartiality, and who are completely independent (see Section 

3.4.5). However, the very need to have experienced individuals on a peer review, along with the 

desire to have appropriate technical balance and representation, means that the field of potential 

peer reviewers may include those that are less than independent, have a conflict of interest, or 

might at least appear to lack impartiality. Sometimes selecting individuals who have served a 

variety of organizations rather than a single one for an extended period may provide expertise 

with diverse perspective. The emphasis on independence and expertise applies equally to 

government experts and experts from the larger scientific community 

To ensure that the selected reviewers have the appropriate expertise, knowledge, skills, and 

experience, the charge to the reviewers should be determined in advance of the selection of the 

reviewers. Selection of peer reviewers should be made by identifying reviewers with the 

appropriate expertise and then narrowing the field of potential peer reviewers to those 

individuals that are independent, do not have a conflict of interest and do not appear to lack 

impartiality. 

You should also consider requesting that the public, including scientific and professional 

societies, nominate peer reviewers. 

3.4.2 Where Do You Find Peer Reviewers? 

Recommendations for potential peer reviewers can be identified by a number of organizations. 

These include external groups such as the affected party(ies), special interest groups, public 

interest groups, environmental groups, professional societies, trade or business associations, 

State organizations or agencies, Native American Tribes, colleges and universities, the National 

Research Council, and other Federal agencies with an involvement in or familiarity with the 

issue. Agency associated groups include the staff of the SAB or the SAP, and relevant scientific 

and technical experts from Program or Regional Offices. 
 

 

 Peer Review Handbook Page 61 

In certain circumstances, existing organizations such as the SAB, SAP or BOSC may be used to 

conduct a peer review. These groups establish their own criteria for accepting work. If you use 

such an organization, you should coordinate directly with the organization (e.g., see Appendix E 

- Guidance on Requesting a Review by the Science Advisory Board, for its procedures). SAB, 

SAP and BOSC conduct formal, public, external peer reviews. 

Another method which might be used to find peer reviewers is public solicitation. The Science 

Inventory can assist the public solicitation process by announcing the opportunity for public 

nominations and by providing background on the review topic the public may find useful for 

identifying potential reviewers they may wish to nominate. EPA’s Science Advisory Board uses 

the Internet to solicit names for both ad hoc and standing advisory committees. These names, 

along with short biographical sketches, are also posted so that the public may not only nominate, 

but also comment on potential advisory committee members. See “Advisory Committee 

Meetings and Report Development: Process for Public Involvement” available from SAB’s 

website at http://www.epa.gov/sab/pdf/sabso_04_001.pdf 

Occasionally, a member of the scientific community will offer his/her services for peer review 

during an ongoing peer review. These offers may be at no cost or based on an expectation that 

reimbursement will be made. Disposition of these unsolicited offers should be handled on a case 

by case basis by the Peer Review Leader, and as necessary, in consultation with the Peer Review 

Coordinator, the Office of General Counsel (OGC), and appropriate Decision Makers. 

If you use a contract mechanism to conduct a peer review, the contractor may have its own pool 

of scientific and technical experts for peer review. EPA may provide contractors with 

information on potential peer reviewers for conducting a peer review, if such a listing is prepared 

in alphabetical order. EPA should not require that the contractor select from a prepared list, nor 

require that the contractor receive EPA approval before selecting any given peer reviewer 

(sometimes known as a “subcontractor”). However, EPA should review the list of peer 

reviewers for conformance to work assignment specifications and adherence to conflict of 

interest and appearance of a lack of impartiality concerns. Furthermore, when utilizing a 

contract mechanism to conduct peer review, EPA is not permitted to direct the prime contractor 

to a specific subcontractor (or peer reviewer), nor is EPA permitted to direct the peer reviewer 

(subcontractor). All interactions with the peer reviewers are to be coordinated through the prime 

contractor. See Section 3.6 for further information. 

Keep in mind that for contracting purposes, contractors are required to obtain Contracting 

Officer (CO) approval of subcontractors and the CO generally seeks the input of the Contracting 

Officer Representative (the COR may be the Peer Review Leader in many cases) before 

approving the use of subcontractors. In this case, as noted above, the Peer Review Leader should 

review the list of potential reviewers for conformance to the work assignment specifications and 

adherence to conflict of interest and appearance of a lack of impartiality concerns. 
 Peer Review Handbook Page 62 

3.4.3 Are External or Internal Peer Reviewers Preferred? 

External peer reviewers are generally preferred for influential scientific information, and are 

expected for highly influential scientific assessments. For some work products, like those 

reviewed at interim steps, either external or internal peer review may be appropriate. Selection 

of internal peer reviewers should be based upon technical expertise, available time and 

affiliation. That is, they should not come from the same Office or group producing the product 

or have any other connection with the product or document being peer reviewed. The choice of 

reviewers should be analyzed on a case-by-case basis. Reviewers employed by other Federal 

and State agencies may possess unique or indispensable expertise. Don’t forget that for highly 

influential scientific assessments, the use of internal peer reviewers is constrained by the OMB 

Bulletin. External peer reviewers should be selected based upon technical expertise as well, 

however, you should not use individuals who have been involved in the development of the work 

product. (See Section 1.2.6; see also Sections 1.2.7 to 1.2.9). 

3.4.4 What is Important in the Mix of a Peer Review Panel? 

A peer review panel or group can number from just a few individuals to ten or more, depending 

on the issue, the time and resources available, and the broad spectrum of expertise necessary to 

treat the range of issues/questions in the charge. Objective technical expertise and no conflict of 

interest or appearance of a lack of impartiality are critical characteristics for selecting peer 

reviewers. Naturally, experts whose understanding of the specific technical area(s) being 

evaluated are necessary; nevertheless, it is also important to include a broad enough spectrum of 

other related experts to completely evaluate the relevant impacts on other less obvious concerns 

(i.e., to comment not only if the job is being done right, but also whether the right job is being 

done). For example, for health related peer reviews, experts in such fields as ecology and 

economics may provide very useful insights. Although persons who are familiar with and have a 

substantial reputation in the field are often called upon repeatedly to be reviewers, it is important 

to keep a balance with new people who bring fresh perspectives to the review of a work product. 

The idea here is to avoid the repeated use of the same reviewer on multiple assessments unless 

his/her participation is essential and cannot be obtained elsewhere. 

There is usually a continuum of views on any issue. To the extent practicable, selected experts 

should have technically legitimate points of view that fall along the continuum. A review panel 

should include experts that are considered “mainstream” (nearer the center of the continuum) as 

well as those further along the continuum while generally avoiding the extremes. This will help 

maintain a balanced review panel, while allowing a broad range of views to be expressed and 

discussed. A balanced panel will allow consensus building if consensus is the object of a 

particular peer review; if not, it provides a spectrum of views for the Agency to evaluate. As 

such, the group of reviewers should be sufficiently broad and diverse to fairly represent the 

relevant scientific and technical perspectives and fields of knowledge. 
 Peer Review Handbook Page 63 

As a general rule, experts who have made public pronouncements or have had a predominant 

influence on the position for a given organization on an issue, those who have clearly "taken 

sides," may have an appearance of a lack of impartiality (see “Ethical Standards” below) and 

should be avoided. 

3.4.5 Ethical Standards 

As discussed previously (see Section 2.4), there are various mechanisms through which peer 

reviews may be accomplished. Internal peer reviews can be conducted by independent experts 

from within EPA, either individually or as ad hoc peer review panels. External peer reviews can 

be conducted by individual experts (e.g., through letter reviews) who are either Regular 

Government Employees (RGEs) of another agency, experts hired as EPA Special Government 

Employees (SGEs), or experts whose services are procured through a contract mechanism. In 

addition, external peer review panels can be convened through a contract mechanism or under 

the auspices of a Federal Advisory Committee. Finally, peer reviews may be conducted by 

outside organizations such as the National Academy of Sciences (NAS). 

To ensure a credible peer review process, high ethical standards are applied to each of these 

mechanisms. These standards are embodied in the various laws, regulations, and other 

requirements that apply to peer reviewers who are either RGEs, SGEs, contractors, or who are 

employed by outside organizations (e.g., NAS). 

There are specific regulations that describe ethics considerations for contractor personnel. The 

Federal Acquisition Regulations (FAR), the Environmental Protection Agency Acquisition 

Regulations (EPAAR), and other internal Agency documents define ethical standards, such as 

Conflict of Interest (COI), and describe the Agency’s policies and procedures. This Peer Review 

Handbook provides some guidance regarding the application of this information to peer review 

(see Section 3.6 and Appendix F – Example Statements of Work for Contracts); however, 

specific questions should be addressed to the Office of Acquisition Management. 

For peer reviews conducted by outside organizations, the peer review leader or other appropriate 

official should be thoroughly familiar with the ethics policies and requirements of the 

organization conducting the review. For example, the NAS National Research Council has 

published their “Policy on Committee Composition and Balance and Conflicts of Interest.” 

The remainder of this section is focused on the laws and regulations governing the standards of 

ethical conduct for RGEs and SGEs. The standards for government employees are very 

thorough and, as such, may help the peer review leader or other appropriate official to identify 

issues that may be more generically of concern with respect to the selection of peer reviewers. 
 

 

 Peer Review Handbook Page 64 

Conflicts of Interest: 

Public service is a public trust. As such, it is vitally important that government employees 

maintain the highest ethical standards. While there are many aspects of ethical behavior, for the 

purposes of peer review we are mostly concerned with compliance with applicable Federal ethics 

statutes and regulations. The basic regulations are issued by the Office of Government Ethics 

(OGE) and are occasionally supplemented with additional regulations by EPA. 

Compliance with Federal ethics regulations can be a confusing and complex process. However, 

the process is usually concerned with determining whether a RGE or a potential SGE has an 

ethics issue with respect to a “conflict of interest” or “an appearance of a lack of impartiality.” 

Each of these two concepts will be explained in more detail below. However, sometimes the two 

concepts are confused. A “conflict of interest” is concerned with matters of financial interest. 

There is either a conflict of interest or there is not. An “appearance of a lack of impartiality” 

(sometimes mischaracterized as a “potential conflict of interest”), can be financial or not 

financial in nature. The determination of an “appearance issue” is not quite as clear-cut as a 

“conflict of interest” and calls for more of a judgment on the part of the determining official as to 

whether one exists. 

Let’s look at each of these two concepts in detail. 

Generally, a conflict of interest arises when the person’s financial interests are affected by 

his/her participation in a particular matter, when he/she, his/her associates or other individuals 

whose interests are imputed (as provided under Federal law) would derive benefit from 

incorporation of their point of view in an Agency product, and/or when their professional 

standing and status or the significance of their principal area of work might be affected by the 

outcome of the peer review. Whenever there are questions about conflicts of interest, you should 

contact the appropriate Deputy Ethics Official, Ethics Assistant or the Designated Agency Ethics 

Official for clarification and assistance. For a list of EPA's Deputy Ethics Officials, see 

http://intranet.epa.gov/ogc/ethics/deos.htm. 

For Financial Conflict of Interest (COI) issues, the basic 18 USC § 208 provision states that: 

An employee is prohibited from participating personally and substantially in an official 

capacity in any particular matter in which he, to his knowledge, or any person whose interests 

are imputed to him under this statute has a financial interest, if the particular matter will have 

a direct and predictable effect on that interest. 

For a conflict of interest to be present, all elements in the above provision must be present. If an 

element is missing, it is unlikely that there is a conflict of interest issue. However, the 

regulations concerning an appearance of a lack of impartiality may still apply and need to be 

considered. 
 Peer Review Handbook Page 65 

One very important aspect to consider is whether a charge to an advisory committee or other peer 

review panel constitutes a “particular matter.” A “particular matter” refers to matters that will 

involve deliberation, decision, or action that is focused upon the interests of specific people, or a 

discrete and identifiable class of people. It does not refer to consideration or adoption of broad 

policy options directed to the interests of a large and diverse group of people. A particular 

matter of general applicability means a particular matter that is focused on the interests of a 

discrete and identifiable class of persons, but does not involve specific parties. A particular 

matter involving specific parties means any judicial or other proceeding, application, request for 

a ruling or other determination, contract, claim, controversy, investigation, charge, accusation, 

arrest or other particular matter involving a specific party or parties in which the United States is 

a party or has a direct and substantial interest. Participating personally means direct participation 

in the matter to be reviewed. Participating substantially refers to involvement that is of 

significance to the matter under consideration. For more information regarding the definition of a 

particular matter, see the Office of Government Ethics document “Standards of Ethical Conduct 

for Employees of the Executive Branch” available at http://www.usoge.gov. 

Another consideration is whether there will be a direct and predictable effect on the financial 

interests of the reviewers. A direct effect on a participant's financial interest exists if a close 

causal link exists between any decision or action to be taken in the matter and any expected 

effect of the matter on the financial interest. A particular matter does not have a direct effect if 

the chain of causation is attenuated or is contingent upon the occurrence of events that are 

speculative or that are independent of, and unrelated to, the matter. A particular matter that has 

an effect on a financial interest only as a consequence of its effects on the general economy is not 

considered to have a direct effect. A predictable effect exists if there is an actual, as opposed to a 

speculative, possibility that the matter will affect the financial interest. 

The question of whether a charge is a particular matter involving a specific party or parties or 

whether it is a particular matter of general applicability has important ramifications for reviewers 

who are Special Government Employees (SGEs) as defined under 18 USC § 202(a). SGEs 

serving on FACA (Federal Advisory Committee Act) committees are covered by certain 

exemptions from the conflict of interest statutes. The most significant of these is 5 CFR 

2640.203(g), which pertains to certain financial interests arising from an SGEs’s outside 

employment. Specifically, this exemption permits SGEs serving on FACA committees to 

participate in particular matters of general applicability (e.g., development of general regulations, 

policies or standards) where the disqualifying interest arises from the SGEs non-Federal 

employment or prospective employment (but NOT due to financial holdings or consultancies). 

Furthermore, the question of whether a charge is a particular matter involving a specific party or 

parties, or whether it is a particular matter of general applicability, has other ramifications for 

SGEs (and other Federal employees for that matter). For example, if a charge is a particular 

matter involving a specific party or parties, there are certain restrictions regarding obtaining U.S. 

Government contracts/grants as well as future representation back to the Federal government. 

For more information, see 18 USC § 207 or contact your Deputy Ethics Official/Ethics Assistant. 
 Peer Review Handbook Page 66 

Even if an SGE is determined to have a conflict of interest, that does not mean that it is 

impossible for her or him to participate in an advisory committee or other peer review. For 

example, in some cases it is possible to obtain a waiver for her or his participation. It is also 

possible to have a particular SGE be recused from certain portions of a review that would 

otherwise create a conflict of interest. However, you should always weigh the advantages and 

disadvantages of this approach very carefully. In addition, public disclosure of these conflicts in 

a manner compliant with the Privacy Act and other applicable regulations and practice, will add 

to the integrity of the advisory committee and peer review process. Additional information for 

SGEs may be found in “Conflict of Interest and the Special Government Employee: A Summary 

of Ethical Requirements Applicable to SGEs” available from the U.S. Office of Government 

Ethics (U.S. OGE) at http://www.usoge.gov. 

Appearance of a Lack of Impartiality: 

The Code of Federal Regulations at 5 CFR § 2635.502(a) states that: 

Where an employee knows that a particular matter involving specific parties is likely to have 

a direct and predictable effect on the financial interest of a member of his household, or 

knows that a person with whom he has a covered relationship is or represents a party to such 

matter, and where the person determines that the circumstances would cause a reasonable 

person with knowledge of the relevant facts to question his impartiality in the matter, the 

employee should not participate in the matter unless he has informed the agency designee of 

the appearance problem and received authorization from the agency designee.” 

Further, § 2635.502(a)(2) states that, 

An employee who is concerned that circumstances other than those specifically described in 

this section would raise a question regarding his impartiality should use the process 

described in this section to determine whether he should or should not participate in a 

particular matter. 

Even though circumstances for specific candidates may not raise an issue of a conflict of interest, 

each advisory committee member or peer reviewer should be evaluated to ensure that an 

appearance of a lack of impartiality does not preclude their participation. 

How do you evaluate candidates for appearance issues? In addition to collecting confidential 

financial disclosure forms (described below) and public comment (where appropriate), it may be 

necessary on occasion to seek additional information from the potential advisory committee 

member. For example, you might want to consider asking additional questions which will help 

you to determine whether there is an appearance issue. Some suggested questions would be: 
 

 

 Peer Review Handbook Page 67 

a) Do you know of any reason that you might be unable to provide impartial advice on the 

matter to come before the Panel or any reason that your impartiality in the matter might 

be questioned? 

b) Have you had any previous involvement with the review document(s) under 

consideration? If so, please identify and describe that involvement. 

c) Have you served on previous advisory panels, committees or subcommittees that have 

addressed the topic under consideration? If so please identify those activities. 

d) Have you made any public statements (written or oral) on the issue? If so, please identify 

those statements. 

e) Have you made any public statements that would indicate to an observer that you have 

taken a position on the issue under consideration? If so, please identify those statements. 

If an SGE has an appearance of a lack of impartiality, it may still be possible to make a written 

determination that she or he may serve on an advisory committee or to ensure that a particular 

advisory committee member or peer reviewer is recused from certain areas of a review. See your 

Deputy Ethics Official/Ethics Assistant for more information. 

A Few More Ethical Considerations: 

In addition to the above considerations, you should check for a few more items to ensure 

compliance with Federal ethics laws: 

Restrictions on Representation: 

There are two statutes (18 USC § 203 and 18 USC § 205) that impose related restrictions on the 

outside activities of RGEs and SGEs, particularly involving activities involving the 

representation of others before the Federal government. Section 203 prohibits an employee from 

receiving, agreeing to receive, or soliciting compensation for representation services before any 

court or Federal agency/entity in connection with any particular matter in which the United 

States is a party or has a direct and substantial interest. Section 205 prohibits an employee from 

personally representing anyone before any court or Federal agency or other specified Federal 

entity, in connection with any particular matter in which the United States is a party or has a 

direct and substantial interest. 

Fortunately, these restrictions are limited in their application to SGEs. One of the most 

significant limitations is that SGEs are restricted only in connection with particular matters 

involving specific parties. Furthermore, the restrictions on SGEs are narrowly drawn to focus 

only on those matters in which the SGE actually participated for the Government, as well as, in 

some cases, those matters actually pending in the SGE’s own agency (in our case, EPA). 
 Peer Review Handbook Page 68 

Emoluments Clause of the U.S. Constitution: 

The Department of Justice (DOJ) originally held that SGEs may be subject to the “Emoluments 

Clause” of the United States Constitution (Article I, Section 9, clause 8) which prohibits persons 

who “hold office of profit or trust” in the Federal government from having any position or 

receiving any payment from a foreign government. However, an opinion from DOJ/Office of 

Legal Counsel (March 9, 2005) states that SGEs who do not have access to classified 

information and who serve in purely advisory positions do not hold an “office of profit or trust” 

in the Federal government and are not subject to the Emoluments Clause. Therefore, it is 

extremely unlikely that any SGE performing a peer review for EPA would be subject to the 

Emoluments Clause. As always, check with your Deputy Ethics Official/Ethics Assistant if you 

need more information. 

Putting It All Together: 

The fine art of determining whether an advisory committee member or other peer reviewer has a 

conflict of interest or an appearance of a lack of impartiality can be difficult, confusing and 

complex. Needless to add, it is also important that any decision that is made concerning advisory 

committee members or peer reviewers be appropriately documented and all applicable Federal 

record regulations and Privacy Act regulations be strictly adhered to. 

The matter of obtaining a fair and credible peer review, as well as maintaining the credibility of 

the Agency and the Agency’s scientific products, is of paramount importance. Peer review 

leaders are strongly encouraged to obtain peer reviewers who do not have a conflict of interest or 

an appearance of a lack of impartiality. Finding a totally independent peer reviewer is a difficult 

and often daunting task. See Section 3.4.6 for additional steps to take to ensure a credible peer 

review. 

For additional information on putting together an advisory panel (e.g., for highly influential 

scientific assessments), please refer to the SAB brochure “Overview of the Panel Formation 

Process at the Environmental Protection Agency Science Advisory Board, September 2002" at 

http://www.epa.gov/sab/pdf/ec02010.pdf. 

3.4.6 What Techniques Help Ensure Disclosure and Appropriate Resolution of 

Conflicts of Interest? 

Before finalizing the selection of reviewers, the Peer Review Leader should ascertain whether 

each potential peer reviewer’s involvement in certain activities could pose a conflict of interest 

(COI) or create an appearance of a lack of impartiality. One way of identifying conflicts is to 

ask potential reviewers about work and clients (both current and prior) that might create conflicts 

or the appearance of a lack of impartiality in carrying out peer review activities. When the peer 

review process is being conducted by a contractor, the requirement for addressing peer 

reviewers’ possible conflicts of interest should be highlighted in the Statement of Work or the 
 Peer Review Handbook Page 69 

work ordering instrument (i.e., Work Assignment, Delivery Order, Task Order, etc.) and is a 

matter that is bound by contractual clauses with the Contracting Officer as the final Decision 

Maker in contracting matters. 

You should take care to avoid ethical issues with respect to advisory committee members or 

other peer reviewers and the work product under review. Remember, each situation is unique 

and should be treated on a case-by-case basis. The following are some considerations that 

should be addressed in evaluating ethical issues: 

a) Paying attention to the employment, financial, and professional affiliations of the 

participants; 

b) Exploring the issue directly with each participant before the review process takes place; 

c) Disclosing publicly at the beginning of meetings any previous involvement with the 

issue; 

d) Obtaining confidential financial disclosure forms. In the cases of regular government 

employees (RGEs), use either the OGE Form 450 or the SF-278 Form as appropriate. In 

the case of Special Government Employees (SGEs), use EPA Form 3110-48 

(Confidential Financial Disclosure Form for Special Government Employees Serving on 

Federal Advisory Committees at the U.S. Environmental Protection Agency). All these 

must be filed annually (with some exceptions). Remember, you can always discuss any 

conflicts of interest or appearances of a lack of impartiality with your Deputy Ethics 

Official, Ethics Assistant or Deputy Agency Ethics Official as appropriate, but it is 

important to remember that there is no “attorney-client privilege” for these discussions; 

e) Collecting additional information through public comment, additional ethics questions 

posed to potential advisory panel members and other appropriate means; 

f)	 In the case of non-Federal peer reviewers not on a FACA advisory panel (e.g., SAB, SAP 

or BOSC), providing them a copy of the peer review COI inquiry. This form is sent to 

each prospective peer reviewer by the Peer Review Leader (or contractor, in the case of 

contractor-run reviews) to advise them of the need to address COI issues prior to the 

actual review taking place. A follow-up contact with the Peer Review Leader (or 

contractor, in the case of contractor-run reviews) is then made to discuss any relevant 

issues. The Peer Review Leader then documents this effort in the peer review record; this 

includes a summary provided by the contractor documenting their inquiries and efforts. 

The Peer Review Leader should ensure that the peer review COI inquiry took place and that it 

appears in the peer review record. 
 Peer Review Handbook Page 70 

However, it should be strictly understood that Federal records maintained on individuals (defined 

as any item, collection, or grouping of information about an individual that is maintained by an 

agency, including, but not limited to, her or his education, financial transactions, medical history, 

and criminal or employment history and that contains his name, or the identifying number, 

symbol, or other identifying particular assigned to the individual, such as a finger or voice print 

or a photograph) are protected by the Privacy Act of 1974 (5 USC § 552a - Records maintained 

on individuals). A copy can be found at http://www.usdoj.gov/04foia/privstat.htm. This Act 

states that no agency shall disclose any record which is contained in a system of records by any 

means of communication to any person, or to another agency, except pursuant to a written 

request by, or with the prior written consent of, the individual to whom the record pertains. 

There are certain exceptions (e.g., pursuant to certain civil or criminal law enforcement activity). 

Thus, the peer review leader should ensure that any documentation in the peer review record 

does not contain information subject to the Privacy Act. Furthermore, Peer Review Leaders 

should ensure that any applicable EPA record-keeping procedures are followed (for details, see 

EPA’s National Records Management Program see h, http://www.epa.gov/records/). For 

information on Freedom of Information Act (FOIA) requests, please ttp://www.epa.gov/foia. If 

you have any questions, be sure to ask your appropriate Office of General Counsel and/or Office 

contract official(s). 

Established peer review groups such as the Science Advisory Board (SAB), the Scientific 

Advisory Panel (SAP) and the Board of Scientific Counselors (BOSC) provide useful models for 

addressing balance and ethics issues. Assistance in identifying conflicts of interest/appearance 

of a lack of impartiality and in providing an appropriate response can be obtained from the 

Deputy Agency Ethics Official (located in the Office of the General Counsel) as well as any of 

the Agency’s deputy ethics officials (DEOs) or Ethics Assistants. For assistance in evaluating 

conflicts of interest or appearance issues in a contractual situation, obtain the involvement of the 

Contracting Officer and the resources available within the Office of Acquisition Management. 

Of course, ethical issues do not necessarily arise merely because a peer reviewer knows 

something about the subject matter. In fact, experts with a stake in the outcome – and therefore a 

conflict or an appearance issue – may be some of the most knowledgeable and up-to-date experts 

because they have concrete reasons to maintain their expertise. Such experts could be used 

provided conflicts of interest are properly disclosed and appropriate waivers from the Office of 

Government Ethics are obtained or, in the case of an appearance of a lack of impartiality, proper 

disclosure is made. In some cases, however, the conflict or appearance issue may be so direct 

and substantial as to rule out a particular expert. For instance, a potential peer reviewer may 

have a client or employer with a direct financial stake in the particular specific party matter 

under review, such as a Federal grant or contract to the potential peer reviewer or his/her 

employer that relates to the matter under review or the potential peer reviewer’s or their 

company’s work on a specific chemical under review. However, review of a general 

methodology that applies to numerous chemicals would not necessarily raise such a concern. 
 Peer Review Handbook Page 71 

(Note: COI language should be made part of contracts/statements of work (SOW) or purchase 

orders (PO) – see Section 3.6.4.) 

A Peer Review Leader may also want to adopt measures that will prevent creation of conflicts as 

the peer review is underway. Any measures contemplated that involve a contractual action must 

be coordinated with the cognizant Contracting Officer. Some measures might include clauses in 

a contract or purchase order that require reviewers to receive advance approval on future work, 

or place limits on such work, while they are performing the current peer review. Note that at 

some level these types of measures will discourage experts from serving as peer reviewers. (See 

Section 3.6.4 for further information dealing with contracts and suggestions for appropriate 

management controls.) 

3.4.7 Are There Constraints to Selecting Peer Reviewers? 

Sometimes the schedule for a peer review is accelerated due to a court-ordered deadline or other 

time-sensitive requirements. In such cases, it is difficult, if not impossible to obtain external peer 

reviewers in time to conduct a full external peer review. It may even be impossible to conduct a 

small scale internal peer review using just a few individuals. Mechanisms for identifying and 

using a small number of peer reviewers should be developed so that quick, effective peer review 

can be included for even the most rapidly moving products. 

Another possible constraint involves confidential business information (CBI). There are 

different definitions and types of CBI, depending on the statute that governs your action (e.g., 

Toxic Substances Control Act). To evaluate certain Agency-generated studies properly, some 

peer reviewers may need access to CBI. However, unless the reviewers are Federal employees 

with CBI clearance, the Agency does not have the independent authority to disclose CBI to them. 

Therefore, whenever contemplating the use of outside peer reviewers, Agency staff should 

determine whether the reviewers will need access to CBI. If they do not have CBI clearance, the 

Office of the General Counsel should be consulted on whether it is practical to obtain the consent 

of CBI submitters to disclose the information to peer reviewers. 

Offices need to be aware of the requirements of the Federal Advisory Committee Act (FACA) 

when establishing peer review mechanisms (see Section 2.8). Federal advisory committees that 

are subject to chartering by the General Services Administration must hold meetings that are 

open to the public, and have balanced membership requirements. The Office of the General 

Counsel (OGC) should be consulted regarding the applicability of FACA to peer review panels. 

Occasionally, you may want to appoint a non-U.S. citizen to an advisory committee. Citizens of 

other countries may be appointed as representative members of an advisory committee. 

However, whether a non-U.S. citizen may be appointed as a special government employee (SGE) 

is a complicated question involving the Appropriations Act. 
 

 

 Peer Review Handbook Page 72 

Section 605 of the General Provisions of the Treasury and General Government Appropriations 

Act provides, with some exceptions, that no appropriations can be used "to pay the compensation 

of any employee of the US Government" unless the person is a U.S. citizen or has filed a 

declaration that they intend to become a U.S. citizen. There are a few exceptions to this 

restriction. One exception is for "nationals of those countries allied with the United States in a 

current defense effort." The appropriations restriction also provides that citizens of specific 

countries, refugees from certain countries, and students from China also are not subject to the 

restriction. If you want to hire a foreign national as an SGE, first contact OGC so that they can 

determine whether any exception applies to the person you are considering. Generally, to 

determine whether a country is allied with the U.S. in a current defense effort, OGC will consult 

with the Department of State. 

If the foreign nation in question does not have a qualifying defense relationship with the U.S. and 

no other exceptions apply to the individual, we may still hire the person as an SGE, but they 

would have to agree in writing to forgo any compensation. For an SGE serving without 

compensation, we may reimburse the individual for his/her expenses (travel and per diem), but 

we may not make payments for "home work time" or time spent at advisory committee meetings. 

The main point is to contact OGC if a foreign national is being considered for a FACA so that 

OGC can determine whether the person may be compensated. 

3.4.8 If State Employees Are Used as Peer Reviewers, Can EPA Pay Them for This 

Service? 

In some cases, this may be possible. First, the Peer Review candidate should determine if the 

State agency has a policy on whether its employees can perform this type of work, and whether 

the State policy will allow compensation. In most instances, EPA can also pay travel expenses 

(consult with your administrative staff for details). If the State person is not being paid for their 

peer review services, they must sign an agreement stating that they do not expect payment (see 

Section 3.6.1 – Gratuitous Services). 

3.4.9 Does the Agency Recognize Peer Reviews That External Parties Conduct 

and/or Pay For? 

There may be instances where parties external to EPA will want to conduct and/or pay for a peer 

review on a particular work product. This may look benign at first blush, but is a very complex 

and sensitive situation that can raise significant concerns for conflicts of interest or an 

appearance of a lack of impartiality for interested parties “paying” for a peer review of their own 

work product. While the Agency cannot prevent external parties from conducting and paying for 

a peer review, it is desirable that any such peer review is consistent with the intent of the 

Agency’s Peer Review Policy and implements the principles and guidance in this Handbook. If 

the external party submits their work product and accompanying peer review, the materials 

should be treated by the Agency as anything else submitted for the Agency’s evaluation (i.e., 
 

 

 Peer Review Handbook Page 73 

evaluation for scientific credibility and validity, as well as consistency with the provisions of the 

Peer Review Handbook). 

3.5 Materials for Peer Reviewers 

3.5.1 What Instructions Do You Give Peer Reviewers? 

The Peer Review Leader is responsible for ensuring that peer reviewers understand their 

responsibilities (see Sections 3.2 and 3.6 if a contract is involved): 

a) Advise the Agency of actual or potential organizational or personal conflicts of interest or 

other matters that would create the appearance of a lack of impartiality (see Section 

3.4.6); 

b)	 Provide written comments in specified format by the specified deadline that are 

responsive to the charge; 

c)	 Comply with any requests for not disclosing draft work products to the public; 

d)	 Inform on the planned extent of disclosure of names and attribution of comments. 

3.5.2 What Materials Should be Sent to Peer Reviewers? 

For a peer review to be successful, peer reviewers should receive several documents at the 

beginning of the process. Typically, the most important among these documents are the charge 

letter and the current work product. The charge letter describes what the peer reviewers are 

being asked to do, and should serve to focus and structure the review. The work product is, of 

course, the material being subject to peer review. 

Remember, documents should not be provided directly to a potential peer reviewer if that 

reviewer is going to be working under a contract or purchase order. In the case of a contract, the 

Agency provides the work product with associated background material to be peer reviewed to 

the prime contractor who in turn distributes these documents to the peer reviewers. In the case of 

a purchase order, the “charge or statement of work” must be part of the PO (purchase order) and 

the provision of any documents should be coordinated with the purchasing agent handling the 

order. 

Essential documentation for each peer reviewer includes: 

1) A current copy of the work product to be peer reviewed with associated background 

material. The work product should be of the best possible scientific/technical quality to 

ensure an adequate and useful peer review; 
 Peer Review Handbook Page 74 

2)	 A clear charge or statement of work, seeking informed comment on identified specific 

issues to properly focus the efforts of the peer reviewers and ensure that their individual 

efforts can be compared or contrasted; 

3)	 Information concerning the process that you use for the peer review, including the due 

date of reviewer comments, the format of those responses, and a point of contact in case 

the peer reviewer has questions. Responses should be written and submitted to the Peer 

Review Leader by an agreed upon deadline. In certain rare cases, oral commentary may 

be sufficient; however, in such cases, a follow-up written response for the record should 

be made; 

4) For highly influential scientific assessments, copies of significant public comments raised 

on scientific issues; 

5)	 In some cases, Agency materials being peer reviewed will be available to the public, even 

if they are marked as drafts. For example, all materials reviewed by the SAB and the 

FIFRA SAP are available. Agency managers may also decide that a broad accessibility 

has benefits for the Agency. Offices should include a disclaimer on draft work products 

that are submitted for peer review (See Section 1.3.4). In other cases, confidentiality 

needs to be maintained. In these cases, each peer reviewer should be informed of the 

need for confidentiality with regard to the release of Agency products that are stamped as 

"DRAFT" or "DRAFT - Do Not Cite, Quote, or Release." Premature release of draft 

Agency products, views, or positions may be inappropriate and can be damaging to the 

credibility of the Agency or the peer reviewer. Although it may not have legal effect, 

such language should be included in the charge to the peer reviewers when necessary. 

Other mechanisms to use in discouraging premature release include a disclaimer that 

appears in a separate section at the front of the document and creating the document with 

watermarks clearly delineating DRAFT status (or a header or footer that states DRAFT 

status) on every page. In addition, in any solicitation for peer reviewers, the necessity for 

confidentiality and the non-release of materials should be emphasized. 

Useful, but not critical materials that may be sent to peer reviewers include: 

1)	 The name, address, and phone and fax numbers, and/or e-mail address of all peer 

reviewers working on the specific review; 

2) A bibliography and/or any particularly relevant scientific articles from the literature; 

3) A work product that has line numbering added in the margin for ease in providing and 

referencing comments. 

Peer Reviewers should be given what is necessary to complete their task – they should not be 

overburdened with excess material. 
 Peer Review Handbook Page 75 

3.5.3 How Closely can EPA Interact with Peer Reviewers During the Review? 

a)	 When EPA Conducts the Peer Review - The Peer Review Leader normally has 

administrative contacts with the reviewers during the development and conduct of the 

peer review. In some cases (e.g., SAB peer review), peer reviewers may also receive a 

briefing on the product to be peer reviewed. Otherwise, the Peer Review Leader and 

other EPA staff should not contact the reviewers during the course of the review. Such 

contact can lead to perceived conflicts or inappropriate direction that could compromise 

the independence of the review. 

b)	 When the Contractor Conducts the Peer Review - If peer review is conducted under a 

contract mechanism, EPA should limit direct contact to the prime contractor’s designated 

representative and should not have general contact and direction to the contractor’s staff 

or peer reviewers (sub-contractors). Note, when a peer review is conducted under a 

contract, there are constraints where EPA staff are prohibited from contacting peer 

reviewers to avoid personal services arrangements. Personal services contracts exist 

when the nature of the relationship between the contractor and EPA can be characterized 

as an employer-employee relationship. Any communications with peer reviewers should 

be coordinated through the prime contractor. 

3.6 Peer Review Services 

A range of peer review services are available to the Agency including internal, external 

(gratuitous services, contracts, purchase order), and Special Government Employee (SGE) 

mechanisms. The mechanism selected is generally based on the nature of the scientific or 

technical work product. 

Peer review services are “advisory and assistance services”, as defined in Federal Acquisition 

Regulation (FAR) 37.201. These types of services require special approvals and management 

oversight. See Figure 5, Item B Management Approvals, in Chapter 7 of the Contracts 

Management Manual (CMM) for current approval levels. The CMM is available on the intranet 

at http://intranet.epa.gov/oamintra/policy/cmm.pdf. 

3.6.1 What are Gratuitous Services for Peer Review? 

The provision of peer review products or services to EPA without compensation is provided as 

so-called “gratuitous” services. If a person wishes to perform peer review services for EPA 

without compensation, EPA must ask them to sign an agreement whereby the person agrees to 

provide the prescribed peer review services as gratuitous services, with no expectation of 

receiving compensation for these services from EPA. An agreement, such as EPA Form 3100-14 

or a gratuitous services contract, must be executed because the Antideficiency Act (31 USC § 

1342) prohibits the Agency’s acceptance of “voluntary” services. For a copy of EPA Form 

3100.14, see http://intranet.epa.gov/oas/fmsd/forms/f2.htm. “Voluntary services” are services 
 Peer Review Handbook Page 76 

provided to EPA without an agreement in advance that such services are provided at no cost to 

EPA. Note that persons cannot waive compensation (i.e., agree to provide gratuitous services) 

for which there is a statutory right to payment, unless a law permits the waiver. For situations 

concerning State employees, see Section 3.4.9. Examples of laws that permit services without 

compensation are 5 USC § 3109 for experts or consultants and 5 USC § 3111 for student 

volunteers. If you are dealing with such gratuitous services, contact the appropriate OGC 

attorney for advice. 

3.6.2 Can You Use a Contract to Obtain Peer Review Services? 

The Agency may obtain peer review services through a contract or purchase order. Typically, 

peer review services would be available under a “mission contract,” i.e., a contract with a broad 

scope covering a variety of services. It is also possible to have a contract or purchase order 

solely for peer reviews. 

A contract is awarded if the cost is over $100,000. If the cost is under $100,000, then a purchase 

order is issued. For assistance in preparing the necessary pre-award contract documents, consult 

Chapter 7 of the CMM. 

For assistance in preparing simplified acquisition packages for purchase orders, the Office of 

Acquisition Management (OAM) has a guide called SAME: Simplified Acquisition Made Easy, 

which is available on the intranet at http://intranet.epa.gov/oamintra/policy/index.htm. 

3.6.3 How Do You Write a Statement of Work for Contracts? 

For general assistance in preparing the Statement of Work (SOW), consult Chapter 11 of the 

CMM. The SOW should clearly specify that the contractor is responsible for preparing peer 

review evaluations and set forth guidelines for the peer review of scientific or technical 

documents. The contractor may perform the peer review with in-house staff, subcontractors or 

consultants. Any guidelines for performing peer reviews to ensure soundness and defensibility 

should be developed by the Program Office and made part of the contract. The contractor would 

then ensure that the peer reviews adhere to the guidelines. 

The SOW should include the list of questions that EPA wants the contractor to include in the 

charge to the peer reviewers. While the contractor will be the one that will prepare and send the 

formal charge to the peer reviewers, EPA should provide the list of questions to the contractor 

(see Section 3.2 for general discussion on charge to peer reviewers). 

The SOW cannot simply define the role of the prime contractor as arranging for the services of 

others to perform peer reviews and logistics for meetings. Unless the prime contractor is clearly 

tasked with responsibility for performing peer reviews, individual peer reviewers' fees and 

associated travel expenses are not payable under the contract. 
 Peer Review Handbook Page 77 

EPA may pay for the reviewer's comments or evaluation, and also for attendance at a meeting 

with the Agency and other reviewers to discuss the results of the peer review. If the SOW calls 

for the preparation of comments or an evaluation, and specifies a meeting with the Agency and 

other peer reviewers to discuss the results of the peer review, payment is appropriate. The peer 

reviewer's attendance at the meeting would then be part of contract performance. 

Example statements of work are found in Appendix F. 

3.6.4 What are Some Management Controls for Contracts? 

Management controls ensure several things: 

1) the contractor does not perform inherently governmental activities (IGA); 

2) the contractor’s work is free from conflicts of interest; 

3) if provided to the contractor, confidential business information or sensitive information is 

appropriately safeguarded, and; 

4) improper relationships with contractor employees and subcontractors are avoided. 

Agency and Federal acquisition regulations prohibit contractors from performing IGA. The 

Office of Management and Budget Circular A-76, revised May 29, 2003, defines “inherently 

governmental activities” as activities that are so intimately related to the public interest as to 

mandate performance by government personnel. These activities require the exercise of 

substantial official discretion in the application of government authority and/or in making 

decisions for the government. 

The Agency cannot award contracts for IGA; however, certain functions, such as those involved 

in peer review, may cross over into IGA if not properly managed. With peer review services, the 

Agency is seeking only a contractor's recommendations, advice or analysis of a document, not a 

determination of what the policy should be. Determining Agency policy is an IGA. Agency 

officials make the official Agency decision regarding acceptability and/or quality of the 

document. To ensure that Agency officials are not improperly influenced by recommendations 

in the peer review, the contract should include management controls. One possible control 

would be to direct the peer reviewers to submit with their evaluations or comments a description 

of the procedures used to arrive at their recommendations, a summary of their findings, a list of 

sources relied upon, and make clear and substantiate the methods and considerations upon which 

their recommendations are based. To the extent possible, the contract should set forth any 

guidelines or criteria for performance of the peer review. Agency officials should document 

their evaluations of the quality and validity of the peer review. You should ensure that there is a 

clear record of their review of the contractor's work and that Agency personnel made the final 
 Peer Review Handbook Page 78 

decisions. Such records of review could include notes from reviews of draft and final documents 

by Agency personnel, minutes from progress meetings with contractors, and so forth. 

Conflict of Interest (COI) – Another important factor is that the objectivity of the peer review 

should not be improperly influenced or undermined by the contractor performing the review. To 

identify and avoid or mitigate actual or potential COI, the contract should include controls. 

Inclusion of Agency-developed individual and organizational COI clauses in the contract or 

purchase order is critical for peer review services. Usually, the EPA contracting officer (CO) 

will include COI solicitation provisions and contract clauses as a matter of course without 

involvement by the EPA Project Officer. As a safeguard, the Project Officer should: 

a) Highlight the conflict of interest requirements in the Statement of Work (SOW) for the 

procurement of the peer review services; 

b)	 Review the solicitation/contract to make sure that the appropriate conflict of interest 

clauses have been included, particularly EPAAR clause 1552.209-73, Notification of 

Conflicts of Interest Regarding Personnel; 

c) Work with the CO to develop contract-specific language regarding the peer review to 

assist the contractor with identifying actual or potential conflicts of interest that might 

impair the objectivity of peer reviewers. For example, a contractor may be advised to 

consider the following questions and issues when determining if a proposed peer 

reviewer may have an actual or potential conflict of interest: 

1) The sources and nature (obtained from a brief description of the work) of any 

compensated and non-compensated employment of the panel member and their 

spouse, including any government service, for the preceding two years. 

2) The sources of research support and project funding, including from any 

government source, for which the panel member served as the Principal 

Investigator, Significant Collaborator, Project Manager or Director during the 

preceding two years. For the panel member’s spouse, a general description of 

research and project activities in the preceding two years. 

3) The compensated consulting activities of the panel member during the preceding 

two years, including the names of clients if compensation provided 15% or more 

of annual compensation. For the panel member’s spouse, a general description of 

consulting activities for the preceding two years. 

4) The sources of compensated expert witness activities of the panel member and a 

brief description of the issue and testimony during the preceding two years. For 

the panel member’s spouse, a general description of expert testimony provided in 

the preceding two years. 
 

 

 

 Peer Review Handbook Page 79 

5) The assets, including stocks, bonds, real estate, business, patents, trademarks, and 

royalties, for the panel member, their spouse and dependent children. 

Specifically, the financial holdings that collectively had a fair market value 

greater than $15,000 at any time during the preceding two year period (excluding, 

for example, well-diversified mutual funds, money market funds, treasury bonds 

and personal residences). 

6) The liabilities over $10,000 owed by the panel member, their spouse, and 

dependent children at any time in the preceding one year (excluding, for example, 

a mortgage on a personal residence, home equity loans, automobile and consumer 

loans). 

7) A brief description of any public statements and/or positions of the panel member 

on, or closely related to, the matter under review. 

8) A brief description of any previous involvement of the panel member with the 

development of the document (or review materials) the individual has been asked 

to review (including previous peer reviews). 

9) A brief description of any other information that might reasonably raise a question 

about an actual or potential personal conflict of interest or bias including any 

financial benefit that might be gained by the panel member (or anyone whose 

interests are imputed to the panel member) as a result of the outcome of the 

review. 

Confidential Business Information (CBI)/Privacy Act Protected Information and Other 

Sensitive Information – When peer reviewers are not employees of the United States 

Government, it is unlikely that EPA will have authority to give reviewers access to confidential 

business information in the absence of consent for such disclosure by the CBI submitter. 

Therefore, all documents provided to non-Federal reviewers must be screened for information 

claimed as CBI. Even where business information has not been explicitly claimed as CBI, if it is 

of a kind where the submitter might be expected to object to its release, prior to release the 

submitter must be asked whether it wishes to assert a claim, unless the submitter has previously 

been informed that failure to assert a CBI claim may result in disclosure without notice. If the 

contractor should have access to CBI for your peer review, notify the CO so that the appropriate 

clauses can be included in the contract or purchase order. These clauses will clearly identify any 

required procedures or processes prior to release of any protected information, including any 

requirements for confidentiality agreements, as well as limits on use and disclosure of the data by 

contractor personnel. 

Personal services – Unless your organization has statutory authority to engage in personal 

service contracts, avoid treating contractor employees as Agency employees. For example, 

deciding who to hire, supervising, and assigning tasks to contractor employees are personal 
 Peer Review Handbook Page 80 

services. For additional information, program officials should consult EPA Order 1901.1A, Use 

of Contractor Services to Avoid Improper Contracting Relationships. It is very important that all 

Agency employees follow this order when interacting with contractors. The Order is available 

on the intranet at http://epawww.epa.gov/oamintra/policy/eo19011a.pdf. 

To avoid these improper relationships, program officials should write well-defined SOWs. The 

SOWs should set forth in detail a description of the work to be performed independently, 

including the manner in which it will be evaluated. The SOW should set forth what work is to be 

performed not how the work is to be performed. Technical direction may be used to clarify 

ambiguous provisions to ensure efficient and effective contractor performance, and is not 

considered supervision or assignment of tasks. 

3.6.5 Can You Identify and/or Select Peer Reviewers When Using a Contract? 

The prime contractor is responsible for selecting who will perform the peer review (EPA’s doing 

so may invoke FACA; see Section 2.8), whether these individuals work directly for the 

contractor or are consultants or subcontractors to the prime contractor. Interfering in this process 

is a violation of Federal and Agency acquisition regulations. In accordance with FAR clause 

52.244-2 Subcontracts, contractors may be required to obtain Contracting Officer (CO) consent 

or approval of subcontractors and the CO generally seeks the input of the work assignment 

manager (WAM – the WAM may be the Peer Review Leader in many cases) before approving 

the use of subcontractors. 

EPA can establish criteria for the sort of individuals that might participate on a peer review 

panel. However, the Agency should not be involved in the selection of individual peer 

reviewers, and should avoid commenting on the contractor’s selection of peer reviewers other 

than to determine whether the panel, once selected, meets the criteria established. EPA may 

identify a pool of qualified subcontractors and consultants to the prime contractor (listed in 

alphabetical order). If a list is provided, note on the list that this is a suggested list and other 

qualified candidates may exist who are not on the list. This is to prevent the impression that the 

prime contractor can only choose someone on our list. The prime contractor is required to 

include a conflicts of interest clause substantially similar to the conflicts of interest clause in the 

primary contract in its subcontract to the peer reviewer. 

3.6.6 How is Travel Handled with Contracts or Purchase Orders? 

Funds obligated on a contract or purchase order are available to pay for the costs of producing 

the peer review including the travel costs and fee of the peer reviewer. 

EPA acquires peer reviews through simplified acquisitions issued directly to peer reviewers or 

through contracts with companies, which provide the peer review services. By issuing a 

purchase order or awarding a contract for peer review services, EPA may pay not only for the 

peer review services/comments, but also for participation in a meeting with the Agency and other 
 Peer Review Handbook Page 81 

reviewers to discuss comments. The scope of work of the contract must require the contractor or 

individual peer reviewer, as appropriate, to provide peer review services and indicate whether the 

contractor or peer reviewer will be required to discuss a specific peer review work product with 

the Agency and/or with other peer reviewers. Participation in a meeting to discuss a peer review 

work product would then be part of the contract's performance. Thus, the contract may serve as 

the mechanism to pay for peer review services and associated travel expenses to provide 

comments to EPA. 

3.6.7 How is Travel Handled with Special Government Employees? 

The term Special Government Employee (SGE) is defined in 18 USC § 202(a) as an officer or 

employee of an agency who performs temporary duties, with or without compensation, for not 

more than 130 days in a period of 365 days, either on a full-time or intermittent basis. 

Travel and per diem expenses of experts hired as SGEs for peer review may only be paid through 

the issuance of invitational travel orders (5 USC § 5703). These invitational travel and per diem 

expenses should be charged to an appropriate EPA travel account. The Federal Travel 

Regulations govern the invited travelers reimbursement. 

Members of the SAB, SAP, and other FACA advisory committees are often classified as SGEs. 

It is not appropriate to reimburse travel or per diem expenses of advisory committee members 

(SGEs) through a contract. 
 Peer Review Handbook Page 82 

4. Completing A Peer Review 

4.1 Overview 

Performance of the formal peer review of the work product is not the final stage in the product’s 

development. Rather, it is an important stage in developing the work product, with the final 

version (with all comments addressed) representing the true end of the peer review. As a result, 

the peer review process closes with four major activities: 

a) Evaluating peer review comments and recommendations 

b) Utilizing peer review comments for completing the final document, 

c) Organizing and maintaining a record of the peer review, and 

d) Completing the peer review record in the Science Inventory. 

Careful attention to all of these elements, singly and together, assures a credible and transparent 

peer review process. Conversely, inattention can nullify the peer review effort. A well-planned 

peer review applied to a reasonable quality starting work product, followed by a responsible 

utilization of peer review suggestions in the final product assures a credible and defensible 

product for use in Agency decision-making. 

The peer review of a work product is not complete until the peer review comments are 

incorporated into the final version, or reasons are stated why such comments are not to be 

incorporated. However, for the purposes of documentation to the Agency’s Science Inventory 

only, the work product can be designated in the Science Inventory as having completed peer 

review once the peer review report is available on-line. 

The peer review record is complete only when it contains a copy of the final work product (when 

there is one) that addresses the peer review comments and a copy of the documentation outlining 

why comments were not incorporated, if any were not incorporated . (See Sec. 2.5.3 for a list of 

the minimum documentation for the peer review record.) 

4.2 Final Work Product 

4.2.1 How Do You Incorporate Peer Review Comments into the Final Work 

Product? 

The Peer Review Leader should carefully evaluate and analyze all peer review comments and 

recommendations. As discussed earlier, a carefully crafted charge to the peer reviewers 
 Peer Review Handbook Page 83 

simplifies organizing and analyzing comments. Also, any other issues that are raised should be 

identified and evaluated. 

The validity and objectivity of the comments should be evaluated. Analyses may include 

consultation with other experts and staff within the Office and Agency. Adequate documentation 

is needed to show that comments are accepted or rejected. The documentation can be brief, but 

should address the legitimate, valid comments, whether accepted for incorporation in the final 

work product or not. The peer review record should contain a document describing the Agency’s 

response to the peer review comments. The Agency’s response to the peer review report for 

highly influential scientific assessments should be posted on the Science Inventory. 

The Peer Review Leader should brief the Decision Maker (including all appropriate managers in 

the Peer Review Leader’s chain of command) on the charge, profile of peer reviewers, the peer 

review comments, and provide a proposal on how to address the peer review comments. It is the 

responsibility of the Peer Review Leader to obtain Decision Maker approval of the approach to 

addressing the peer review comments. The Peer Review Leader should clearly identify for the 

Decision Maker any major peer review comments that will not be accepted and why, as well as 

any controversial comments that should be resolved. 

Comments that have significant impact on time, budgetary, or resource needs are particularly 

important and should be evaluated in consultation with 

management. These comments may lead to allocation of The peer review is not complete until the additional resources and a revised schedule for the peer review comments are incorporated completion of the work product if appropriate for the into the final work product. final work products intended use. 

4.2.2 How Might Peer Review Comments 

Impact Your Work Product?
 

Peer review, when appropriate, enhances the quality of the information EPA disseminates by 

ensuring the information that governs EPA’s mission is reliable and accurate (objective) and is 

appropriate for the intended use (utility). A variety of changes to a work product may result 

from the comments provided during peer review: 

a)	 Peer review comments and recommendations may entail significant impacts on the 

planned project schedule, budget, or other resource needs. Management decisions to 

adjust one or more of these areas may be appropriate. 

b)	 The substantive issues or concerns expressed by peer reviewers may suggest that wider 

scientific and technical consultation is needed to ensure the adequacy of the work product 

relative to its intended use. If the Agency agrees with the reviewers, additional resources 

and an extended delivery schedule may be necessary. 
 Peer Review Handbook Page 84 

c) The peer review comments and recommendations on a final product may provide a basis 

for bringing the associated project to closure. At times the Agency and its partners and 

stakeholders may not be able to come to consensus about a scientific or technical aspect 

of a work product. The scientific or technical issue in question can be noted in the charge 

to the reviewers as an area for which the Agency is specifically seeking peer review. 

4.2.3 What Should the Final Work Product Say About the Peer Review Process? 

If the product has been peer reviewed, you should describe the peer review in the document. 

Frequently, this will be part of a description of the process of developing the product along with 

descriptions of other public participation processes. It can be brief and does not need to describe 

the process or discuss the peer-review comments in great detail. The description can be included 

in an introduction, preamble, or appendix. For influential scientific information and highly 

influential scientific assessments that support rulemaking, the peer review should be discussed in 

the preamble. 

When there are significant peer-review comments, and particularly if they are not being 

accepted, the document should generally discuss the issue and describe the reasons for the 

Agency’s choices in the appropriate sections of the document. The level of detail that is 

provided is a matter of judgment and should reflect the importance and degree of controversy of 

the issue. 

If a scientific or technical work product has not been peer-reviewed, this fact should be noted in 

the document, perhaps in an introduction or description of its scope. This section should briefly 

indicate the reasons that peer review was not conducted. 

Derivative products of scientific or technical products (such as fact sheets, press releases, and 

brochures) do not need to discuss whether the underlying products were peer reviewed. 

4.2.4 Can the Identity of Peer Reviewers be Kept Anonymous? 

Generally, no. At a minimum, the names and affiliations of peer reviewers should be listed in 

the peer review report for influential scientific information and highly influential scientific 

assessments. Depending on the peer review process used, the reviewers names and affiliations 

may be made available to the public prior to the review commencing. Release of any reviewer 

information retrieved by a personal identifier must be performed in accordance with the Privacy 

Act, 5 USC § 552a as amended, and as interpreted in OMB implementing guidance, 40 FR 

28,948 (July 9, 1975). 

Though the identity of reviewers becomes known at some point in the peer review process, the 

attribution of specific comments to any given reviewer is kept confidential to the extent possible. 

In the ordinary course of events, you can often discuss comments received without attributing the 

comments to a specific reviewer. However, if a matter goes to litigation, the litigating parties 
 Peer Review Handbook Page 85 

can discover the names of anyone who contributed to a Federal product, including peer 

reviewers. Also, in most cases, though specific comments are not attributed to specific peer 

reviewers, the names and affiliations of the peer reviewers are included in the Peer Review 

Report, which is included in the Science Inventory. Therefore, it is not possible to totally shield 

peer reviewers. As noted previously, peer reviewers should be aware of EPA plans for releasing 

their names, credentials, and the extent of attribution of comments. 

If a peer reviewer requests anonymity at the outset of the peer review, the Peer Review Leader 

should inform the peer reviewer of the above possible eventualities. The Agency will in the 

ordinary course of events attempt to maintain the confidentiality of the peer reviewers and their 

attributable comments from public disclosure, but it is recognized in many instances, for 

example open public meetings, litigation, and Agency responses to Freedom of Information Act 

(FOIA) requests, this can’t be assured. Remember, the Agency is committed to working with the 

fullest possible transparency to the public (except where statutorily constrained, such as with 

confidential business information). It is recognized that this may be a deterrent to possible peer 

reviewers, but this is a reality that has to be understood. 

4.3 Completing the Peer Review Record 

4.3.1 How Do You Complete the Peer Review Record? 

Once the Peer Review Leader has completed the peer review and the final work product (where 

one is prepared), the peer review record is brought up to date and then archived according to that 

organization’s procedures (see Section 4.3.2). The peer review record should be indexed and 

maintained in an organization’s archive (repository). The location of the peer review record 

should be readily identifiable so interested parties can locate and obtain materials easily and 

quickly. The peer review record should be placed in any associated established public docket, if 

so required, in addition to the organizational archive. As a courtesy, a copy of the revised work 

product may be sent to the peer reviewers for information. 

The Peer Review Leader should collect the materials identified in Section 2.5.3 and submit them 

for archiving. Finally, the Agency Science Inventory entry for the project should be completed 

and updated, including attachment of electronic copies of the documents. 

4.3.2 Where Should the Peer Review Records be Kept, and for How Long? 

Though the Science Inventory will become the primary public active record and archive for the 

peer review record, the Peer Review Leader is responsible for maintaining the on-site record. 

During the active conduct of the peer review, the Peer Review Leaders maintain the peer review 

record themselves until the peer review is totally completed. Minimally, the file should be 

maintained until one year after the completed peer review is reported in the next annual 

reporting. After that, the peer review record should be maintained for a “reasonable period of 
 

 

 Peer Review Handbook Page 86 

time.” Establishment and maintenance of the archive where the peer review records ultimately 

reside are an organization’s responsibility (i.e., not that of an individual program manager or 

Peer Review Leader). 

Generally, to allow flexibility, individual Offices and Regions should decide the appropriate 

level of organizational responsibility and how to ensure the record will meet “routinely 

available.” The peer review record may be kept with other records relating to the overall project, 

as long as it is easily and separately identifiable. The peer review record should be maintained in 

accordance with the Agency’s record-keeping schedule for such records. One long term 

archiving mechanism may be the formal archiving at the Federal Records Center; the Science 

Inventory will also maintain an electronic copy of the record. 

4.3.3 Is Information Regarding a Peer Review Subject to Release under FOIA? 

Yes, it is subject to release if EPA receives a Freedom of Information Act (FOIA) request, unless 

the peer review information meets the criteria for an exemption under the FOIA. (See 

http://www.epic.org/open_gov/foia/us_foia_act.html). 
 

 

 Peer Review Handbook Page A-1 

Appendix A 

EPA’s Peer Review Policy
 

January 31, 2006 
 Peer Review Handbook Page A-2 
 

 

 Peer Review Handbook Page A-3 

MEMORANDUM 

SUBJECT: Peer Review Program 

TO: Assistant Administrators 

General Counsel 

Inspector General 

Associate Administrators 

Regional Administrators 

Staff Office Directors 

We have made tremendous strides in improving our peer review program at EPA 

since the Agency’s Peer Review Policy was reaffirmed in 1994. Today I am updating the 

Peer Review Policy to emphasize the critical role that peer review plays in our efforts to 

ensure that EPA’s decisions rest on sound, credible science and data (see attached policy 

statement). 

Peer review at EPA takes several different forms, ranging from informal 

consultations with Agency colleagues who were not involved in developing the product 

to the formal, public processes of the Science Advisory Board (SAB) and the FIFRA 

Scientific Advisory Panel (SAP). In any form, peer review assists EPA’s work by 

bringing independent expert experience and judgment to bear on issues before the 

Agency to the benefit of the final product. 

In 1994 the Science Policy Council (SPC) and its Steering Committee were asked 

to undertake an initiative to ensure that EPA has a comprehensive Agency-wide program 

for implementing its Peer Review Policy. I commend the SPC for its diligence and 

success in meeting this objective. The SPC has made substantial improvements in the 

Peer Review Handbook, sponsored training of Agency managers and staff in peer review 

procedures, identified scientific and technical work products that merit peer review, and 

developed a publicly available data base of the peer review activities across the Agency. 

EPA has a strong and well-recognized peer review program as a direct result of these 

efforts. 
 Peer Review Handbook Page A-4 

In 2004 the Office of Management and Budget (OMB) issued a “Final 

Information Quality Bulletin for Peer Review” that contains provisions for peer review at 

all federal agencies. The OMB Bulletin applies to influential scientific information and 

highly influential scientific assessments. The SPC has updated the Agency's Peer 

Review Handbook, in part to incorporate the provisions of the OMB Bulletin, and to 

reflect the experience gained from implementing the program over the last decade. 

I ask that you continue to implement fully the provisions of our Peer Review 

Policy, and I expect the Science Policy Council to continue its role in overseeing and 

strengthening EPA’s peer review program. We must ensure that our decisions are based 

on the highest quality, peer-reviewed scientific and technical information. 

Attachment 

cc: Science Policy Council 

Science Policy Council Steering Committee 
 Peer Review Handbook Page A-5 

PEER REVIEW AND PEER INVOLVEMENT 

AT THE U.S. ENVIRONMENTAL PROTECTION AGENCY
 

This document establishes the policy of the United States Environmental Protection 

Agency (EPA) for peer review of scientifically and technically based work products, including 

economic and social science products, that are intended to inform Agency decisions. Peer 

review, a form of peer involvement, is one process through which EPA staff augment their 

capabilities by inviting independent subject-matter experts to provide objective evaluation of the 

work product. 

PEER REVIEW 

EPA strives to ensure that the scientific and technical bases of its decisions meet two 

important criteria: (1) they are based upon the best current knowledge from science, engineering, 

and other domains of technical expertise; and (2) they are credible. Peer review, a process based 

on the principles of obtaining the best technical and scientific expertise with appropriate 

independence, is central to sound science and helps the Agency meet these important criteria. 

Peer review occurs when scientifically and technically based work products are evaluated by 

relevant experts who were not involved in creating the product. Properly applied, peer review 

not only enriches the quality of work products but also adds a degree of credibility that cannot be 

achieved in any other way. Furthermore, peer review early in the development of work products 

in some cases may conserve future resources by steering the development along the most 

efficacious course. 

Peer review generally takes one of two approaches: 

• Internal, in which the reviewers are independent experts from inside EPA. 

• External, in which the reviewers are independent experts from outside EPA. 

POLICY STATEMENT 

Peer review of all scientific and technical information that is intended to inform or 

support Agency decisions is encouraged and expected. Influential scientific information, 

including highly influential scientific assessments, should be peer reviewed in accordance with 

the Agency’s Peer Review Handbook. All Agency managers are accountable for ensuring that 

Agency policy and guidance are appropriately applied in determining if their work products are 

influential or highly influential, and for deciding the nature, scope, and timing of their peer 

review. For highly influential scientific assessments, external peer review is the expected 

procedure. For influential scientific information intended to support important decisions, or for 

work products that have special importance in their own right, external peer review is the 

approach of choice. Peer review is not restricted to the nearly final version of work products; in 

fact, peer review at the planning stage can often be extremely beneficial. 
 

 

 Peer Review Handbook Page A-6 

LEGAL EFFECT 

This policy statement does not establish or affect legal rights or obligations. Rather, it 

confirms the importance of peer review where appropriate, outlines relevant principles, and 

identifies factors Agency staff should consider in implementing the policy. On a continuing 

basis, Agency management is expected to evaluate the policy as well as the results of its 

application throughout the Agency and undertake revisions as necessary. Therefore, the policy 

does not stand alone; nor does it establish a binding norm that is finally determinative of the 

issues addressed. 

IMPLEMENTATION 

The Science Policy Council is responsible for overseeing Agency-wide implementation 

of this policy, including: promoting consistent interpretation; assessing Agency-wide progress; 

developing recommendations for revisions of the policy as necessary; and issuing the Peer 

Review Handbook, which provides additional information and procedures on implementing this 

policy. Assistant Administrators, Regional Administrators, and other senior managers remain 

ultimately responsible for ensuring the appropriate application of Agency policy and guidance in 

identifying work products subject to peer review, determining the type and timing of such 

review, documenting the process and outcome of each peer review, ensuring that the Science 

Inventory is kept current, and otherwise implementing the policy within their organizational 

units. 

The policy is effective immediately. 
 Peer Review Handbook Page B-1 

Appendix B 

OMB Information Quality Bulletin for 

Peer Review and Preamble 
 

 

 

 

 

 Peer Review Handbook Page B-2 

OFFICE OF MANAGEMENT AND BUDGET 

Final Information Quality Bulletin for Peer Review 

INTRODUCTION 

This Bulletin establishes that important scientific information shall be peer reviewed by qualified 

specialists before it is disseminated by the federal government. We published a proposed 

Bulletin on September 15, 2003. Based on public comments, we published a revised proposal 

for additional comment on April 28, 2004. We are now finalizing the April version, with minor 

revisions responsive to the public’s comments. 

The purpose of the Bulletin is to enhance the quality and credibility of the government’s 

scientific information. We recognize that different types of peer review are appropriate for 

different types of information. Under this Bulletin, agencies are granted broad discretion to 

weigh the benefits and costs of using a particular peer review mechanism for a specific 

information product. The selection of an appropriate peer review mechanism for scientific 

information is left to the agency’s discretion. Various types of information are exempted from 

the requirements of this Bulletin, including time-sensitive health and safety determinations, in 

order to ensure that peer review does not unduly delay the release of urgent findings. 

This Bulletin also applies stricter minimum requirements for the peer review of highly influential 

scientific assessments, which are a subset of influential scientific information. A scientific 

assessment is an evaluation of a body of scientific or technical knowledge that typically 

synthesizes multiple factual inputs, data, models, assumptions, and/or applies best professional 

judgment to bridge uncertainties in the available information. To ensure that the Bulletin is not 

too costly or rigid, these requirements for more intensive peer review apply only to the more 

important scientific assessments disseminated by the federal government. 

Even for these highly influential scientific assessments, the Bulletin leaves significant discretion 

to the agency formulating the peer review plan. In general, an agency conducting a peer review 

of a highly influential scientific assessment must ensure that the peer review process is 

transparent by making available to the public the written charge to the peer reviewers, the peer 

reviewers’ names, the peer reviewers’ report(s), and the agency’s response to the peer reviewers’ 

report(s). The agency selecting peer reviewers must ensure that the reviewers possess the 

necessary expertise. In addition, the agency must address reviewers’ potential conflicts of 

interest (including those stemming from ties to regulated businesses and other stakeholders) and 

independence from the agency. This Bulletin requires agencies to adopt or adapt the committee 

selection policies employed by the National Academy of Sciences (NAS)14 when selecting peer 

reviewers who are not government employees. Those that are government employees are subject 

to federal ethics requirements. The use of a transparent process, coupled with the selection of 

14 National Academy of Sciences, “Policy and Procedures on Committee Composition and Balance and Conflicts of 

Interest for Committees Used in the Development of Reports,” May 2003: Available at: 

http://www.nationalacademies.org/coi/index.html. 
 

 

 

 Peer Review Handbook Page B-3 

qualified and independent peer reviewers, should improve the quality of government science 

while promoting public confidence in the integrity of the government’s scientific products. 

PEER REVIEW 

Peer review is one of the important procedures used to ensure that the quality of published 

information meets the standards of the scientific and technical community. It is a form of 

deliberation involving an exchange of judgments about the appropriateness of methods and the 

strength of the author’s inferences.15 Peer review involves the review of a draft product for 

quality by specialists in the field who were not involved in producing the draft. 

The peer reviewer’s report is an evaluation or critique that is used by the authors of the draft to 

improve the product. Peer review typically evaluates the clarity of hypotheses, the validity of the 

research design, the quality of data collection procedures, the robustness of the methods 

employed, the appropriateness of the methods for the hypotheses being tested, the extent to 

which the conclusions follow from the analysis, and the strengths and limitations of the overall 

product. 

Peer review has diverse purposes. Editors of scientific journals use reviewer comments to help 

determine whether a draft scientific article is of sufficient quality, importance, and interest to a 

field of study to justify publication. Research funding organizations often use peer review to 

evaluate research proposals. In addition, some federal agencies make use of peer review to 

obtain evaluations of draft information that contains important scientific determinations. 

Peer review should not be confused with public comment and other stakeholder processes. The 

selection of participants in a peer review is based on expertise, with due consideration of 

independence and conflict of interest. Furthermore, notice-and-comment procedures for agency 

rulemaking do not provide an adequate substitute for peer review, as some experts -- especially 

those most knowledgeable in a field -- may not file public comments with federal agencies. 

The critique provided by a peer review often suggests ways to clarify assumptions, findings, and 

conclusions. For instance, peer reviews can filter out biases and identify oversights, omissions, 

and inconsistencies.16 Peer review also may encourage authors to more fully acknowledge 

limitations and uncertainties. In some cases, reviewers might recommend major changes to the 

draft, such as refinement of hypotheses, reconsideration of research design, modifications of data 

collection or analysis methods, or alternative conclusions. However, peer review does not 

always lead to specific modifications in the draft product. In some cases, a draft is in excellent 

shape prior to being submitted for review. In others, the authors do not concur with changes 

suggested by one or more reviewers. 

15 Carnegie Commission on Science, Technology, and Government, Risk and the Environment: Improving
 

Regulatory Decision Making, Carnegie Commission, New York, 1993: 75. 16 William W. Lowrance, Modern Science and Human Values, Oxford University Press, New York, NY 1985: 85.
 
 

 

 

 

 Peer Review Handbook Page B-4 

Peer review may take a variety of forms, depending upon the nature and importance of the 

product. For example, the reviewers may represent one scientific discipline or a variety of 

disciplines; the number of reviewers may range from a few to more than a dozen; the names of 

each reviewer may be disclosed publicly or may remain anonymous (e.g., to encourage candor); 

the reviewers may be blinded to the authors of the report or the names of the authors may be 

disclosed to the reviewers; the reviewers may prepare individual reports or a panel of reviewers 

may be constituted to produce a collaborative report; panels may do their work electronically or 

they may meet together in person to discuss and prepare their evaluations; and reviewers may be 

compensated for their work or they may donate their time as a contribution to science or public 

service. 

For large, complex reports, different reviewers may be assigned to different chapters or topics. 

Such reports may be reviewed in stages, sometimes with confidential reviews that precede a 

public process of panel review. As part of government-sponsored peer review, there may be 

opportunity for written and/or oral public comments on the draft product. 

The results of peer review are often only one of the criteria used to make decisions about journal 

publication, grant funding, and information dissemination. For instance, the editors of scientific 

journals (rather than the peer reviewers) make final decisions about a manuscript’s 

appropriateness for publication based on a variety of considerations. In research-funding 

decisions, the reports of peer reviewers often play an important role, but the final decisions about 

funding are often made by accountable officials based on a variety of considerations. Similarly, 

when a government agency sponsors peer review of its own draft documents, the peer review 

reports are an important factor in information dissemination decisions but rarely are the sole 

consideration. Agencies are not expected to cede their discretion with regard to dissemination or 

use of information to peer reviewers; accountable agency officials must make the final decisions. 

THE NEED FOR STRONGER PEER REVIEW POLICIES 

There are a multiplicity of science advisory procedures used at federal agencies and across the 

wide variety of scientific products prepared by agencies.17 In response to congressional inquiry, 

the U.S. General Accounting Office (now the Government Accountability Office) documented 

the variability in both the definition and implementation of peer review across agencies.18 The 

Carnegie Commission on Science, Technology and Government19 has highlighted the 

importance of “internal” scientific advice (within the agency) and “external” advice (through 

scientific advisory boards and other mechanisms). 

17 Sheila Jasanoff, The Fifth Branch: Science Advisors as Policy Makers, Harvard University Press, Boston, 1990. 18 U.S. General Accounting Office, Federal Research: Peer Review Practices at Federal Agencies Vary, 

GAO/RCED-99-99, Washington, D.C., 1999. 

19 Carnegie Commission on Science, Technology, and Government, Risk and the Environment: Improving
 

Regulatory Decision Making, Carnegie Commission, New York, 1993: 90. 
 

 

 

 

 

 

 Peer Review Handbook Page B-5 

A wide variety of authorities have argued that peer review practices at federal agencies need to 

be strengthened.20 Some arguments focus on specific types of scientific products (e.g., 

assessments of health, safety and environmental hazards). 21 The Congressional/Presidential 

Commission on Risk Assessment and Risk Management suggests that “peer review of economic 

and social science information should have as high a priority as peer review of health, ecological, 

and engineering information.”22 

Some agencies have formal peer review policies, while others do not. Even agencies that have 

such policies do not always follow them prior to the release of important scientific products. 

Prior to the development of this Bulletin, there were no government-wide standards concerning 

when peer review is required and, if required, what type of peer review processes are 

appropriate. No formal interagency mechanism existed to foster cross-agency sharing of 

experiences with peer review practices and policies. Despite the importance of peer review for 

the credibility of agency scientific products, the public lacked a consistent way to determine 

when an important scientific information product is being developed by an agency, the type of 

peer review planned for that product, or whether there would be an opportunity to provide 

comments and data to the reviewers. 

This Bulletin establishes minimum standards for when peer review is required for scientific 

information and the types of peer review that should be considered by agencies in different 

circumstances. It also establishes a transparent process for public disclosure of peer review 

planning, including a web-accessible description of the peer review plan that the agency has 

developed for each of its forthcoming influential scientific disseminations. 

LEGAL AUTHORITY FOR THE BULLETIN 

This Bulletin is issued under the Information Quality Act and OMB’s general authorities to 

oversee the quality of agency information, analyses, and regulatory actions. In the Information 

Quality Act, Congress directed OMB to issue guidelines to “provide policy and procedural 

20 National Academy of Sciences, Peer Review in the Department of Energy – Office of Science and Technology, 

Interim Report, National Academy Press, Washington, D.C., 1997; National Academy of Sciences, Peer Review in 

Environmental Technology Development: The Department of Energy – Office of Science and Technology, National 

Academy Press, Washington, D.C., 1998; National Academy of Sciences, Strengthening Science at the U.S. 

Environmental Protection Agency: Research-Management and Peer-Review Practices, National Academy Press, 

Washington, D.C. 2000; U.S. General Accounting Office, EPA’s Science Advisory Board Panels: Improved 

Policies and Procedures Needed to Ensure Independence and Balance, GAO-01-536, Washington, D.C., 2001; U. S. 

Environmental Protection Agency, Office of Inspector General, Pilot Study: Science in Support of Rulemaking
 

2003-P-00003, Washington, D.C., 2002; Carnegie Commission on Science, Technology, and Government, In the 

National Interest: The Federal Government in the Reform of K-12 Math and Science Education, Carnegie 

Commission, New York, 1991; U.S. General Accounting Office, Endangered Species Program: Information on
 

How Funds Are Allocated and What Activities are Emphasized, GAO-02-581, Washington, D.C. 2002. 21 National Research Council, Science and Judgment in Risk Assessment, National Academy Press, Washington, 

D.C., 1994. 

22 Presidential/Congressional Commission on Risk Assessment and Risk Management, Risk Commission Report, 

Volume 2, Risk Assessment and Risk Management in Regulatory Decision-Making, 1997:103. 
 Peer Review Handbook Page B-6 

guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility and 

integrity of information” disseminated by Federal agencies. Pub. L. No. 106-554, § 515(a). The 

Information Quality Act was developed as a supplement to the Paperwork Reduction Act, 44 

U.S.C. § 3501 et seq., which requires OMB, among other things, to “develop and oversee the 

implementation of policies, principles, standards, and guidelines to . . . apply to Federal agency 

dissemination of public information.” In addition, Executive Order 12866, 58 Fed. Reg. 51,735 

(Oct. 4, 1993), establishes that OIRA is “the repository of expertise concerning regulatory 

issues,” and it directs OMB to provide guidance to the agencies on regulatory planning. E.O. 

12866, § 2(b). The Order also requires that “[e]ach agency shall base its decisions on the best 

reasonably obtainable scientific, technical, economic, or other information.” E.O. 12866, § 

1(b)(7). Finally, OMB has authority in certain circumstances to manage the agencies under the 

purview of the President’s Constitutional authority to supervise the unitary Executive Branch. 

All of these authorities support this Bulletin. 

THE REQUIREMENTS OF THIS BULLETIN 

This Bulletin addresses peer review of scientific information disseminations that contain findings 

or conclusions that represent the official position of one or more agencies of the federal 

government. 

Section I: Definitions 

Section I provides definitions that are central to this Bulletin. Several terms are identical to or 

based on those used in OMB’s government-wide information quality guidelines, 67 Fed. Reg. 

8452 (Feb. 22, 2002), and the Paperwork Reduction Act, 44 U.S.C. § 3501 et seq. 

The term “Administrator” means the Administrator of the Office of Information and Regulatory 

Affairs in the Office of Management and Budget (OIRA). 

The term “agency” has the same meaning as in the Paperwork Reduction Act, 44 U.S.C. § 

3502(1). 

The term “Information Quality Act” means Section 515 of Public Law 106-554 (Pub. L. No. 

106-554, § 515, 114 Stat. 2763, 2763A-153-154 (2000)). 

The term “dissemination” means agency initiated or sponsored distribution of information to the 

public. Dissemination does not include distribution limited to government employees or agency 

contractors or grantees; intra- or inter-agency use or sharing of government information; or 

responses to requests for agency records under the Freedom of Information Act, the Privacy Act, 

the Federal Advisory Committee Act, the Government Performance and Results Act, or similar 

laws. This definition also excludes distribution limited to correspondence with individuals or 

persons, press releases, archival records, public filings, subpoenas and adjudicative processes. In 

the context of this Bulletin, the definition of “dissemination” modifies the definition in OMB’s 

government-wide information quality guidelines to address the need for peer review prior to 
 

 

 Peer Review Handbook Page B-7 

official dissemination of the information product. Accordingly, under this Bulletin, 

“dissemination” also excludes information distributed for peer review in compliance with this 

Bulletin or shared confidentially with scientific colleagues, provided that the distributing agency 

includes an appropriate and clear disclaimer on the information, as explained more fully below. 

Finally, the Bulletin does not directly cover information supplied to the government by third 

parties (e.g., studies by private consultants, companies and private, non-profit organizations, or 

research institutions such as universities). However, if an agency plans to disseminate 

information supplied by a third party (e.g., using this information as the basis for an agency's 

factual determination that a particular behavior causes a disease), the requirements of the 

Bulletin apply, if the dissemination is "influential". 

In cases where a draft report or other information is released by an agency solely for purposes of 

peer review, a question may arise as to whether the draft report constitutes an official 

"dissemination" under information-quality guidelines. Section I instructs agencies to make this 

clear by presenting the following disclaimer in the report: 

“THIS INFORMATION IS DISTRIBUTED SOLELY FOR THE PURPOSE OF PREDISSEMINATION PEER REVIEW UNDER APPLICABLE INFORMATION QUALITY 

GUIDELINES. IT HAS NOT BEEN FORMALLY DISSEMINATED BY [THE AGENCY]. IT 

DOES NOT REPRESENT AND SHOULD NOT BE CONSTRUED TO REPRESENT ANY 

AGENCY DETERMINATION OR POLICY.” 

In cases where the information is highly relevant to specific policy or regulatory 

deliberations, this disclaimer shall appear on each page of a draft report. Agencies also 

shall discourage state, local, international and private organizations from using 

information in draft reports that are undergoing peer review. Draft influential scientific 

information presented at scientific meetings or shared confidentially with colleagues for 

scientific input prior to peer review shall include the disclaimer: “THE FINDINGS AND 

CONCLUSIONS IN THIS REPORT (PRESENTATION) HAVE NOT BEEN FORMALLY 

DISSEMINATED BY [THE AGENCY] AND SHOULD NOT BE CONSTRUED TO REPRESENT ANY 

AGENCY DETERMINATION OR POLICY.” 

An information product is not covered by the Bulletin unless it represents an official view of one 

or more departments or agencies of the federal government. Accordingly, for the purposes of 

this Bulletin, “dissemination” excludes research produced by government-funded scientists (e.g., 

those supported extramurally or intramurally by federal agencies or those working in state or 

local governments with federal support) if that information is not represented as the views of a 

department or agency (i.e., they are not official government disseminations). For influential 

scientific information that does not have the imprimatur of the federal government, scientists 

employed by the federal government are required to include in their information product a clear 

disclaimer that “the findings and conclusions in this report are those of the author(s) and do not 

necessarily represent the views of the funding agency.” A similar disclaimer is advised for nongovernment employees who publish government-funded research. 
 

 

 Peer Review Handbook Page B-8 

For the purposes of the peer review Bulletin, the term “scientific information” means factual 

inputs, data, models, analyses, technical information, or scientific assessments related to such 

disciplines as the behavioral and social sciences, public health and medical sciences, life and 

earth sciences, engineering, or physical sciences. This includes any communication or 

representation of knowledge such as facts or data, in any medium or form, including textual, 

numerical, graphic, cartographic, narrative, or audiovisual forms. This definition includes 

information that an agency disseminates from a web page, but does not include the provision of 

hyperlinks on a web page to information that others disseminate. This definition excludes 

opinions, where the agency’s presentation makes clear that an individual’s opinion, rather than a 

statement of fact or of the agency’s findings and conclusions, is being offered. 

The term “influential scientific information” means scientific information the agency reasonably 

can determine will have or does have a clear and substantial impact on important public policies 

or private sector decisions. In the term “influential scientific information,” the term "influential" 

should be interpreted consistently with OMB's government-wide information quality guidelines 

and the information quality guidelines of the agency. Information dissemination can have a 

significant economic impact even if it is not part of a rulemaking. For instance, the economic 

viability of a technology can be influenced by the government’s characterization of its attributes. 

Alternatively, the federal government's assessment of risk can directly or indirectly influence the 

response actions of state and local agencies or international bodies. 

One type of scientific information is a scientific assessment. For the purposes of this Bulletin, 

the term “scientific assessment” means an evaluation of a body of scientific or technical 

knowledge, which typically synthesizes multiple factual inputs, data, models, assumptions, 

and/or applies best professional judgment to bridge uncertainties in the available information. 

These assessments include, but are not limited to, state-of-science reports; technology 

assessments; weight-of-evidence analyses; meta-analyses; health, safety, or ecological risk 

assessments; toxicological characterizations of substances; integrated assessment models; hazard 

determinations; or exposure assessments. Such assessments often draw upon knowledge from 

multiple disciplines. Typically, the data and models used in scientific assessments have already 

been subject to some form of peer review (e.g., refereed journal peer review or peer review under 

Section II of this Bulletin). 

Section II: Peer Review of Influential Scientific Information 

Section II requires each agency to subject "influential" scientific information to peer review prior 

to dissemination. For dissemination of influential scientific information, Section II provides 

agencies broad discretion in determining what type of peer review is appropriate and what 

procedures should be employed to select appropriate reviewers. Agencies are directed to chose a 

peer review mechanism that is adequate, giving due consideration to the novelty and complexity 

of the science to be reviewed, the relevance of the information to decision making, the extent of 

prior peer reviews, and the expected benefits and costs of additional review. 
 

 

 

 

 Peer Review Handbook Page B-9 

The National Academy of Public Administration suggests that the intensity of peer review 

should be commensurate with the significance of the information being disseminated and the 

likely implications for policy decisions.23 Furthermore, agencies need to consider tradeoffs 

between depth of peer review and timeliness.24 More rigorous peer review is necessary for 

information that is based on novel methods or presents complex challenges for interpretation. 

Furthermore, the need for rigorous peer review is greater when the information contains 

precedent-setting methods or models, presents conclusions that are likely to change prevailing 

practices, or is likely to affect policy decisions that have a significant impact. 

This tradeoff can be considered in a benefit-cost framework. The costs of peer review include 

both the direct costs of the peer review activity and those stemming from potential delay in 

government and private actions that can result from peer review. The benefits of peer review are 

equally clear: the insights offered by peer reviewers may lead to policy with more benefits and/or 

fewer costs. In addition to contributing to strong science, peer review, if performed fairly and 

rigorously, can build consensus among stakeholders and reduce the temptation for courts and 

legislators to second-guess or overturn agency actions.25 While it will not always be easy for 

agencies to quantify the benefits and costs of peer review, agencies are encouraged to approach 

peer review from a benefit-cost perspective. 

Regardless of the peer review mechanism chosen, agencies should strive to ensure that their peer 

review practices are characterized by both scientific integrity and process integrity. “Scientific 

integrity,” in the context of peer review, refers to such issues as “expertise and balance of the 

panel members; the identification of the scientific issues and clarity of the charge to the panel; 

the quality, focus and depth of the discussion of the issues by the panel; the rationale and 

supportability of the panel’s findings; and the accuracy and clarity of the panel report.” “Process 

integrity” includes such issues as “transparency and openness, avoidance of real or perceived 

conflicts of interest, a workable process for public comment and involvement,” and adherence to 

defined procedures.26 

When deciding what type of peer review mechanism is appropriate for a specific information 

product, agencies will need to consider at least the following issues: individual versus panel 

review; timing; scope of the review; selection of reviewers; disclosure and attribution; public 

participation; disposition of reviewer comments; and adequacy of prior peer review. 

23 National Academy of Public Administration, Setting Priorities, Getting Results: A New Direction for EPA, 

National Academy Press, Washington, D.C., 1995:23.

24 Presidential/Congressional Commission on Risk Assessment and Risk Management, Risk Commission Report, 

1997. 

25 Mark R. Powell, Science at EPA: Information in the Regulatory Process, Resources for the Future, Washington, 

D.C., 1999: 148, 176; Sheila Jasanoff, The Fifth Branch: Science Advisors as Policy Makers, Harvard University 

Press, Boston, 1990: 242. 

26 ILSI Risk Sciences Institute, “Policies and Procedures: Model Peer Review Center of 

Excellence,” 2002: 4. Available at http://rsi.ilsi.org/file/Policies&Procedures.pdf. 
 

 

 

 

 Peer Review Handbook Page B-10 

Individual versus Panel Review 

Letter reviews by several experts generally will be more expeditious than convening a panel of 

experts. Individual letter reviews are more appropriate when a draft document covers only one 

discipline or when premature disclosure of a sensitive report to a public panel could cause harm 

to government or private interests. When time and resources warrant, panels are preferable, as 

they tend to be more deliberative than individual letter reviews and the reviewers can learn from 

each other. There are also multi-stage processes in which confidential letter reviews are 

conducted prior to release of a draft document for public notice and comment, followed by a 

formal panel review. These more rigorous and expensive processes are particularly valuable for 

highly complex, multidisciplinary, and more important documents, especially those that are 

novel or precedent-setting. 

Timing of Peer Review 

As a general rule, it is most useful to consult with peers early in the process of producing 

information. For example, in the context of risk assessments, it is valuable to have the choice of 

input data and the specification of the model reviewed by peers before the agency invests time 

and resources in implementing the model and interpreting the results. "Early" peer review occurs 

in time to "focus attention on data inadequacies in time for corrections. 

When an information product is a critical component of rule-making, it is important to obtain 

peer review before the agency announces its regulatory options so that any technical corrections 

can be made before the agency becomes invested in a specific approach or the positions of 

interest groups have hardened. If review occurs too late, it is unlikely to contribute to the course 

of a rulemaking. Furthermore, investing in a more rigorous peer review early in the process 

“may provide net benefit by reducing the prospect of challenges to a regulation that later may 

trigger time consuming and resource-draining litigation.”27 

Scope of the Review 

The “charge” contains the instructions to the peer reviewers regarding the objective of the peer 

review and the specific advice sought. The importance of the information, which shapes the goal 

of the peer review, influences the charge. For instance, the goal of the review might be to 

determine the utility of a body of literature for drawing certain conclusions about the feasibility 

of a technology or the safety of a product. In this context, an agency might ask reviewers to 

determine the relevance of conclusions drawn in one context for other contexts (e.g., different 

exposure conditions or patient populations). 

27 Fred Anderson, Mary Ann Chirba Martin, E Donald Elliott, Cynthia Farina, Ernest Gellhorn, John D. Graham, C. 

Boyden Gray, Jeffrey Holmstead, Ronald M. Levin, Lars Noah, Katherine Rhyne, Jonathan Baert Wiener, 

"Regulatory Improvement Legislation: Risk Assessment, Cost-Benefit Analysis, and Judicial Review," Duke 

Environmental Law and Policy Forum, Fall 2000, vol. XI (1): 132. 
 

 

 

 Peer Review Handbook Page B-11 

The charge to the reviewers should be determined in advance of the selection of the reviewers. 

In drafting the charge, it is important to remember the strengths and limitations of peer review. 

Peer review is most powerful when the charge is specific and steers the reviewers to specific 

technical questions while also directing reviewers to offer a broad evaluation of the overall 

product. 

Uncertainty is inherent in science, and in many cases individual studies do not produce 

conclusive evidence. Thus, when an agency generates a scientific assessment, it is presenting its 

scientific judgment about the accumulated evidence rather than scientific fact.28 Specialists 

attempt to reach a consensus by weighing the accumulated evidence. Peer reviewers can make 

an important contribution by distinguishing scientific facts from professional judgments. 

Furthermore, where appropriate, reviewers should be asked to provide advice on the 

reasonableness of judgments made from the scientific evidence. However, the charge should 

make clear that the reviewers are not to provide advice on the policy (e.g., the amount of 

uncertainty that is acceptable or the amount of precaution that should be embedded in an 

analysis). Such considerations are the purview of the government.29 

The charge should ask that peer reviewers ensure that scientific uncertainties are clearly 

identified and characterized. Since not all uncertainties have an equal effect on the conclusions 

drawn, reviewers should be asked to ensure that the potential implications of the uncertainties for 

the technical conclusions drawn are clear. In addition, peer reviewers might be asked to consider 

value-of-information analyses that identify whether more research is likely to decrease key 

uncertainties.30 Value-of-information analysis was suggested for this purpose in the report of the 

Presidential/Congressional Commission on Risk Assessment and Risk Management.31 A 

description of additional research that would appreciably influence the conclusions of the 

assessment can help an agency assess and target subsequent efforts. 

Selection of Reviewers 

Expertise. The most important factor in selecting reviewers is expertise: ensuring that the 

selected reviewer has the knowledge, experience, and skills necessary to perform the review. 

Agencies shall ensure that, in cases where the document being reviewed spans a variety of 

scientific disciplines or areas of technical expertise, reviewers who represent the necessary 

spectrum of knowledge are chosen. For instance, expertise in applied mathematics and statistics 

is essential in the review of models, thereby allowing an audit of calculations and claims of 

28 Mark R. Powell, Science at EPA: Information in the Regulatory Process, Resources for the Future, Washington, 

D.C., 1999: 139. 

29 Ibid. 

30 Granger Morgan and Max Henrion, “The Value of Knowing How Little You Know,”
 

Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, 

Cambridge University Press, 1990: 307. 

31 Presidential/Congressional Commission on Risk Assessment and Risk Management, Risk Commission Report, 

1997, Volume 1: 39, Volume 2: 91. 
 

 

 

 

 

 Peer Review Handbook Page B-12 

significance and robustness based on the numeric data.32 For some reviews, evaluation of 

biological plausibility is as important as statistical modeling. Agencies shall consider requesting 

that the public, including scientific and professional societies, nominate potential reviewers. 

Balance. While expertise is the primary consideration, reviewers should also be selected to 

represent a diversity of scientific perspectives relevant to the subject. On most controversial 

issues, there exists a range of respected scientific viewpoints regarding interpretation of the 

available literature. Inviting reviewers with competing views on the science may lead to a 

sharper, more focused peer review. Indeed, as a final layer of review, some organizations (e.g., 

the National Academy of Sciences) specifically recruit reviewers with strong opinions to test the 

scientific strength and balance of their reports. The NAS policy on committee composition and 

balance33 highlights important considerations associated with perspective, bias, and objectivity. 

Independence. In its narrowest sense, independence in a reviewer means that the reviewer was 

not involved in producing the draft document to be reviewed. However, for peer review of some 

documents, a broader view of independence is necessary to ensure credibility of the process. 

Reviewers are generally not employed by the agency or office producing the document. As the 

National Academy of Sciences has stated, “external experts often can be more open, frank, and 

challenging to the status quo than internal reviewers, who may feel constrained by organizational 

concerns.”34 The Carnegie Commission on Science, Technology, and Government notes that 

“external science advisory boards serve a critically important function in providing regulatory 

agencies with expert advice on a range of issues.”35 However, the choice of reviewers requires a 

case-by-case analysis. Reviewers employed by other federal and state agencies may possess 

unique or indispensable expertise. 

A related issue is whether government-funded scientists in universities and consulting firms have 

sufficient independence from the federal agencies that support their work to be appropriate peer 

reviewers for those agencies.36 This concern can be mitigated in situations where the scientist 

initiates the hypothesis to be tested or the method to be developed, which effectively creates a 

buffer between the scientist and the agency. When an agency awards grants through a 

competitive process that includes peer review, the agency’s potential to influence the scientist’s 

research is limited. As such, when a scientist is awarded a government research grant through an 

investigator-initiated, peer-reviewed competition, there generally should be no question as to that 

scientist's ability to offer independent scientific advice to the agency on other projects. This 

contrasts, for example, to a situation in which a scientist has a consulting or contractual 

arrangement with the agency or office sponsoring a peer review. Likewise, when the agency 

32 William W. Lowrance, Modern Science and Human Values, Oxford University Press, New York, NY 1985: 86.
 33 National Academy of Sciences, “Policy and Procedures on Committee Composition and Balance and Conflicts of 

Interest for Committees Used in the Development of Reports,” May 2003: Available at: 

http://www.nationalacademies.org/coi/index.html.

34 National Research Council, Peer Review in Environmental Technology Development Programs: The Department
 

of Energy’s Office of Science and Technology, National Academy Press, Washington, D.C., 1998: 3.
 35 Carnegie Commission on Science, Technology, and Government, Risk and the Environment: Improving
 

Regulatory Decision Making, Carnegie Commission, New York, 1993: 90. 36 Lars Noah, “Scientific ‘Republicanism’: Expert Peer Review and the Quest for Regulatory Deliberation, Emory
 

Law Journal, Atlanta, Fall 2000:1066. 
 

 

 

 Peer Review Handbook Page B-13 

and a researcher work together (e.g., through a cooperative agreement) to design or implement a 

study, there is less independence from the agency. Furthermore, if a scientist has repeatedly 

served as a reviewer for the same agency, some may question whether that scientist is 

sufficiently independent from the agency to be employed as a peer reviewer on agencysponsored projects. 

As the foregoing suggests, independence poses a complex set of questions that must be 

considered by agencies when peer reviewers are selected. In general, agencies shall make an 

effort to rotate peer review responsibilities across the available pool of qualified reviewers, 

recognizing that in some cases repeated service by the same reviewer is needed because of 

essential expertise. 

Some agencies have built entire organizations to provide independent scientific advice while 

other agencies tend to employ ad hoc scientific panels on specific issues. Respect for the 

independence of reviewers may be enhanced if an agency collects names of potential reviewers 

(based on considerations of expertise and reputation for objectivity) from the public, including 

scientific or professional societies. The Department of Energy’s use of the American Society of 

Mechanical Engineers to identify potential peer reviewers from a variety of different scientific 

societies provides an example of how professional societies can assist in the development of an 

independent peer review panel.37 

Conflict of Interest. The National Academy of Sciences defines “conflict of interest” as any 

financial or other interest that conflicts with the service of an individual on the review panel 

because it could impair the individual’s objectivity or could create an unfair competitive 

advantage for a person or organization.38 This standard provides a useful benchmark for 

agencies to consider in selecting peer reviewers. Agencies shall make a special effort to examine 

prospective reviewers’ potential financial conflicts, including significant investments, consulting 

arrangements, employer affiliations and grants/contracts. Financial ties of potential reviewers to 

regulated entities (e.g., businesses), other stakeholders, and regulatory agencies shall be 

scrutinized when the information being reviewed is likely to be relevant to regulatory policy. 

The inquiry into potential conflicts goes beyond financial investments and business relationships 

and includes work as an expert witness, consulting arrangements, honoraria and sources of grants 

and contracts. To evaluate any real or perceived conflicts of interest with potential reviewers and 

questions regarding the independence of reviewers, agencies are referred to federal ethics 

requirements, applicable standards issued by the Office of Government Ethics, and the prevailing 

practices of the National Academy of Sciences. Specifically, peer reviewers who are federal 

employees (including special government employees) are subject to federal requirements 

governing conflicts of interest. See, e.g., 18 U.S.C. § 208; 5 C.F.R. Part 2635 (2004). With 

37 American Society for Mechanical Engineers, Assessment of Technologies Supported by the Office of Science and 

Technology, Department of Energy: Results of the Peer Review for Fiscal Year 2002, ASME Technical Publishing,
 

Danvers, MA, 2003. 

38 National Academy of Sciences, “Policy and Procedures on Committee Composition and 

Balance and Conflicts of Interest for Committees Used in the Development of Reports,” May 

2003: Available at: http://www.nationalacademies.org/coi/index.html. 
 

 

 

 Peer Review Handbook Page B-14 

respect to reviewers who are not federal employees, agencies shall adopt or adapt the NAS 

policy for committee selection with respect to evaluating conflicts of interest.39 Both the NAS 

and the federal government recognize that under certain circumstances some conflict may be 

unavoidable in order to obtain the necessary expertise. See, e.g., 18 U.S.C. § 208(b)(3); 5 

U.S.C. App. § 15 (governing NAS committees). To improve the transparency of the process, 

when an agency determines that it is necessary to use a reviewer with a real or perceived conflict 

of interest, the agency should consider publicly disclosing those conflicts. In such situations, the 

agency shall inform potential reviewers of such disclosure at the time they are recruited. 

Disclosure and Attribution: Anonymous versus Identified 

Peer reviewers must have a clear understanding of how their comments will be conveyed to the 

authors of the document and to the public. When peer review of government reports is 

considered, the case for transparency is stronger, particularly when the report addresses an issue 

with significant ramifications for the public and private sectors. The public may not have 

confidence in the peer review process when the names and affiliations of the peer reviewers are 

unknown. Without access to the comments of reviewers, the public is incapable of determining 

whether the government has seriously considered the comments of reviewers and made 

appropriate revisions. Disclosure of the slate of reviewers and the substance of their comments 

can strengthen public confidence in the peer review process. It is common at many journals and 

research funding agencies to disclose annually the slate of reviewers. Moreover, the National 

Academy of Sciences now discloses the names of its peer reviewers, without disclosing the 

substance of their comments. The science advisory committees to regulatory agencies typically 

disclose at least a summary of the comments of reviewers as well as their names and affiliations. 

For agency-sponsored peer review conducted under Sections II and III, this Bulletin strikes a 

compromise by requiring disclosure of the identity of the reviewers, but not public attribution of 

specific comments to specific reviewers. The agency has considerable discretion in the 

implementation of this compromise (e.g., summarizing the views of reviewers as a group or 

disclosing individual reviewer comments without attribution). Whatever approach is employed, 

the agency must inform reviewers in advance of how it intends to address this issue. Information 

about a reviewer retrieved from a record filed by the reviewer's name or other identifier may be 

disclosed only as permitted by the conditions of disclosure enumerated in the Privacy Act, 5 

U.S.C. § 552a as amended, and as interpreted in OMB implementing guidance, 40 Fed. Reg. 

28,948 (July 9, 1975). 

Public Participation 

Public comments can be important in shaping expert deliberations. Agencies may decide that 

peer review should precede an opportunity for public comment to ensure that the public receives 

the most scientifically strong product (rather than one that may change substantially as a result of 

peer reviewer suggestions). However, there are situations in which public participation in peer 

39 Ibid. 
 

 

 Peer Review Handbook Page B-15 

review is an important aspect of obtaining a high-quality product through a credible process. 

Agencies, however, should avoid open-ended comment periods, which may delay completion of 

peer reviews and complicate the completion of the final work product. 

Public participation can take a variety of forms, including opportunities to provide oral 

comments before a peer review panel or requests to provide written comments to the peer 

reviewers. Another option is for agencies to publish a “request for comment” or other notice in 

which they solicit public comment before a panel of peer reviewers performs its work. 

Disposition of Reviewer Comments 

A peer review is considered completed once the agency considers and addresses the reviewers’ 

comments. All reviewer comments should be given consideration and be incorporated where 

relevant and valid. For instance, in the context of risk assessments, the National Academy of 

Sciences recommends that peer review include a written evaluation made available for public 

inspection.40 In cases where there is a public panel, the agency should plan publication of the 

peer review report(s) and the agency’s response to peer reviewer comments. 

In addition, the credibility of the final scientific report is likely to be enhanced if the public 

understands how the agency addressed the specific concerns raised by the peer reviewers. 

Accordingly, agencies should consider preparing a written response to the peer review report 

explaining: the agency's agreement or disagreement, the actions the agency has undertaken or 

will undertake in response to the report, and (if applicable) the reasons the agency believes those 

actions satisfy any key concerns or recommendations in the report. 

Adequacy of Prior Peer Review 

In light of the broad range of information covered by Section II, agencies are directed to choose a 

peer review mechanism that is adequate, giving due consideration to the novelty and complexity 

of the science to be reviewed, the relevance of the information to decision making, the extent of 

55prior peer reviews, and the expected benefits and costs of additional review. 

Publication in a refereed scientific journal may mean that adequate peer review has been 

performed. However, the intensity of peer review is highly variable across journals. There will 

be cases in which an agency determines that a more rigorous or transparent review process is 

necessary. For instance, an agency may determine a particular journal review process did not 

address questions (e.g., the extent of uncertainty inherent in a finding) that the agency determines 

should be addressed before disseminating that information. As such, prior peer review and 

publication is not by itself sufficient grounds for determining that no further review is necessary. 

40 National Research Council, Risk Assessment in the Federal Government: Managing the Process, National 

Academy Press, Washington, D.C., 1983. 
 

 

 

 

 Peer Review Handbook Page B-16 

Section III: Peer Review of Highly Influential Scientific Assessments 

Whereas Section II leaves most of the considerations regarding the form of the peer review to the 

agency’s discretion, Section III requires a more rigorous form of peer review for highly 

influential scientific assessments. The requirements of Section II of this Bulletin apply to Section 

III, but Section III has some additional requirements, which are discussed below. In planning a 

peer review under Section III, agencies typically will have to devote greater resources and 

attention to the issues discussed in Section II, i.e., individual versus panel review; timing; scope 

of the review; selection of reviewers; disclosure and attribution; public participation; and 

disposition of reviewer comments. 

A scientific assessment is considered "highly influential" if the agency or the OIRA 

Administrator determines that the dissemination could have a potential impact of more than $500 

million in any one year on either the public or private sector or that the dissemination is novel, 

controversial, or precedent-setting, or has significant interagency interest. One of the ways 

information can exert economic impact is through the costs or benefits of a regulation based on 

the disseminated information. The qualitative aspect of this definition may be most useful in 

cases where it is difficult for an agency to predict the potential economic effect of dissemination. 

In the context of this Bulletin, it may be either the approach used in the assessment or the 

interpretation of the information itself that is novel or precedent-setting. Peer review can be 

valuable in establishing the bounds of the scientific debate when methods or interpretations are a 

source of controversy among interested parties. If information is covered by Section III, an 

agency is required to adhere to the peer review procedures specified in Section III. 

Section III (2) clarifies that the principal findings, conclusions and recommendations in official 

reports of the National Academy of Sciences that fall under this Section are generally presumed 

not to require additional peer review. All other highly influential scientific assessments require a 

review that meets the requirements of Section III of this Bulletin. 

With regard to the selection of reviewers, Section III(3)(a) emphasizes consideration of expertise 

and balance. As discussed in Section II, expertise refers to the required knowledge, experience 

and skills required to perform the review whereas balance refers to the need for diversity in 

scientific perspective and disciplines. We emphasize that the term "balance" here refers not to 

balancing of stakeholder or political interests but rather to a broad and diverse representation of 

respected perspectives and intellectual traditions within the scientific community, as discussed in 

the NAS policy on committee composition and balance.41 

Section III(3)(b) instructs agencies to consider barring participation by scientists with a conflict 

of interest. The conflict of interest standards for Sections II and III of the Bulletin are identical. 

As discussed under Section II, those peer reviewers who are federal employees, including 

Special Government Employees, are subject to applicable statutory and regulatory standards for 

41 National Academy of Sciences, “Policy and Procedures on Committee Composition and Balance and Conflicts of 

Interest for Committees Used in the Development of Reports,” May 2003: Available at: 

http://www.nationalacademies.org/coi/index.html. 
 

 

 

 Peer Review Handbook Page B-17 

federal employees. For non-government employees, agencies shall adopt or adapt the NAS 

policy for committee member selection with respect to evaluating conflicts of interest. 

Section III(3)(c) instructs agencies to ensure that reviewers are independent of the agency 

sponsoring the review. Scientists employed by the sponsoring agency are not permitted to 

serve as reviewers for highly influential scientific assessments. This does not preclude Special 

Government Employees, such as academics appointed to advisory committees, from serving as 

peer reviewers. The only exception to this ban would be the rare situation in which a scientist 

from a different agency of a Cabinet-level department than the agency that is disseminating the 

scientific assessment has expertise, experience and skills that are essential but cannot be obtained 

elsewhere. In evaluating the need for this exception, agencies shall use the NAS criteria for 

assessing the appropriateness of using employees of sponsors (e.g., the government 

scientist must not have had any part in the development or prior review of the scientific 

information and must not hold a position of managerial or policy responsibility). 

We also considered whether a reviewer can be independent of the agency if that reviewer 

receives a substantial amount of research funding from the agency sponsoring the review. 

Research grants that were awarded to the scientist based on investigator-initiated, competitive, 

peer-reviewed proposals do not generally raise issues of independence. However, significant 

consulting and contractual relationships with the agency may raise issues of independence or 

conflict, depending upon the situation. 

Section III(3)(d) addresses concerns regarding repeated use of the same reviewer in multiple 

assessments. Such repeated use should be avoided unless a particular reviewer’s expertise is 

essential. Agencies should rotate membership across the available pool of qualified reviewers. 

Similarly, when using standing panels of scientific advisors, it is suggested that the agency rotate 

membership among qualified scientists in order to obtain fresh perspectives and reinforce the 

reality and perception of independence from the agency. 

Section III(4) requires agencies to provide reviewers with sufficient background information, 

including access to key studies, data and models, to perform their role as peer reviewers. In this 

respect, the peer review envisioned in Section III is more rigorous than some forms of journal 

peer review, where the reviewer is often not provided access to underlying data or models. 

Reviewers shall be informed of applicable access, objectivity, reproducibility and other quality 

standards under federal information quality laws. 

Section III(5) addresses opportunity for public participation in peer review, and provides that the 

agency shall, wherever possible, provide for public participation. In some cases, an assessment 

may be so sensitive that it is critical that the agency’s assessment achieve a high level of quality 

before it is publicized. In those situations, a rigorous yet confidential peer review process may 

be appropriate, prior to public release of the assessment. If an agency decides to make a draft 

assessment publicly available at the onset of a peer review process, the agency shall, whenever 

possible, provide a vehicle for the public to provide written comments, make an oral presentation 

before the peer reviewers, or both. When written public comments are received, the agency shall 
 

 

 Peer Review Handbook Page B-18 

ensure that peer reviewers receive copies of comments that address significant scientific issues 

with ample time to consider them in their review. To avoid undue delay of agency activities, the 

agency shall specify time limits for public participation throughout the peer review process. 

Section III(6) requires that agencies instruct reviewers to prepare a peer review report that 

describes the nature and scope of their review and their findings and conclusions. The report 

shall disclose the name of each peer reviewer and a brief description of his or her organizational 

affiliation, credentials and relevant experiences. The peer review report should either summarize 

the views of the group as a whole (including any dissenting views) or include a verbatim copy of 

the comments of the individual reviewers (with or without attribution of specific views to 

specific names). The agency shall also prepare a written response to the peer review report, 

indicating whether the agency agrees with the reviewers and what actions the agency has taken 

or plans to take to address the points made by reviewers. The agency is required to disseminate 

the peer review report and the agency's response to the report on the agency's website, including 

all the materials related to the peer review such as the charge statement, peer review report, and 

agency response to the review. If the scientific information is used to support a final rule then, 

where practicable, the peer review report shall be made available to the public with enough time 

for the public to consider the implications of the peer review report for the rule being considered. 

Section III(7) authorizes but does not require an agency to commission an entity independent of 

the agency to select peer reviewers and/or manage the peer review process in accordance with 

this Bulletin. The entity may be a scientific or professional society, a firm specializing in peer 

review, or a non-profit organization with experience in peer review. 

Section IV: Alternative Procedures 

Peer review as described in this Bulletin is only one of many procedures that agencies can 

employ to ensure an appropriate degree of pre-dissemination quality of influential scientific 

information. For example, Congress has assigned the NAS a special role in advising the federal 

government on scientific and technical issues. The procedures of the NAS are generally quite 

rigorous, and thus agencies should presume that major findings, conclusions, and 

recommendations of NAS reports meet the performance standards of this Bulletin. 

As an alternative to complying with Sections II and III of this Bulletin, an agency may instead 

(1) rely on scientific information produced by the National Academy of Sciences, (2) 

commission the National Academy of Sciences to peer review an agency draft scientific 

information product, or (3) employ an alternative procedure or set of procedures, specifically 

approved by the OIRA Administrator in consultation with the Office of Science and Technology 

Policy (OSTP), that ensures that the scientific information product meets applicable 

information-quality standards. 

An example of an alternative procedure is to commission a respected third party other than the 

NAS (e.g., the Health Effects Institute or the National Commission on Radiation Protection and 

Measurement) to conduct an assessment or series of related assessments. Another example of 
 

 

 

 Peer Review Handbook Page B-19 

an alternative set of procedures is the three-part process used by the National Institutes of Health 

(NIH) to generate scientific guidance. Under that process, a scientific proposal or white paper is 

generated by a working group composed of external, independent scientific experts; that paper is 

then forwarded to a separate external scientific council, which then makes recommendations to 

the agency. The agency, in turn, decides whether to adopt and/or modify the proposal. For 

large science agencies that have diverse research portfolios and do not have significant 

regulatory responsibilities, such as NIH, an acceptable alternative would be 

to allow scientists from one part of the agency (for example, an NIH institute) to participate in 

the review of documents prepared by another part of the agency, as long as the head of the 

agency confirms in writing that each of the reviewers meets the NAS criteria relating to 

the appropriateness of using employees of sponsors (e.g., the government scientist must not 

have had any part in the development or prior review of the scientific information and must not 

hold a position of managerial or policy responsibility). The purpose of Section IV is to 

encourage these types of innovation in the methods used to ensure pre-dissemination quality 

control of influential scientific information. 

The mere existence of a public comment process (e.g., notice-and-comment procedures under the 

Administrative Procedure Act) does not constitute adequate peer review or an “alternative 

process,” because it does not assure that qualified, impartial specialists in relevant fields have 

performed a critical evaluation of the agency's draft product.42 

Section V: Peer Review Planning 

Section V requires agencies to begin a systematic process of peer review planning for influential 

scientific information (including highly influential scientific assessments) that the agency plans 

to disseminate in the foreseeable future. A key feature of this planning process is a webaccessible listing of forthcoming influential scientific disseminations (i.e., an agenda) that is 

regularly updated by the agency. By making these plans publicly available, agencies will be 

able to gauge the extent of public interest in the peer review process for influential scientific 

information, including highly influential scientific assessments. These web-accessible agendas 

can also be used by the public to monitor agency compliance with this Bulletin. 

 Each entry on the agenda shall include a preliminary title of the planned report, a short 

paragraph describing the subject and purpose of the planned report, and an agency contact 

person. The agency shall provide its prediction regarding whether the dissemination will be 

“influential scientific information” or a “highly influential scientific assessment,” as the 

designation can influence the type of peer review to be undertaken. The agency shall discuss the 

timing of the peer review, as well as the use of any deferrals. Agencies shall include entries in 

the agenda for influential scientific information, including highly influential scientific 

assessments, for which the Bulletin’s requirements have been deferred or waived. If the agency, 

in consultation with the OIRA Administrator, has determined that it is appropriate to use a 

42 William W. Lowrance, Modern Science and Human Values, Oxford University Press, New York, NY 1985: 86. 
 

 

 

 Peer Review Handbook Page B-20 

Section IV “alternative procedure” for a specific dissemination, a description of that alternative 

procedure shall be included in the agenda. 

Furthermore, for each entry on the agenda, the agency shall describe the peer review plan. Each 

peer review plan shall include: (i) a paragraph including the title, subject and purpose of the 

planned report, as well as an agency contact to whom inquiries may be directed to learn the 

specifics of the plan; (ii) whether the dissemination is likely to be influential scientific 

information or a highly influential scientific assessment; (iii) the timing of the review (including 

deferrals); (iv) whether the review will be conducted through a panel or individual letters (or 

whether an alternative procedure will be exercised); (v) whether there will be opportunities for 

the public to comment on the work product to be peer reviewed, and if so, how and when these 

opportunities will be provided; (vi) whether the agency will provide significant and relevant 

public comments to the peer reviewers before they conduct their review; (vii) the anticipated 

number of reviewers (3 or fewer; 4-10; or more than 10); (viii) a succinct description of the 

primary disciplines or expertise needed in the review; (ix) whether reviewers will be selected by 

the agency or by a designated outside organization; and (x) whether the public, including 

scientific or professional societies, will be asked to nominate potential peer reviewers. The 

agency shall provide a link from the agenda to each document made public pursuant to this 

Bulletin. Agencies shall link their peer review agendas to the U.S. Government’s official web 

portal: firstgov at http://www.FirstGov.gov 

Agencies should update their peer review agendas at least every six months. However, in some 

cases -- particularly for highly influential scientific assessments and other particularly important 

information -- more frequent updates of existing entries on the agenda, or the addition of new 

entries to the agenda, may be warranted. When new entries are added to the agenda of 

forthcoming reports and other information, the public should be provided with sufficient time to 

comment on the agency's peer review plan for that report or product. Agencies shall consider 

public comments on the peer review plan. Agencies are encouraged to offer a listserve or similar 

mechanism for members of the public who would like to be notified by email each time an 

agency’s peer review agenda has been updated. 

The peer review planning requirements of this Bulletin are designed to be implemented in 

phases. Specifically, the planning requirements of the Bulletin will go into effect for documents 

subject to Section III of the Bulletin (highly influential scientific assessments) six months after 

publication. However, the planning requirements for documents subject to Section II of the 

Bulletin do not go into effect until one year after publication. It is expected that agency 

experience with the planning requirements of the Bulletin for the smaller scope of documents 

encompassed in Section III will be used to inform implementation of these planning 

requirements for the larger scope of documents covered under Section II. 

Section VI: Annual Report 

Each agency shall prepare an annual report that summarizes key decisions made pursuant to this 

Bulletin. In particular, each agency should provide to OIRA the following: 1) the number of 
 

 

 

 Peer Review Handbook Page B-21 

peer reviews conducted subject to the Bulletin (i.e., for influential scientific information and 

highly influential scientific assessments); 2) the number of times alternative procedures were 

invoked; 3) the number of times waivers or deferrals were invoked (and in the case of deferrals, 

the length of time elapsed between the deferral and the peer review); 4) any decision to appoint a 

reviewer pursuant to any exception to the applicable independence or conflict of interest 

standards of the Bulletin, including determinations by the Secretary or Deputy Secretary 

pursuant to Section III (3) (c); 5) the number of peer review panels that were conducted in public 

and the number that allowed public comment; 6) the number of public comments provided on the 

agency’s peer review plans; and 7) the number of peer reviewers that the agency used that were 

recommended by professional societies. 

Section VII: Certification in the Administrative Record 

If an agency relies on influential scientific information or a highly influential scientific 

assessment subject to the requirements of this Bulletin in support of a regulatory action, the 

agency shall include in the administrative record for that action a certification that explains how 

the agency has complied with the requirements of this Bulletin and the Information Quality Act. 

Relevant materials are to be placed in the administrative record. 

Section VIII: Safeguards, Deferrals, and Waivers 

Section VIII recognizes that individuals serving as peer reviewers have a privacy interest in 

information about themselves that the government maintains and retrieves by name or identifier 

from a system of records. To the extent information about a reviewer (name, credential, 

affiliation) will be disclosed along with his/her comments or analysis, the agency must comply 

with the requirements of the Privacy Act, 5 U.S.C. 552a, as amended, and OMB Circular A-130, 

Appendix I, 61 Fed. Reg. 6428 (February 20, 1996) to establish appropriate routine uses in a 

published System of Records Notice. Furthermore, the peer review must be conducted in a 

manner that respects confidential business information as well as intellectual property. 

Section VIII also allows for a deferral or waiver of the requirements of the Bulletin where 

necessary. Specifically, the agency head may waive or defer some or all of the peer review 

requirements of Sections II or III of this Bulletin if there is a compelling rationale for waiver or 

deferral. Waivers will seldom be warranted under this provision because the Bulletin already 

provides significant safety valves, such as: the exemptions provided in Section IX, including the 

exemption for time-sensitive health and safety information; the authorization for alternative 

procedures in Section IV; and the overall flexibility provided for peer reviews of influential 

scientific information under Section II. Nonetheless, we have included this waiver and deferral 

provision to ensure needed flexibility in unusual and compelling situations not otherwise covered 

by the exemptions to the Bulletin, such as situations where unavoidable legal deadlines prevent 

full compliance with the Bulletin before information is disseminated. Deadlines found in 

consent decrees agreed to by agencies after the Bulletin is issued will not ordinarily warrant 

waiver of the Bulletin’s requirements because those deadlines should be negotiated to permit 

time for all required procedures, including peer review. In addition, when an agency is 
 

 

 

 Peer Review Handbook Page B-22 

unavoidably up against a deadline, deferral of some or all requirements of the Bulletin (as 

opposed to outright waiver of all of them) is the most appropriate accommodation between the 

need to satisfy immovable deadlines and the need to undertake proper peer review. If the agency 

head defers any of the peer review requirements prior to dissemination, peer review should be 

conducted as soon as practicable thereafter. 

Section IX: Exemptions 

There are a variety of situations where agencies need not conduct peer review under this 

Bulletin. These include, for example, disseminations of sensitive information related to certain 

national security, foreign affairs, or negotiations involving international treaties and trade where 

compliance with this Bulletin would interfere with the need for secrecy or promptness. 

This Bulletin does not cover official disseminations that arise in adjudications and permit 

proceedings, unless the agency determines that peer review is practical and appropriate and that 

the influential dissemination is scientifically or technically novel (i.e., a major change in 

accepted practice) or likely to have precedent-setting influence on future adjudications or permit 

proceedings. This exclusion is intended to cover, among other things, licensing, approval and 

registration processes for specific product development activities as well as site-specific 

activities. The determination as to whether peer review is practical and appropriate is left to the 

discretion of the agency. While this Bulletin is not broadly applicable to adjudications, agencies 

are encouraged to hold peer reviews of scientific assessments supporting adjudications to the 

same technical standards as peer reviews covered by the Bulletin, including transparency and 

disclosure of the data and models underlying the assessments. Protections apply to confidential 

business information. 

The Bulletin does not cover time-sensitive health and safety disseminations, for example, a 

dissemination based primarily on data from a recent clinical trial that was adequately peer 

reviewed before the trial began. For this purpose, “health” includes public health, or plant or 

animal infectious diseases. 

This Bulletin covers original data and formal analytic models used by agencies in Regulatory 

Impact Analyses (RIAs). However, the RIA documents themselves are already reviewed 

through an interagency review process under E.O. 12866 that involves application of the 

principles and methods defined in OMB Circular A-4. In that respect, RIAs are excluded from 

coverage by this Bulletin, although agencies are encouraged to have RIAs reviewed by peers 

within the government for adequacy and completeness. 

The Bulletin does not cover accounting, budget, actuarial, and financial information including 

that which is generated or used by agencies that focus on interest rates, banking, currency, 

securities, commodities, futures, or taxes. 
 

 

 Peer Review Handbook Page B-23 

Routine statistical information released by federal statistical agencies (e.g., periodic demographic 

and economic statistics) and analyses of these data to compute standard indicators and trends 

(e.g., unemployment and poverty rates) is excluded from this Bulletin. 

The Bulletin does not cover information disseminated in connection with routine rules that 

materially alter entitlements, grants, user fees, or loan programs, or the rights and obligations of 

recipients thereof. 

If information is disseminated pursuant to an exemption to this Bulletin, subsequent 

disseminations are not automatically exempted. For example, if influential scientific information 

is first disseminated in the course of an exempt agency adjudication, but is later disseminated in 

the context of a non-exempt rulemaking, the subsequent dissemination will be subject to the 

requirements of this Bulletin even though the first dissemination was not. 

Section X: OIRA and OSTP Responsibilities 

OIRA, in consultation with OSTP, is responsible for overseeing agency implementation of this 

Bulletin. In order to foster learning about peer review practices across agencies, OIRA and 

OSTP shall form an interagency workgroup on peer review that meets regularly, discusses 

progress and challenges, and recommends improvements to peer review practices. 

Section XI: Effective Date and Existing Law 

The requirements of this Bulletin, with the exception of Section V, apply to information 

disseminated on or after six months after publication of this Bulletin. However, the Bulletin 

does not apply to information that is already being addressed by an agency-initiated peer review 

process (e.g., a draft is already being reviewed by a formal scientific advisory committee 

established by the agency). An existing peer review mechanism mandated by law should be 

implemented by the agency in a manner as consistent as possible with the practices and 

procedures outlined in this Bulletin. The requirements of Section V apply to “highly influential 

scientific assessments,” as designated in Section III of the Bulletin, within six months of 

publication of the final Bulletin. The requirements in Section V apply to documents subject to 

Section II of the Bulletin one year after publication of the final Bulletin. 

Section XII: Judicial Review 

This Bulletin is intended to improve the internal management of the Executive Branch and is not 

intended to, and does not, create any right or benefit, substantive or procedural, enforceable at 

law or in equity, against the United States, its agencies or other entities, its officers or 

employees, or any other person. 
 Peer Review Handbook Page B-24 
 Peer Review Handbook Page B-25 

Bulletin for Peer Review 

I. Definitions. 

For purposes of this Bulletin --

1. the term “Administrator” means the Administrator of the Office of Information and 

Regulatory Affairs in the Office of Management and Budget (OIRA); 

2. the term “agency” has the same meaning as in the Paperwork Reduction Act, 44 U.S.C. § 

3502(1); 

3. the term “dissemination” means agency initiated or sponsored distribution of information to 

the public (see 5 C.F.R. 1320.3(d) (definition of “Conduct or Sponsor”)). Dissemination does 

not include distribution limited to government employees or agency contractors or grantees; 

intra- or inter-agency use or sharing of government information; or responses to requests for 

agency records under the Freedom of Information Act, the Privacy Act, the Federal Advisory 

Committee Act, the Government Performance and Results Act or similar law. This definition 

also excludes distribution limited to correspondence with individuals or persons, press releases, 

archival records, public filings, subpoenas and adjudicative processes. The term “dissemination” 

also excludes information distributed for peer review in compliance with this Bulletin, provided 

that the distributing agency includes a clear disclaimer on the information as follows: “THIS 

INFORMATION IS DISTRIBUTED SOLELY FOR THE PURPOSE OF PREDISSEMINATION PEER REVIEW UNDER APPLICABLE INFORMATION QUALITY 

GUIDELINES. IT HAS NOT BEEN FORMALLY DISSEMINATED BY [THE AGENCY]. 

IT DOES NOT REPRESENT AND SHOULD NOT BE CONSTRUED TO REPRESENT ANY 

AGENCY DETERMINATION OR POLICY.” For the purposes of this Bulletin, 

“dissemination” excludes research produced by government-funded scientists (e.g., those 

supported extramurally or intramurally by federal agencies or those working in state or local 

governments with federal support) if that information does not represent the views of an agency. 

To qualify for this exemption, the information should display a clear disclaimer that “the 

findings and conclusions in this report are those of the author(s) and do not necessarily represent 

the views of the funding agency”; 

4. the term “Information Quality Act” means Section 515 of Public Law 106-554 (Pub. L. No. 

106-554, § 515, 114 Stat. 2763, 2763A-153-154 (2000)); 

5. the term “scientific information” means factual inputs, data, models, analyses, technical 

information, or scientific assessments based on the behavioral and social sciences, public health 

and medical sciences, life and earth sciences, engineering, or physical sciences. This includes 

any communication or representation of knowledge such as facts or data, in any medium or form, 

including textual, numerical, graphic, cartographic, narrative, or audiovisual forms. This 

definition includes information that an agency disseminates from a web page, but does not 

include the provision of hyperlinks to information that others disseminate. This definition does 

not include opinions, where the agency’s presentation makes clear that what is being offered is 

someone’s opinion rather than fact or the agency’s views; 

6. the term “influential scientific information” means scientific information the agency 

reasonably can determine will have or does have a clear and substantial impact on important 

public policies or private sector decisions; and 
 Peer Review Handbook Page B-26 

7. the term “scientific assessment” means an evaluation of a body of scientific or technical 

knowledge, which typically synthesizes multiple factual inputs, data, models, assumptions, 

and/or applies best professional judgment to bridge uncertainties in the available information. 

These assessments include, but are not limited to, state-of-science reports; technology 

assessments; weight-of-evidence analyses; meta-analyses; health, safety, or ecological risk 

assessments; toxicological characterizations of substances; integrated assessment models; hazard 

determinations; or exposure assessments. 

II. Peer Review of Influential Scientific Information. 

1. In General: To the extent permitted by law, each agency shall conduct a peer review on all 

influential scientific information that the agency intends to disseminate. Peer reviewers shall be 

charged with reviewing scientific and technical matters, leaving policy determinations for the 

agency. Reviewers shall be informed of applicable access, objectivity, reproducibility and other 

quality standards under the federal laws governing information access and quality. 

2. Adequacy of Prior Peer Review: For information subject to this section of the Bulletin, 

agencies need not have further peer review conducted on information that has already been 

subjected to adequate peer review. In determining whether prior peer review is adequate, 

agencies shall give due consideration to the novelty and complexity of the science to be 

reviewed, the importance of the information to decision making, the extent of prior peer reviews, 

and the expected benefits and costs of additional review. Principal findings, conclusions and 

recommendations in official reports of the National Academy of Sciences are generally 

presumed to have been adequately peer reviewed. 

3. Selection of Reviewers: 

a. Expertise and Balance: Peer reviewers shall be selected based on expertise, experience 

and skills, including specialists from multiple disciplines, as necessary. The group of reviewers 

shall be sufficiently broad and diverse to fairly represent the relevant scientific and technical 

perspectives and fields of knowledge. Agencies shall consider requesting that the public, 

including scientific and professional societies, nominate potential reviewers. 

b. Conflicts: The agency – or the entity selecting the peer reviewers – shall (i) ensure that 

those reviewers serving as federal employees (including special government employees) comply 

with applicable federal ethics requirements; (ii) in selecting peer reviewers who are not 

government employees, adopt or adapt the National Academy of Sciences policy for committee 

selection with respect to evaluating the potential for conflicts (e.g., those arising from 

investments; agency, employer, and business affiliations; grants, contracts and consulting 

income). For scientific information relevant to specific regulations, the agency shall examine a 

reviewer’s financial ties to regulated entities (e.g., businesses), other stakeholders, and the 

agency. 

c. Independence: Peer reviewers shall not have participated in development of the work 

product. Agencies are encouraged to rotate membership on standing panels across the pool of 

qualified reviewers. Research grants that were awarded to scientists based on investigatorinitiated, competitive, peer-reviewed proposals generally do not raise issues as to independence 

or conflicts. 
 

 

 Peer Review Handbook Page B-27 

4. Choice of Peer Review Mechanism: The choice of a peer review mechanism (for example, 

letter reviews or ad hoc panels) for influential scientific information shall be based on the 

novelty and complexity of the information to be reviewed, the importance of the information to 

decision making, the extent of prior peer review, and the expected benefits and costs of review, 

as well as the factors regarding transparency described in II(5). 

5. Transparency: The agency -- or entity managing the peer review -- shall instruct peer 

reviewers to prepare a report that describes the nature of their review and their findings and 

conclusions. The peer review report shall either (a) include a verbatim copy of each reviewer's 

comments (either with or without specific attributions) or (b) represent the views of the group as 

a whole, including any disparate and dissenting views. The agency shall disclose the names of 

the reviewers and their organizational affiliations in the report. Reviewers shall be notified in 

advance regarding the extent of disclosure and attribution planned by the agency. The agency 

shall disseminate the final peer review report on the agency's website along with all materials 

related to the peer review (any charge statement, the peer review report, and any agency 

response). The peer review report shall be discussed in the preamble to any related rulemaking 

and included in the administrative record for any related agency action. 

6. Management of Peer Review Process and Reviewer Selection: The agency may commission 

independent entities to manage the peer review process, including the selection of peer 

reviewers, in accordance with this Bulletin. 

III. Additional Peer Review Requirements for Highly Influential Scientific Assessments. 

1. Applicability: This section applies to influential scientific information that the agency or 

the Administrator determines to be a scientific assessment that: 

(i) could have a potential impact of more than $500 million in any year, or 

(ii) is novel, controversial, or precedent-setting or has significant interagency interest. 

2. In General: To the extent permitted by law, each agency shall conduct peer reviews on all 

information subject to this Section. The peer reviews shall satisfy the requirements of Section II 

of this Bulletin, as well as the additional requirements found in this Section. Principal findings, 

conclusions and recommendations in official reports of the National Academy of Sciences that 

fall under this Section are generally presumed not to require additional peer review. 

3. Selection of Reviewers: 

a. Expertise and Balance: Peer reviewers shall be selected based on expertise, experience 

and skills, including specialists from multiple disciplines, as necessary. The group of reviewers 

shall be sufficiently broad and diverse to fairly represent the relevant scientific and technical 

perspectives and fields of knowledge. Agencies shall consider requesting that the public, 

including scientific and professional societies, nominate potential reviewers. 

b. Conflicts: The agency – or the entity selecting the peer reviewers – shall (i) ensure that 

those reviewers serving as federal employees (including special government employees) comply 

with applicable federal ethics requirements; (ii) in selecting peer reviewers who are not 

government employees, adopt or adapt the National Academy of Sciences’ policy for committee 

selection with respect to evaluating the potential for conflicts (e.g., those arising from 

investments; agency, employer, and business affiliations; grants, contracts and consulting 

income). For scientific assessments relevant to specific regulations, a reviewer’s financial ties to 

regulated entities (e.g., businesses), other stakeholders, and the agency shall be examined. 
 

 

 

 Peer Review Handbook Page B-28 

c. Independence: In addition to the requirements of Section II (3)(c), which shall apply to 

all reviews conducted under Section III, the agency -- or entity selecting the reviewers -- 

shall bar participation of scientists employed by the sponsoring agency unless the reviewer is 

employed only for the purpose of conducting the peer review (i.e., special government 

employees). The only exception to this bar would be the rare case where the agency determines, 

using the criteria developed by NAS for evaluating use of “employees of sponsors,” that a 

premier government scientist is (a) not in a position of management or policy responsibility and 

(b) possesses essential expertise that cannot be obtained elsewhere. Furthermore, to be eligible 

for this exception, the scientist must be employed by a different agency of the Cabinet-level 

department than the agency that is disseminating the scientific information. The agency’s 

determination shall be documented in writing and approved, on a non-delegable basis, by the 

Secretary or Deputy Secretary of the department prior to the scientist’s appointment. 

d. Rotation: Agencies shall avoid repeated use of the same reviewer on multiple 

assessments unless his or her participation is essential and cannot be obtained elsewhere. 

4. Information Access: The agency -- or entity managing the peer review -- shall provide the 

reviewers with sufficient information -- including background information about key studies or 

models -- to enable them to understand the data, analytic procedures, and assumptions used to 

support the key findings or conclusions of the draft assessment. 

5. Opportunity for Public Participation: Whenever feasible and appropriate, the agency shall 

make the draft scientific assessment available to the public for comment at the same time it is 

submitted for peer review (or during the peer review process) and sponsor a public meeting 

where oral presentations on scientific issues can be made to the peer reviewers by interested 

members of the public. When employing a public comment process as part of the peer review, 

the agency shall, whenever practical, provide peer reviewers with access to public comments that 

address significant scientific or technical issues. To ensure that public participation does not 

unduly delay agency activities, the agency shall clearly specify time limits for public 

participation throughout the peer review process. 

6. Transparency: In addition to the requirements specified in II(5), which shall apply to all 

reviews conducted under Section III, the peer review report shall include the charge to the 

reviewers and a short paragraph on both the credentials and relevant experiences of each peer 

reviewer. The agency shall prepare a written response to the peer review report explaining (a) 

the agency's agreement or disagreement with the views expressed in the report, (b) the actions 

the agency has undertaken or will undertake in response to the report, and (c) the reasons the 

agency believes those actions satisfy the key concerns stated in the report (if applicable). The 

agency shall disseminate its response to the peer review report on the agency's website with the 

related material specified in Section II(5). 

7. Management of Peer Review Process and Reviewer Selection: The agency may commission 

independent entities to manage the peer review process, including the selection of peer 

reviewers, in accordance with this Bulletin. 
 

 

 Peer Review Handbook Page B-29 

IV. Alternative Procedures. 

As an alternative to complying with Sections II and III of this Bulletin, an agency may instead: 

(i) rely on the principal findings, conclusions and recommendations of a report produced by the 

National Academy of Sciences; (ii) commission the National Academy of Sciences to peer 

review an agency’s draft scientific information; or (iii) employ an alternative scientific procedure 

or process, specifically approved by the Administrator in consultation with the Office of Science 

and Technology Policy (OSTP), that ensures the agency’s scientific information satisfies 

applicable information quality standards. The alternative procedure(s) may be applied to a 

designated report or group of reports. 

V. Peer Review Planning. 

1. Peer Review Agenda: Each agency shall post on its website, and update at least every six 

months, an agenda of peer review plans. The agenda shall describe all planned and ongoing 

influential scientific information subject to this Bulletin. The agency shall provide a link from 

the agenda to each document that has been made public pursuant to this Bulletin. Agencies are 

encouraged to offer a listserve or similar mechanism to alert interested members of the public 

when entries are added or updated. 

2. Peer Review Plans: For each entry on the agenda the agency shall describe the peer review 

plan. Each peer review plan shall include: (i) a paragraph including the title, subject and purpose 

of the planned report, as well as an agency contact to whom inquiries may be directed to learn 

the specifics of the plan; (ii) whether the dissemination is likely to be influential scientific 

information or a highly influential scientific assessment; (iii) the timing of the review (including 

deferrals); (iv) whether the review will be conducted through a panel or individual letters (or 

whether an alternative procedure will be employed); (v) whether there will be opportunities for 

the public to comment on the work product to be peer reviewed, and if so, how and when these 

opportunities will be provided; (vi) whether the agency will provide significant and relevant 

public comments to the peer reviewers before they conduct their review; (vii) the anticipated 

number of reviewers (3 or fewer; 4-10; or more than 10); (viii) a succinct description of the 

primary disciplines or expertise needed in the review; (ix) whether reviewers will be selected by 

the agency or by a designated outside organization; and (x) whether the public, including 

scientific or professional societies, will be asked to nominate potential peer reviewers.

 3. Public Comment: Agencies shall establish a mechanism for allowing the public to comment 

on the adequacy of the peer review plans. Agencies shall consider public comments on peer 

review plans. 

VI. Annual Reports. 

Each agency shall provide to OIRA, by December 15 of each year, a summary of the peer 

reviews conducted by the agency during the fiscal year. The report should include the following: 

1) the number of peer reviews conducted subject to the Bulletin (i.e., for influential scientific 

information and highly influential scientific assessments); 2) the number of times alternative 

procedures were invoked; 3) the number of times waivers or deferrals were invoked (and in the 
 

 

 Peer Review Handbook Page B-30 

case of deferrals, the length of time elapsed between the deferral and the peer review); 4) any 

decision to appoint a reviewer pursuant to any exception to the applicable independence or 

conflict of interest standards of the Bulletin, including determinations by the Secretary pursuant 

to Section III(3)(c); 5) the number of peer review panels that were conducted in public and the 

number that allowed public comment; 6) the number of public comments provided on the 

agency’s peer review plans; and 7) the number of peer reviewers that the agency used that were 

recommended by professional societies. 

VII. Certification in the Administrative Record. 

If an agency relies on influential scientific information or a highly influential scientific 

assessment subject to this Bulletin to support a regulatory action, it shall include in the 

administrative record for that action a certification explaining how the agency has complied with 

the requirements of this Bulletin and the applicable information quality guidelines. Relevant 

materials shall be placed in the administrative record. 

VIII. Safeguards, Deferrals, and Waivers. 

1. Privacy: To the extent information about a reviewer (name, credentials, affiliation) will be 

disclosed along with his/her comments or analysis, the agency shall comply with the 

requirements of the Privacy Act, 5 U.S.C. § 522a as amended, and OMB Circular A-130, 

Appendix I, 61 Fed. Reg. 6428 (February 20, 1996) to establish appropriate routine uses in a 

published System of Records Notice. 

2. Confidentiality: Peer review shall be conducted in a manner that respects (i) confidential 

business information and (ii) intellectual property. 

3. Deferral and Waiver: The agency head may waive or defer some or all of the peer review 

requirements of Sections II and III of this Bulletin where warranted by a compelling rationale. If 

the agency head defers the peer review requirements prior to dissemination, peer review shall be 

conducted as soon as practicable. 

IX. Exemptions. 

Agencies need not have peer review conducted on information that is:

 1. related to certain national security, foreign affairs, or negotiations involving international 

trade or treaties where compliance with this Bulletin would interfere with the need for secrecy or 

promptness; 

2. disseminated in the course of an individual agency adjudication or permit proceeding 

(including a registration, approval, licensing, site-specific determination), unless the agency 

determines that peer review is practical and appropriate and that the influential dissemination is 

scientifically or technically novel or likely to have precedent-setting influence on future 

adjudications and/or permit proceedings; 

3. a health or safety dissemination where the agency determines that the dissemination is timesensitive (e.g., findings based primarily on data from a recent clinical trial that was adequately 

peer reviewed before the trial began); 
 

 

 Peer Review Handbook Page B-31 

4. an agency regulatory impact analysis or regulatory flexibility analysis subject to interagency 

review under Executive Order 12866, except for underlying data and analytical models used; 

5. routine statistical information released by federal statistical agencies (e.g., periodic 

demographic and economic statistics) and analyses of these data to compute standard indicators 

and trends (e.g., unemployment and poverty rates); 

6. accounting, budget, actuarial, and financial information, including that which is generated 

or used by agencies that focus on interest rates, banking, currency, securities, commodities, 

futures, or taxes; or 

7. information disseminated in connection with routine rules that materially alter entitlements, 

grants, user fees, or loan programs, or the rights and obligations of recipients thereof. 

X. Responsibilities of OIRA and OSTP. 

OIRA, in consultation with OSTP, shall be responsible for overseeing implementation of this 

Bulletin. An interagency group, chaired by OSTP and OIRA, shall meet periodically to foster 

better understanding about peer review practices and to assess progress in implementing this 

Bulletin. 

XI. Effective Date and Existing Law. 

The requirements of this Bulletin, with the exception of those in Section V (Peer Review 

Planning), apply to information disseminated on or after six months following publication of this 

Bulletin, except that they do not apply to information for which an agency has already provided a 

draft report and an associated charge to peer reviewers. Any existing peer review mechanisms 

mandated by law shall be employed in a manner as consistent as possible with the practices and 

procedures laid out herein. The requirements in Section V apply to “highly influential scientific 

assessments,” as designated in Section III of this Bulletin, within six months of publication of 

this Bulletin. The requirements in Section V apply to documents subject to Section II of this 

Bulletin one year after publication of this Bulletin. 

XII. Judicial Review 

This Bulletin is intended to improve the internal management of the executive branch, and is not 

intended to, and does not, create any right or benefit, substantive or procedural, enforceable at 

law or in equity, against the United States, its agencies or other entities, its officers or employees, 

or any other 
 Peer Review Handbook Page C-1 
 Peer Review Handbook Page C-2 

Appendix C 

Sound Science and Peer Review in 

Rulemaking Policy
 
 Peer Review Handbook Page C-3 

SOUND SCIENCE AND PEER REVIEW IN RULEMAKING 

Several provisions of the December 2004, OMB Bulletin for peer review made it necessary for 

the Office of Policy, Economics, and Innovation (OPEI) to create conditional template language 

(Attachment A). This language should be used by rulewriters in the preamble of regulations that 

rely on influential scientific information or a highly influential scientific assessment, which are 

two categories of information defined in Section 2.2 of this handbook. 

For proposed and final regulations that rely on influential scientific information or a highly 

influential scientific assessment, rulewriters should use the template as a model to discuss peer 

review in the preamble where appropriate. In addition, peer review leaders should communicate 

with rulewriters and workgroup chairs to ensure that all appropriate peer review material is 

included in the docket, and that template language is included in the preamble. 

OPEI also revised the Action Memorandum Framework to include a discussion of peer review 

for influential scientific information or a highly influential scientific assessment (Attachment B). 
 

Λ

 

 Peer Review Handbook Page C-4 

 ATTACHMENT A 

Peer Review (Conditional Template) 

Read this first (but DO NOT insert it in your preamble): 

The OMB Final Information Quality Bulletin for Peer Review directs EPA to include a 

discussion of the peer review report and how the Agency complied with the provisions of the 

Bulletin in the preamble of rulemakings that are supported by influential scientific information or 

highly influential scientific assessments. Peer review reports should either (a) include a verbatim 

copy of each reviewer's comments (either with or without specific attributions) or (b) represent 

the views of the group as a whole, including any disparate and dissenting views. The Agency 

should disclose the names of the reviewers and their organizational affiliations in the report and 

should notify the reviewers in advance regarding the extent of the disclosure and attribution 

planned by the Agency. You should ensure that the peer review report is placed in the docket to 

comply with the OMB Bulletin. 

Use this template if your proposed or final rule is based on a work product containing influential 

scientific information or a highly influential scientific assessment. This language should appear 

in the Supplementary Information section of regulatory preambles under General 

Information. You may want to include the language under the heading: 

Did EPA conduct a peer review before issuing this notice? 

Π PROPOSED & FINAL ACTIONS: If you used a highly influential scientific 

assessment of influential scientific information to support this rulemaking, insert this into 

the preamble of your proposed or final rule, advanced notice of proposed rulemaking, or 

other substantive action: 

This regulatory action was supported by [influential scientific information or a highly 

influential scientific assessment]. Therefore, EPA conducted a peer review in accordance 

with OMB’s Final Information Quality Bulletin for Peer Review. [Insert a brief 

description of the peer review process along with any other relevant information.] The 

peer review report is located in the docket for today's action. According to the report, [insert 

a brief discussion of the peer review report. For more information about the peer 

review report, see the Peer Review Handbook.] 
 Peer Review Handbook Page C-5 

 ATTACHMENT B 

FRAMEWORK FOR ACTION MEMORANDA 

PURPOSE OF THIS FRAMEWORK 

This Framework provides guidance on the format and contents of Action Memoranda that 

accompany packages presented for signature. A draft of the memo also accompanies the Final 

Agency Review (FAR) package. The Framework was developed to promote a consistent 

approach to these memoranda and to preserve the flexibility necessary to accommodate the 

unique needs of each program while ensuring that critical information is provided to Agency 

managers in a clear and concise manner. Generally, the Action Memoranda should not exceed 5 

pages, and follows the format outlined in the Agency’s Correspondence Manual (/asd 

http://epawww/rmpolicy /manuals/1320.pdf.epa.gov). 

As indicated by the use of non-mandatory language such as “should,” “recommend,” and “may,” 

this Framework provides recommendations and does not impose any legally binding 

requirements. Recommendations for each section of Action Memoranda are discussed below. 

Programs may include information on additional topics if they are relevant to a given action, e.g., 

information quality issues, etc. 

Action Memoranda are considered internal, deliberative documents. At the bottom or top of 

each page insert this reminder: “Internal Document Only; Do Not Cite, Quote or Release.” 

A description of the items to be included in an Action Memorandum follows. 

DEADLINE 

Indicate if there is a deadline of any sort and what that deadline is. It may be court imposed, 

statutory, or any one of a number of other deadlines, e.g., upcoming national meeting, 

Administrator’s priority, etc. 

OVERVIEW 

The overview section of an Action Memorandum provides background information which 

briefly describes the rule, characterizes the environmental issue(s) or public health problem(s) 

being addressed, and summarizes the history of the action. It should explain why EPA is taking 

this action, and where appropriate, cover the following points: 

$ Define any court or statutory deadline; 

$ Identify whether the action amends the Code of Federal Regulations, and if so, explain 

what kind of amendment (procedural, etc.); 

$ Identify other actions underway that will affect this particular program or sector; 
 Peer Review Handbook Page C-6 

$ Describe the specific environmental issue(s), public health problem(s) and/or statutory 

requirements being addressed, and the goal intended by taking this action; 

$ Describe what the action does, and specifically, how the regulated community is affected 

(performance standards, specific requirements); and 

$ Describe how flexible the implementation of the action will be for States and regulated 

entities. 

IMPACTS 

Summarize the costs and benefits, including a discussion of any non-monetized benefits, or 

contingent valuation and/or non-quantified benefits, of the action and the results of any economic 

analyses. Explain the roles of cost-benefit and cost-effectiveness analyses in shaping the 

approach chosen. 

Discuss the economic impacts on all affected entities including, where appropriate, possible 

impacts on other Federal agencies, State, or Local governments, and Tribes. 

Summarize the results of the regulatory flexibility analysis and impacts on small entities (small 

businesses, small communities and small not-for-profit entities), including what was done to 

minimize those impacts. If applicable, summarize the SBREFA Panel recommendations, and 

how they were addressed in the rulemaking. 

Describe the reporting and record-keeping burden and what the Agency has done to reduce it. 

STAKEHOLDER INVOLVEMENT 

Briefly discuss the role of both government entities and private sector stakeholders in the 

development of the action. Summarize the concerns they have raised and what the Agency has 

done to address them, or explain why we can’t. 

INTERNAL REVIEW 

Identify whether the action was developed under Tier 1, 2, or 3. If the action was Tier 1 or 2, 

attach the summary memo from the Final Agency Review. Describe any outstanding issues from 

Final Agency Review or other internal review. Identify Program Offices or Regions that 

participated in the development of the action, along with any outstanding issues and why they 

cannot be resolved or accommodated. Also, provide the basis for any decision made to not 

address an identified cross-media impact. 
 

 

 Peer Review Handbook Page C-7 

PEER REVIEW 

If you did not use influential scientific information or a highly influential scientific assessment as 

defined by the Peer Review Handbook to support the action, include the following statement in 

the Action Memorandum: 

There were no influential or highly influential products supporting this action as defined by 

the Agency’s Peer Review Handbook. 

If you did use influential scientific information or a highly influential scientific assessment to 

support the action, include the following statement: 

[Insert Name of AAship] has followed the Agency's Peer Review Policy with respect to the 

underlying [influential scientific information or highly influential scientific assessment] 

supporting this action. 

You may add any details you think are important, but you generally should not modify this 

compliance statement. If you used influential scientific information or a highly influential 

scientific assessment, but were not fully able to meet the Peer Review Policy, explain why. 

PLAIN LANGUAGE 

Briefly describe which elements of plain language you used in the action, e.g., question and 

answer format, addressing the reader as “you,” using short sentences, etc. 

OMB TRANSACTION 

Identify the OMB determination and whether the action went to OMB for review under 

Executive Order 12866. Highlight significant issues resulting from OMB's review under 

Executive Order 12866, including any significant issues raised by other agencies participating in 

the interagency review under Executive Order 12866. Explain any substantive changes made to 

the action as a result of recommendations from OMB or the other agencies. If OMB agreed that 

your action was not significant and, therefore, not subject to EO 12866 review, or if OMB 

waived review of your rule, please so indicate. If OMB was otherwise involved, e.g., was 

briefed or received a courtesy copy for informal review, describe the results of this interaction. 

ANTICIPATED EXTERNAL REACTION 

Describe the type of response anticipated from the various audiences interested or impacted by 

the action. Identify both the involved stakeholders and the nature of their expected response. 

Characterize the likely reaction to the action by all interested parties including industry; 

environmental groups; Congress; State, Local, and Tribal governments; and OMB. Explain what 

the Agency has done to mitigate anticipated adverse reactions. 
 Peer Review Handbook Page C-8 

STAFF CONTACT 

Give the name and phone number of the person to contact if there are questions about the 

package. 

RECOMMENDATION 

Identify the action the Administrator is expected to take (i.e., sign the rule or other document). 

NOTE: Where an Action Memo accompanies another document (i.e., a rule or FR document) to 

be signed by the Administrator (or other authorized official), you should not include a 

concurrence line at the bottom of the Action Memo or anything else that might cause the Action 

Memo to be misinterpreted to be a Decision Memo (which it is not). The FR notice (or other 

document such as an order), after being signed, will contain the Agency’s decision or action (if 

any), not the Action Memo, which is only a pre-decisional briefing document. 
 Peer Review Handbook Page C-9 
 Peer Review Handbook Page D-1 

Appendix D 

Examples Of Peer Review Charges
 
 

 

 

 Peer Review Handbook Page D-2 

Appendix D - Examples Of Peer Review Charges

 Please Note – certain questions that are posed in charges can be responded to with a yes or 

no answer. Clearly, this is not the type of response the Agency generally wants, therefore, it is 

important to phrase charge questions carefully to ensure that you receive a fully satisfactory and 

thoughtful response. Where a yes or no answer might be expected, be sure to ask for a full 

explanation supporting the yes or no answer. 

Charges can run the gamut from rather simplistic to highly complex. The examples shown 

here cover a variety of types. Examples 1 through 4 have less complex questions and are looking 

for the overall quality of the efforts. Examples 5 through 7 have numerous technical questions 

that need to be addressed and are therefore more complex in their nature. 

Other charges that have been used can be found on the SAB website at http://www.epa.gov/sab 

and the SAP website at http://www.epa.gov/scipoly/sap/index.htm 
 Peer Review Handbook Page D-3 

CHARGE EXAMPLE 1 - US EPA Science Advisory Board (SAB) Consultation on 

Suspended and Bedded Sediments 

Background: 

The Ecological Processes and Effects Committee (EPEC) of the EPA Science Advisory Board 

was asked by the Health and Ecological Criteria Division of the Office of Science and 

Technology, Office of Water, to provide a consultation on potential approaches for a strategy for 

developing water-quality criteria for Suspended and Bedded Sediments (SABS). A request for 

nominations for consultant panel members to provide additional expertise to EPEC appeared in 

the Federal Register on July 30, 2003 (68 FR 44758-44760). 

In 1976, EPA issued a water quality criteria recommendation under the Clean Water Act for 

solids and turbidity. For a variety of reasons, the States seldom, if ever, use this criterion. It is 

questionable whether this criterion would achieve intended protection for all different designated 

uses for water bodies. SABS occurs naturally in streams in a wide range of concentrations – 

levels that might be perfectly normal in one water body would be indicative of impairment in 

another. 

Although most States currently have water quality criteria that can be applied to manage SABS, 

these are typically based on turbidity, suspended solids or settle-able solids, and their 

effectiveness for dealing with all water quality impairments caused by SABS, especially as 

benchmarks for aquatic life protection based on natural levels, is questionable. In recent 

consultation with State representatives, the need for new water quality criteria for SABS or 

methodologies for deriving them on a site-specific basis was identified as one of the highest 

priorities for the water quality criteria program. As a result, the EPA Office of Water has 

concluded that to better manage SABS in all types of water bodies and for all designated uses, 

State and Tribal water quality managers need new and updated water quality criteria and 

information for SABS. 

Charge to the panel: 

While many questions and much research remain, EPA seeks the opportunity for a consultation 

with the Science Advisory Board to gain advice and recommendations on the best potential 

approaches to developing water quality criteria for suspended and bedded sediments as will be 

described in a draft Strategy for Developing Water Quality Criteria for Suspended and Bedded 

Sediments (SABS) to be prepared by the Office of Water. The Office of Water is also seeking 

recommendations on additional criteria development and approaches for uses of water bodies 

other than aquatic life, and it is also seeking advice on any potential criteria derivation 

methodology not included in the Strategy. 
 Peer Review Handbook Page D-4 

CHARGE EXAMPLE 2 - Economics - Benefits Transfer from Adults to Children 

Background 

EPA established the Office of Children’s Health Protection (OCHP) in 1997 to support its efforts 

to increase the protection of children’s health throughout its programs. One of the many difficult 

issues the Office is addressing is the appropriate treatment of children’s health effects in the 

economic analyses performed by the Agency. Policy analysis efforts at the Agency often rely on 

the benefits transfer technique, and very few of the Agency’s benefit transfers have explicitly 

addressed children’s health issues. In addition, no accepted systematic process for conducting 

benefits transfer currently exists. To assist the Agency in its efforts, this paper discusses the 

benefits transfer technique as it applies to estimating values for children’s health. The first 

section provides some general background on the technique, and its application to estimate 

health-related values. The second section raises important general issues to consider when 

conducting a benefits transfer for children’s health values. The last section discusses the 

implications of using the benefits transfer method to estimate values for children’s health. The 

scarcity and state of existing child-oriented health valuation literature suggests that it may be 

necessary to transfer adult-oriented values to estimate child-related values (Neumann and 

Greenwood 1999). However, the results of this paper suggest that transfer of these value 

estimates to children at best provides estimates for a scoping analysis. In cases where these 

scoping exercises indicate that children's health values may be a crucial component in the policy 

analysis, primary research should be undertaken to estimate child-related values. 

a) Assess the appropriateness of transferring health benefit values estimated for adult 

populations to children. Describe the specific issues that arise in these transfers. Which 

variables or situations improve or decrease the appropriateness of transferring benefit 

values from adults to children? 

b) Does the analysis support the proposition that the value of children’s health effects 

should be estimated differently than adult health effects? Why or why not? 

c) Identify issues for further research that would improve our ability to estimate values for 

children’s health effects. 
 Peer Review Handbook Page D-5 

CHARGE EXAMPLE 3 - Economics - Study of Municipal Government Costs and 

Financial Impacts from Environmental Regulations 

Background 

Municipalities play a major role in supplying environmental services. Local governments have 

taken responsibility for providing drinking water, sewage treatment, and waste disposal in a 

majority of communities. Over the past fifteen to twenty years, most of the mandates found in 

the Federal environmental legislation enacted in the early 1970s have been met. The increase in 

the number of people served and improvements in the quality of local environmental services 

have been considerable, as has the investment in public infrastructure to meet these laws. 

Recent revisions to the environmental legislation have established a broader and more stringent 

set of standards to be met by suppliers of environmental services. As a result, many Local 

governments are now faced with having to maintain all or some part of their public services at a 

higher level of performance. To meet these new standards will require additional investments in 

capital, and increases in rates charged to customers for environmental services. 

Improvements in environmental services are but one of several demands being made of local 

public infrastructure. Studies prepared on public infrastructure needs and the availability of 

funds to meet these needs indicate that there will be an excess demand for money to rebuild and 

improve upon the existing stock of public infrastructure. Therefore, it is important to recognize 

that additional environmental requirements will have to compete with other infrastructure needs 

(e.g., highways, bridges), as well as other public services (e.g., police, education, health and 

welfare programs) provided at the local level. 

Given the increasing demand for public services, this study examines what additional 

investments the new environmental legislation will require Local governments to undertake, and 

the likelihood that they will face difficulties raising the necessary funds through capital markets 

and revenues from customers. The economic impacts of individual EPA actions are considered 

during the regulatory process in those situations permitted by environmental statutes. The 

unique feature of this study is its attempt to estimate the cumulative costs and impacts of meeting 

a combined set of EPA requirements, and to determine whether they will place a significant 

burden on the fiscal conditions of Local governments, and require them to significantly increase 

existing charges for improved environmental services. 

Please find attached a copy of the draft study and appendices for your review. This version of 

the report reflects Agency comments received on an earlier draft. We expect that this version of 

the report, with some additional minor modifications, will be the final version. Your comments 

will be useful in preparing the final version of the report and discussing the findings of the 

reports with the public. 

To assist in your review of the report, we ask that you pay particular attention to the following 

questions: 
 

 

 Peer Review Handbook Page D-6 

a) Do State and Local governments and financial markets consider household costs 

(measured as a percent of household income) and selected municipal financial 

information (debt service to general revenues or taxable property values) when 

evaluating the ability of enterprise systems and municipalities to issue bonds or obtain 

loans? 

b) Having selected a series of financial indicators, are the criteria used in the analysis 

acceptable? 

c) Do the results support our conclusions? What additional conclusions can be reached 

from the analysis? 

d) What modifications would you suggest be made to the recommendation section? 
 

 

 Peer Review Handbook Page D-7 

CHARGE EXAMPLE 4 - External Peer Review of the Design of the Targeted National 

Sewage Sludge Survey for Pollutants in Sewage Sludge 

Background Information 

The United States Environmental Protection Agency (EPA), Office of Water is charged with 

protecting public health and the environment from adverse exposure to chemicals and microbial 

agents in water media, such as ambient and drinking waters, wastewater/sewage sludge, and 

sediments. In support of this mission the Office of Water/Office of Science and Technology 

(OST) develops human health and environmental standards, health criteria, health advisories, and 

technical guidance documents for water and water-related media. 

Peer review is an important component of the scientific process. It provides a focused, objective 

evaluation of a research proposal, publication, risk assessment, health advisory, guidance or 

other document submitted for review. The criticisms, suggestions and new ideas provided by 

the peer reviewers ensure objectivity, stimulate creative thought, strengthen the reviewed 

document, and confer scientific credibility on the product. Comprehensive, objective peer 

review leads to good science and product acceptance within the scientific community and is 

required for the technical document that support Agency regulations and guidelines. 

Targeted National Survey of Pollutants in Sewage Sludge 

As EPA described in the December 2003 Federal Register notice (68 FR 75531), EPA 

plans to conduct a targeted survey of pollutants (select pollutants) in sewage sludge. EPA has 

designed a targeted survey of selected chemical pollutants for which adequate data are available, 

or will be available, for conducting an exposure and hazard assessment concerning their presence 

in treated sewage sludge. Microbial pollutants (pathogens and indicator organisms) in sewage 

sludge are also included in the survey design. 

A survey will provide feedback for updating the science and technology of sewage sludge 

applied to land, disposed of in a surface disposal unit, or incinerated. The new concentration 

data would be used to assess human and ecological risks of identified, unregulated pollutants 

found in sewage sludge and identify pollutants for potential regulation. In addition, the data on 

microbial constituents in sewage sludge will help in determining the effectiveness of pathogen 

reduction and elimination processes that are specified in the 40 CFR Part 503 Standards. 

EPA is committed in FY 2005 to starting a limited analytical survey of pollutants found in 

sewage sludge. EPA expects this survey to address the pollutants identified by the exposure and 

hazard screening assessment as presenting a potential hazard, as identified in the current section 

405(d)(2)(C) biennial review (68 FR 75531). The Agency has evaluated the extent to which 

sampling and analytical methodologies will allow expansion of the survey scope within available 

resources to include additional pollutants. 

Questions for Peer Review Charge 
 Peer Review Handbook Page D-8 

a) The sample design proposes excluding facilities with certain treatment types (i.e., 

less than secondary treatment, partial treatment, and ponds), remotely located facilities 

(i.e., outside the contiguous states), and non-publicly owned facilities (i.e., Tribal and 

private). Do these exclusions appear to be appropriate? If you disagree with EPA’s 

proposed exclusions, your response should explain which groups should be included and 

why (e.g., what information will be lost if they are excluded). 

b) The sample design proposes including facilities that the Clean Watershed Needs 

Survey (CWNS) identifies as biosolids handling facilities, because preliminary review 

indicates that they have the same operations and characteristics as other POTWs for 

purposes of biosolids sampling. Do you agree with the inclusion of the biosolids 

handling facilities? If you disagree, your response should explain how their 

characteristics (e.g., type of treatment) would differ from other POTWs. 

c) The sample design proposes including facilities with no discharge, because they 

also may produce biosolids as a result of treatment. Do you agree with the inclusion of 

no discharge facilities? If you disagree, your response should explain how their 

characteristics would differ from other POTWs. 

d) The sample design assumes that flow and mass are highly correlated. Do you 

agree that this is a reasonable assumption? If you disagree, your response should identify 

possible data sources for mass data, or describe alternative approaches to estimating mass 

values based upon available data. 

e) The sample design proposes facility selection based upon the binomial 

distribution. Do you agree that this is the appropriate approach? If you disagree, your 

response should describe your objections to the rationale for selecting the binomial 

distribution. Your response also should recommend another approach, describe this 

approach (unless it was included as an alternative in the sample design), and provide 

rationale for its selection. 
 

 

 

 Peer Review Handbook Page D-9 

CHARGE EXAMPLE 5 - External Peer Review of the Draft National Strategy for the 

Development of Water Quality Criteria for Suspended and Bedded Sediments (SABS) 

BACKGROUND INFORMATION: 

EPA’s Office of Science and Technology (OST) has water quality criteria programs for aquatic 

life, human health, and for nutrients (total N, P, DO, chlorophyll, and clarity). EPA is initiating a 

National Strategy for the Development of Water Quality Criteria for Suspended and Bedded 

Sediments (SABS). SABS occur naturally in all types of waterbodies. In appropriate amounts, 

sediment is essential to aquatic ecosystems. However, imbalanced SABS conditions have 

repeatedly ranked high as a major cause of waterbody impairment. An imbalance of sediments 

resulting from human activities can impact ecological integrity at several scales and trophic 

levels. 

EPA’s Strategy for Water Quality Standards and Criteria, dated August 2003, is a document 

which sets priorities to strengthen the foundation for protecting and restoring the Nation’s 

waters. Imbalance in loading of SABS to the aquatic systems is now considered one of the 

greatest causes of water quality impairment in the Nation. Turbidity, suspended solids, 

sediment, and siltation have been consistently listed as dominant polluting factors in 305(b) 

Water Quality Reports in rivers and streams, lakes and reservoirs, ponds, wetlands and ocean 

shoreline waters. In 1998, for example, approximately 40% of assessed river miles in the U.S. 

were not meeting their water quality goals due to excessive sediments. One of the top ten 

priorities listed in the document is producing and implementing a strategy for the development of 

suspended and bedded sediment criteria. These priorities were developed based on extensive 

listening sessions and discussions with States and other partners. 

To assess the effects of this growing environmental stressor, a number of States have identified 

the specific measurements and levels at which sediment impairment occurs in their waters, but 

many States have not adopted such SABS criteria into their State water quality standards. 

Information from an EPA survey conducted in 2001 indicates that numeric sediment criteria of 

some type were identified in 32 of 53 States, Territories and Washington, DC. But many States 

have not adopted such sediment criteria into their water quality standards. As a result, SABS 

problems are identified but perhaps underestimated, and few controls (water quality standards, 

permit limits, TMDLs) are in place to reduce the overall impact of SABS. 

Also, as part of the current effort to develop national SABS criteria, the EPA Science Advisory 

Board met on October 2, 2003 to discuss the strategy, including various approaches for 

establishing criteria. The Science Advisory Board recommended that several approaches to 

SABS criteria development should be synthesized in an overall approach that can identify 

impaired SABS conditions as compared to appropriate reference conditions. These 

recommendations are presented in the draft strategy document. 

The methodologies presented in this draft strategy will be used by the States, Tribes, and 

Territories to better manage SABS in water bodies across the country. This strategy presents 
 

 

 Peer Review Handbook Page D-10 

EPA’s blueprint for achieving the goals of developing sediment criteria methodologies and the 

implementation of these criteria into water quality standards. 

National Strategy for the Development of SABS Criteria 

The Strategy for the development of SABS criteria describes the approach that EPA is planning 

to take for developing scientific information relating to SABS and working with States to 

encourage adoption of SABS criteria into State water quality standards pursuant to Section 

303(c) of the Clean Water Act. The draft strategy for developing water quality criteria for SABS 

is modeled on the approach used by EPA to develop water quality criteria for nutrients, where 

appropriate. However, the strategy does not establish dates by which States must develop water 

quality standards for SABS. 

States, under pressure to develop and issue total maximum daily loads (TMDLs) for SABS 

impaired water bodies, are moving forward on their own to develop new and improved SABS 

criteria. The draft strategy highlights some of the State programs and looks for approaches and 

methods that may be useful, either directly, or with adaptation, to the entire nation. Because 

current States’ efforts may serve as examples for development of numeric SABS criteria, a 

summary of States’ programs are provided. In order to describe a possible SABS criteria 

development process, an Oregon case study was prepared for a hypothetical aquatic system, 

following a watercourse from the watershed to the receiving water (Section III.F of the draft 

strategy). The case study illustrates sediment criteria development through theoretical and actual 

examples. 

EPA does not anticipate developing ecoregional SABS criteria for all waterbody types and all 

geographic regions. The responsibility to develop and adopt SABS criteria, based on the 

information and methodologies presented in the draft strategy, will rest with States. EPA’s draft 

national strategy includes the following major elements: 

a) Outline of actions to be taken by EPA, States, Territories, and Tribes to develop 

criteria, adopt criteria into water quality standards, and manage for SABS. 

b) Identification of appropriate indicators of water resource impairment due to 

imbalances of SABS. 

c) A process to stratify waterbodies by waterbody type, region, and designated uses 

for the development of SABS criteria. 

d) A series of approaches and technical tools for deriving SABS criteria specific to a 

region, waterbody-type, and designated use. 

e) A list of resources from which States and Tribes can find assistance for deriving 

SABS criteria. 
 

 

 

 

 Peer Review Handbook Page D-11 

f) A case study may be developed to illustrate SABS criteria development, 

addressing both the theoretical basis and an example using a synthesis of 

approaches. 

Following peer review and incorporation of peer review comments, the U.S. Environmental 

Protection Agency (EPA) will publish a draft National Strategy for the Development of Water 

Quality Criteria for Suspended and Bedded Sediment (SABS) and will ask the scientific 

community and the public for its views on the scientific direction of its SABS strategy. Peer 

review is an important component of the scientific process. It provides a focused, objective 

evaluation of a research proposal, publication, risk assessment, health advisory, guidance or 

other document submitted for review. The criticisms, suggestions and new ideas provided by 

the peer reviewers ensure objectivity, stimulate creative thought, strengthen the reviewed 

document, and confer scientific credibility on the product. Comprehensive, objective peer 

review leads to good science and product acceptance within the scientific community and is 

required for the technical document that support Agency regulations and guidelines. 

The Peer Reviewers charge is as follows: 

a) Does EPA’s draft national strategy adequately describe methods for assessing the 

expected SABS condition of a waterbody, for developing SABS criteria and for assessing 

attainment of criteria by States. 

b) Is a regional and watershed approach, as described in the strategy, likely to 

accomplish the desired goal of addressing the sediment imbalance problems in the 

nation’s surface waters? If not, what alternatives do you suggest? 

c) Does the draft strategy present a rational, practical and scientifically responsible 

approach that can be implemented by States and Tribes, given their resources? 

d) Is the state-of-the-science sufficient to write technical methods manuals that will 

allow States, Tribes and territories to determine the best method(s) for developing SABS 

criteria into water quality standards pursuant to Section 303(c) and 304(a) of the CWA? 

e) Are the assessment endpoints and indicators described in the strategy reflective of 

the current or best available science? 

f) Are the six major elements of the draft strategy adequate to help States, Tribes, 

and territories develop and adopt SABS criteria? 

g) Does the case study illustrate sediment criteria development through the 

theoretical and actual examples presented? 
 Peer Review Handbook Page D-12 

CHARGE EXAMPLE 6 - CASAC Review of the Agency’s National Ambient Air 

Monitoring Strategy 

Background. 

The draft National Ambient Air Monitoring Strategy (NAAMS or Strategy) was revised after the 

Agency’s prior consultation with the former CASAC National Ambient Air Monitoring Strategy 

(NAAMS) Subcommittee in July 2003. This revision incorporates EPA’s responses to that 

Subcommittee’s recommendations. The primary recommendations from the NAAMS 

Subcommittee included a request for an implementation plan, and added emphasis on rural- and 

ecosystem-oriented monitoring, support for the National Core Monitoring Network 

(NCore) Level 1 program, and training and quality assurance to enhance data consistency across 

the Nation. 

The Strategy foresees moving resources from programs of decreasing value to those of a higher 

value which respects the partnership across EPA, State, Local, and Tribal (SLT) agencies 

retaining stability for the monitoring programs and allowing SLT flexibility. The transition to the 

NCore network creates a need for training that addresses new methods, information transfer 

technologies, and an effective quality assurance program. There are programmatic and technical 

areas where some type of training or a transfer of information is required. These training needs 

will be offered by various mechanisms, e.g., satellite broadcasts and videos; hands-on sessions; 

guidance documents; vendor training of instrumentation; web-based training; and workshops. 

Broadening the Agency’s outreach to the health effects, atmospheric scientists and ecosystem 

assessment communities are included. 

The implementation plan incorporates action-oriented components of the Strategy, e.g., 

regulation revisions, training, funding, and outreach approaches to facilitate the implementation 

of the NCore program. The revised monitoring regulations will provide a legal basis for moving 

forward and will also alleviate some of the unnecessary burdens faced by monitoring agencies 

and enhance the ability to introduce new technologies into our networks. Additionally, the 

regulations introduce the NCore system of: multiple-tiered monitoring stations; adjustment of 

minimum requirements for specific pollutant measurements; new methods performance 

specifications; periodic network assessments; and new quality assurance procedures. Written 

Meeting Materials. OAQPS has posted written review and background materials for this 

Subcommittee meeting on EPA’s Ambient Monitoring Technology Information Center 

(AMTIC) Web site. The Final Draft NAAMS document, which was updated following the July 

2003 meeting of the former CASAC NAAMS Subcommittee, is at 

http://www.epa.gov/ttn/amtic/files/ambient/monitorstrat/allstrat.pdf. Additional background 

materials for this meeting are found on the “CASAC File Area” page of the AMTIC Web site at 

http://www.epa.gov/ttn/amtic/casacinf.html. Furthermore, it is our understanding that the SAB 

Staff Office will post a copy of the final agenda and charge to the Subcommittee for this 

advisory meeting on the SAB Web site at http://www.epa.gov/sab (under “Meeting Agendas”) 

and the Subcommittee’s page at http://www.epa.gov/sab/panels/casac_aamm_subcom.html, 

respectively, in advance of the Subcommittee’s meeting. 
 

 

 Peer Review Handbook Page D-13 

Charge to the Panel: 

For this advisory meeting, OAQPS is requesting that the CASAC AAMM Subcommittee provide 

expert advice and recommendations on the following charge questions, which focus on key 

implementation issues: 

a) The CASAC has expressed its support for the Agency’s proposal to redesign the 

routine PM monitoring network to support PM precursor gas measurements (CO, 

SO2, NO/NOy, NH3, HNO3) at NCore Level II multiple-pollutant sites, and for 

air quality management decisions and to obtain relevant exposure data for 

research programs. 

Questions: Given limited budgetary resources, does this represent both an 

appropriate and adequate balance, as reflected by the relative resource allocations 

provided in Section 11, “Draft Implementation Plan,” of the Final Draft NAAMS 

Document? In addition, are the relative adjustments in the training and guidance 

approaches proposed in the draft implementation plan consistent with the overall 

objectives of the Strategy? 

b) The implementation plan proposes a series of communication actions to advance 

the NCore Level 2 network, in order to more directly support long-term health 

effects research and provide better support to ecosystem assessments through an 

increased level of coordination. 

Questions: Does the Subcommittee have additional suggestions for addressing 

this need for integration and communication to the broader community of “users,” 

including scientific researchers (i.e., human health, atmospheric, ecological) and 

State, Local and Tribal (SLT) Agency representatives? More specifically, what is 

the most effective manner for EPA both to reach-out to this broad user 

community and, where appropriate, to incorporate their feedback and design input 

on such issues as monitoring site locations and parameters? 

c) One of the remaining technical issues relates to harmonizing rural- and urbanbased 2.5 chemical speciation networks such that both categories of networks 

utilize consistent sampling and analysis protocols. For example, EPA is 

considering converting all of the Speciation Trends Network (STN) speciation 

sites to Interagency Monitoring of Protected Visual Environments (IMPROVE) 

samplers and IMPROVE laboratory and sample handling protocols. 

Question: What are strengths and weaknesses of this approach? 

d) As EPA implements the National Ambient Air Monitoring Strategy to address 

multiple monitoring objectives, it will be looking to spatially optimize the 

ambient monitoring networks. This may mean that some redundant monitors in 
 

 

 Peer Review Handbook Page D-14 

adjacent, but separate, geopolitical areas (e.g., neighboring counties) are 

“divested” from a given network. Although technically sound, these divestments 

could result in data gaps which might, in turn, adversely impact regulatory 

decision-making. The Agency is willing to adopt alternative approaches for 

assessing regulatory issues such as non-attainment designations, so long as such 

approaches are scientifically justifiable; hence, the rationale for initiating 

discussion of these issues with the CASAC. 

Question: Is it scientifically acceptable to generate air quality surfaces through 

modeled observations and/or integrated predictive/observational fields that would 

be of appropriate uncertainty for use in the regulatory decision-making process? 
 Peer Review Handbook Page D-15 

CHARGE EXAMPLE 7 - The Multimedia, Multipathway, and Multireceptor Risk 

Assessment (3MRA) Model 

Background: 

The Office of Solid Waste (OSW) asked the Science Advisory Board to review the Multimedia, 

Multipathway, and Multireceptor Risk Assessment (3MRA) Model and this request was accepted 

by the Board, becoming SAB Project #03-13. A “Widecast” Federal Register notice requesting 

nominations for the Panel was published on April 11, 2003 (68 FR 17797). 

In brief, the development of the 3MRA methodology has a history in the Hazardous Waste 

Identification Rule. EPA plans to use the modeling system to help inform managers on a variety 

of decisions in the waste program, such as setting concentration-based exit criteria for wastes in 

the hazardous waste management regulations, or deciding whether technology-based standards 

are protective of human health and the environment. 

In December 1995, EPA's Office of Solid Waste proposed to amend existing regulations for 

disposal of listed hazardous wastes under the Resource Conservation and Recovery Act(RCRA). 

This Hazardous Waste Identification Rule (HWIR) was designed to establish constituent-specific 

exit levels for low risk solid wastes that are currently captured in the RCRA subtitle C hazardous 

waste system. The EPA's Science Advisory Board (SAB) reviewed the proposed HWIR 

methodology for calculating exit concentrations, concluding that the methodology ``lacks the 

scientific defensibility for its intended regulatory use,'' the SAB also made recommendations 

that, when implemented, should provide an adequate scientific basis for establishing a risk-based 

methodology applicable at the national level for the waste program. 

In response, the Office of Solid Waste (OSW) collaborated with the Office of Research and 

Development (ORD) to develop and document a sound science foundation, supporting data for 

an assessment, and related software technology for an integrated, multimedia modeling system 

(entitled 3MRA) following the recommendations of the SAB and other reviewers. The 

Multimedia, Multipathway, and Multireceptor Risk Assessment (3MRA) modeling system 

represents a collection of science-based models and databases that have been integrated into a 

software infrastructure that is based on the FRAMES (Framework for Risk Analysis in 

Multimedia Environmental Systems) concept, which provides a computer-based environment for 

linking environmental models and databases and managing the large amounts of information 

within the system, including the visualization of outputs. This integrated multimedia modeling 

system provides national-level estimates of human and ecological risks resulting from longterm(chronic) chemical release from land-based waste management units. Over 45 experts 

participated in the peer review process of the underlying science within the 3MRA modeling 

system. 

Charge to the Panel: 
 

 

 

 

 

 

 Peer Review Handbook Page D-16 

The SAB Staff Office Director and OSW staff concurred that the panel should focus upon the 

following four areas: assessment methodology, 3MRA modeling system, modeling system 

evaluation, and modeling system documentation. The panel will respond to the following charge 

questions: 

a) Charge Question 1: While EPA had the assessment methodology peer reviewed prior to 

the development of the 3MRA modeling system, does the SAB have any 

additional comments about the methodology as implemented? 

b) Charge Question 2a: Does the 3MRA modeling system provide a tool for performing 

national risk assessments that facilitates consistent use of the science and provides 

a mechanism for reproducing results? 

c) Charge Question 2b: Does the 3MRA modeling system provide Decision Makers 

sufficient flexibility for understanding the impacts on potential chemical 

exemption levels by allowing varying measures of protection based on the 

number of receptors and/or number of sites protected, types of human and 

ecological receptors, and distance? 

d) Charge Question 2c: Does the 3MRA modeling system provide appropriate information 

for setting national risk-based regulations for the waste program? 

e) Charge Question 3a: Is the software development and verification testing approach 

implemented for the 3MRA modeling system sufficient to ensure confidence that 

the modeling results reflect the modeling system design? 

f) Charge Question 3b: Given the thorough evaluations that EPA has implemented using the 

available data resources and technologies, while also recognizing the real world 

limitations that apply to validating the 3MRA modeling system, have we 

reasonably demonstrated through methodology design, peer review, quality 

control, sensitivity analyses, and model comparison, that the 3MRA modeling 

system will produce scientifically sound results of high utility and acceptance 

with respect to multimedia regulatory applications? 

g) Charge Question 4: Has EPA made substantive progress, relative to 1995, in designing 

and preparing documentation for the 3MRA modeling system? Does the SAB 

have additional suggestions for improving the presentation of the comprehensive 

set of materials related to this modeling system? 
 Peer Review Handbook Page E-1 

Appendix E 

Guidance on Requesting a Review by the 

Science Advisory Board (SAB)
 
 Peer Review Handbook Page E-2 
 

 

 Peer Review Handbook Page E-3 

APPENDIX E – GUIDANCE ON REQUESTING A 

REVIEW BY THE SCIENCE ADVISORY BOARD (SAB) 

Introduction 

The Science Advisory Board Staff Office (SAB SO) solicits nominations for project proposals to 

be reviewed by three of the Agency’s independent scientific advisory committees the Science 

Advisory Board (SAB); the Clean Air Scientific Advisory Committee (CASAC); and the 

Advisory Council on Clean Air Compliance Analysis (Council). This Appendix provides 

guidance to EPA Program Offices and Regions regarding the project proposal nomination and 

selection process. More detailed information about the functions and advisory process for the 

SAB can be found in the Implementation Plan for the New Structural Organization of the EPA 

Science Advisory Board (EPA-SAB-04-002) available on the SAB website at 

http://www.epa.gov/science1/pdf/sab04002.pdf. 

Background 

A key priority for the Administrator is to base Agency actions on sound scientific data, analyses, 

and interpretations, as well as independent peer review. The Administrator issued the Agency's 

Peer Review Policy to increase the quality of the technical foundations upon which EPA's 

regulatory structures are built. The SAB, CASAC, and the Council provide mechanisms for 

EPA to receive independent scientific peer review and advice regarding the development and use 

of science at EPA. The following information is intended to help programs and Regions 

determine which scientific advisory projects to submit to the SAB, CASAC and Council. 

The SAB, CASAC, and Council provide several kinds of advisory functions including the 

following kinds of activities: 

• Consultation – Provides non-consensus, oral advice on a technical issue before EPA 

begins substantive work on that issue. 

• Advisory – Provides written advice on EPA's technical works-in-progress. 

• Peer Review – Conducts a review of EPA's final draft technical reports (e.g., 

guidelines, assessments, research strategies) or work products (e.g., analytical methods, 

models, databases). 

• Commentary – Provides forward-looking advice on an important technical or 

emerging issue in the form of a short communication (SAB function). 

• Original Study – Conducts original work on an emerging or overarching topic of 

importance to EPA (SAB function). 
 

 

 

 

 

 

 

 

 

 

 

 

 Peer Review Handbook Page E-4 

• Other Activities – Receives information briefings from EPA and conducts scientific 

workshops on specific technical subject matters. 

Because resources are always limiting, the SAB Staff Office uses several criteria for selecting 

project proposals proposed by the Agency. Advisory project proposals best suited for 

consideration by the SAB, CASAC, and Council are those that meet several of the following 

criteria: 

• General Criterion 

S Provides an opportunity to make a difference in the science that supports the 

Agency's mission 

• Client-Related Criteria
 

S Supports major regulatory or risk management initiatives 

S Serves leadership interests (e.g. the Administrator or Congress) 

S Supports EPA strategic priorities 

• Science-Driven Criteria
 

S Involves scientific approaches that are new to the Agency 

S Addresses areas of substantial uncertainties 

• Problem-Driven Criteria
 

S Involves major environmental risks 

S Relates to emerging environmental issues 

S Exhibits a long-term outlook 

• Organizational Criteria 

S Serves as a model for future Agency methods 

S Requires the commitment of substantial resources to scientific or technological 

development 

S Transcends organizational boundaries, within or outside EPA (includes 

international boundaries) 

S Strengthens the Agency's basic capabilities. 

In addition, the SAB Staff Office considers the overall mix of the nominated project proposals 

for a specific fiscal year, as well as the time and available resources needed to take on the 

projects. Table 1 depicts the processes for identifying, nominating, and selecting Agency 

nominations, emerging issues for strategic advice, and de novo studies. The SAB, CASAC and 

Council employ similar processes, but projects for the latter two advisory committees are linked 

to specific statutory mandates. 

Process for Submitting Nominations 

Any Office desiring to take a product, activity, or issue to the SAB, for a peer review, 

advisory, or consultation, is requested to complete the two-step process, described below. 
 Peer Review Handbook Page E-5 

TABLE 1 – PROCESS FOR SELECTING SCIENCE ADVISORY BOARD (SAB) P

STEPS TO SAB 

TYPE 

PROJECT

 OF SELECTION 

PROJECT 

IDENTIFICATION 

FORA FOR 

DISCUSSION 

AND NOMINATION 

PRIORITIZATION AND SELECTION 

DEFINING THE 

SCOPE AND 

CHARGE 

AGENCY REQUESTS Achieved through 

discussions within EPA Offices to determine projects which would benefit from external review 

Staff Office receives nominations from EPA Assistant Administrators (AAs) or Regional Administrators (RAs) - Annually - As needed basis 

Achieved through joint discussions with: - Science Policy Council - EPA Program/Regional offices - SAB Selection based on: - Statutory Requirement - Agency needs 

Achieved through joint discussions with: - SAB Staff Office - the requesting EPA Program/Regional office 

IDENTIFYING EMERGING ISSUES FOR SAB STRATEGIC ADVICE AND DE NOVO STUDIES 

Achieved through Interactions with: EPA Program/Regional Offices Scientific Community Public Science Advisory Board SAB Staff Office 

SAB Annual Meeting SAB Committee Meetings Board Meetings Meetings with Public Science Policy Council EPA Program/Regional Offices Scientific Workshops 

Achieved through joint discussions with: - Science Policy Council - EPA Program/Regional offices - SAB To identify projects for further consideration. - Strategic Advice by SAB - De novo SAB project 

Selection based on: - Impact on Agency’s mission - Innovative nature of project - Agency needs 

Achieved through joint discussions with: - Designated SAB Committee - EPA Program/ Regional Office 
 

 

 Peer Review Handbook Page E-6 

Step 1 – Project Identification and Nomination. Each year, the Assistant Administrators and 

Regional Administrators are asked to send the SAB Staff Office Director a memorandum that 

lists all advisory project nominations, with the highest priority nominations for the next fiscal 

year identified. SAB Project Sheets (see Item 2 below) for nominated projects should be 

attached to the memorandum. The SAB Staff Office and the Science Policy Council also request 

this memorandum to identify four to six other high-priority science activities that the Assistant 

Administrator or Regional Administrator has decided not to nominate for SAB peer review. The 

latter information is requested, so that the SAB may better understand nominated projects within 

the context of high-priority science activities at EPA. The SAB Staff Office requests this 

information to assist the Science Policy Council in its discussions and decisions about the most 

appropriate set of projects for the Agency to bring to the Board, CASAC , or the Council. 

Step 2 – Electronic Project Sheet. Nominators are asked to submit an electronic project sheet 

for each individual project to be considered for SAB, CASAC, or Council attention. The project 

sheets are created after establishing or updating the related Peer Review Project or Science 

Activity in the Agency’s Science Inventory. The Science Inventory entries must be approved by 

their Peer Review Coordinator or Science Activity Coordinator for the specific Program Office 

or Region. Project sheets should be filled out for all desired projects, including previously 

submitted projects for which no project planning meeting has occurred between the Program 

Office or Region and the SAB Staff Office Director or the Associate Director for Science. The 

electronic Project sheet may be accessed and completed through EPA’s Science Inventory 

database. The information fields required for the Electronic data sheet are provided in Table 2. 

Process for Keeping Informed about the Decisions Made 

After receiving project nominations, the SAB Staff Office will discuss project priorities 

with each EPA Program Offices and Region, and discuss overall Agency priorities with the EPA 

Science Policy Council. The Staff Office will also discuss project priorities with the SAB, which 

includes the CASAC and Council chairs. The SAB Staff Office will then develop a 

recommended list of advisory projects based on EPA and SAB rankings taking into account 

available resources. The SAB Staff Office will consult with the EPA Administrator and Deputy 

Administrator to develop an annual operating plan that includes the highest priority projects. 

Additionally, projects may be added or deleted at any time during the fiscal year, as requested by 

EPA senior executive management, Congress, and the SAB. 

EPA Staff may refer to the SAB Product Database in an ongoing way to identify all the 

projects nominated for the Board and the status of decisions made on them. The database will 

also provide ongoing information about the status of SAB reports and meetings related to 

projects accepted for Board advice in Fiscal Year 2006. 
 

 

 

 

 

 

 Peer Review Handbook Page E-7 

EPA Staff with questions about the SAB Product Database or the process for submitting 

nominations to the EPA SAB may contact Angela Nugent (nugent.angela@epa.gov, 202-343-

9981) or Ms. Patricia Thomas (thomas.patricial.epa.gov, 202-343-9974). 

Table 2: Information Fields for SAB Project Sheet 

1. Project Title 

2. Project Short Title 

3. Fiscal Year SAB Activity Desired to Begin 

4. Quarter SAB Activity Desired to Begin 

5. Requesting Organization 

6. Requesting Office 

7. Requesting Official 

8. Requesting Official's Title 

9. Program Contact 

10. Program Contact's Phone 

11. Program Contact's Mail Code 

12. Background for this Project: 

13. Tentative Charge: 

14. Applicable GPRA Goal and Objective 

15. Description of and citation for any legal obligation/directive for SAB Review: 

16. Principal interested and affected parties 

17. Type of SAB advice requested 

18. Why Should the SAB Advise on this Project? 

19. Disciplinary Expertise 

20.	 Budget: 

FY 

 Extramural Budget 

FTE
 

21. Past Peer Reviews: 

22. Quality Management/Quality Assurance: 

 Peer Review Handbook Page F-1 

Appendix F 

Example Statements Of Work for 

Contracts
 

Users of this Peer Review Handbook need to be aware that the examples contained in this 

Appendix are generalized statements of work prepared (and in some cases modified) to 

emphasize certain important features (e.g., attention to conflicts of interest, responsibilities of 

contractors or contracting officers, development and use of the charge to peer reviewers). Please 

be sure that proper and currently approved contract language is used in any EPA contract 

document at the time of award (for example, see EPAAR 1552.212-71, alternate I). 
 Peer Review Handbook Page F-2 

STATEMENT OF WORK - EXAMPLE 1 - Peer Review of Prioritization Tool Report 

Work Assignment No.: 

Title: Peer Review of Prioritization Tool Report 

Work Assignment Manager (WAM): 

Name: 

Address: 

Phone No.: 

John Q. Government Employee 

Office of Solid Waste 

(202) 564-XXXX 

Background: 

The Waste Minimization Branch (WMB) in the Office of Solid Waste (OSW) is in the process of 

implementing the Waste Minimization National Plan, announced by the Agency on November 

18, 1994. The Plan reaffirms the Agency's commitment to promote source reduction over waste 

management, in keeping with the policy stated in the 1984 amendments to the Resource, 

Conservation, and Recovery Act (RCRA) and in the 1990 Pollution Prevention Act (PPA). The 

Plan outlines major goals, objectives, and action items to achieve national reductions in the 

generation of hazardous wastes. 

One of the objectives of the Plan is to: "develop a framework for setting national priorities; 

develop and distribute a flexible screening tool for identifying priorities at individual facilities; 

[and] identify constituents of concern." This objective is a key building block in implementing 

subsequent objectives of the Plan. 

In September 1995, WMB formed the Waste Minimization Prioritization Team, which includes 

representatives from EPA Regions and states, to implement this objective. The Team has 

worked to assess stakeholder needs for prioritization tools and to evaluate prioritization tools that 

are currently available. The Team plans to summarize this work, along with its 

recommendations, in a report (referred to herein as the Prioritization Tool report) that would be 

available in draft form in July 1996. 

WMB and the Team wish to obtain independent peer review of the Prioritization Tool report 

prior to briefing EPA management. The report is being prepared with the support of ICF, Inc.; 

therefore, for the peer review to be considered independent, it must be performed by another 

contractor. 

Purpose and Scope of Work: 

The purpose of this work assignment is to provide support to WMB and the Team in finalizing 

the Prioritization Tool report by conducting an independent peer review of the report. 
 Peer Review Handbook Page F-3 

Work Statement: 

Task 1 -Management work plan and budget 

Within 15 days of CO approval of this work assignment, the contractor shall deliver a 

management work plan including a proposed level of effort, schedule, and budget for all tasks. 

Task 2 -Provide independent peer review of Prioritization Tool report 

The contractor shall provide support to WMB and the Team in preparing the Prioritization Tool 

report by performing an independent peer review of the report. The contractor shall establish a 

panel of peer reviewers including three senior-level persons who collectively have extensive 

expertise in particular areas to be identified by the WAM upon approval of the work assignment. 

Within three weeks of work assignment approval, receipt from the WAM of the necessary 

qualifications of peer reviewers (in a TD), and receipt from the WAM of the peer review 

“charge” (in a TD), whichever comes latest, the contractor shall identify the three peer reviewers 

and prepare a memo that lists the names of the peer reviewers and their affiliations and includes 

the peer reviewers’ bio’s. Within five weeks of WAM approval of the of the peer reviewers (via 

a TD) and receipt of the draft Prioritization Tool report from the WAM (via a TD), whichever 

comes later, the contractor shall conduct the peer review, assemble the peer review comments 

and recommendations in a peer review report organized by charge question, prepare an 

introduction to the peer review report with a clear and concise overview of the comments, and 

attach to the peer review report any marginal comments the peer reviewers had on the 

Prioritization Tool report. 

It is not necessary that the peer reviewers jointly reach consensus on their findings and 

recommendations, since there may be limited overlap in the peer reviewers' areas of expertise 

and in the charge questions that they focus on. The contractor shall assume, for the purpose of 

estimating costs, that the draft Prioritization Tool report is roughly 100 pages in length with 200 

pages of appendices, and that each peer reviewer will spend 40 hours in reviewing the report and 

writing comments. EPA plans to provide the report to the contractor in mid-July. 

Deliverables and Schedule: 
 

 

 

 

 Peer Review Handbook Page F-4 

Task Deliverable Schedule 

1 Work plan and budget Within 15 days of CO approval of work assignment 

2 

Memo identifying peer reviewers Within 3 weeks of work assignment approval, receipt of peer reviewer 

qualifications from WAM, and receipt of charge from WAM, 

whichever comes latest 

3 Peer review report Within five weeks of WAM approval of peer reviewers and receipt of 

draft Prioritization Tool report from WAM, whichever comes later 

Other Requirements: 

CONTRACTOR COMMUNICATIONS 

Upon approval of the Work Plan, the contractor shall maintain at least weekly communications 

with the Work Assignment Manager regarding the status of work on the Work Assignment. 

CONFLICT OF INTEREST (COI) [Revised Section] 

The contractor must adhere to the following requirements: 

a) Upon receipt of a Work Assignment, QRT, or similar tasking document, and prior 

to commencement of any work, notify both the CO and Project Officer of any 

actual or potential organizational or personal conflicts of interest. 

b) Provide a written certification, within 20 days of receipt of a Work Assignment, 

QRT, or similar tasking document, that: 

1) Either all conflicts of interest have been reported to the CO or that no 

conflicts of interest exist. The contractor is directed to assure that none of 

the conflicts disclosed are so direct and substantial as to rule out a 

particular reviewer. 

2) All personnel who perform work under this Work Assignment or relating 

to this Work Assignment have been informed of their obligation to report 

personal and organizational conflicts of interest to the CO. 

3) The Contractor recognizes its continuing obligation to identify and report 

any conflicts of interest arising during performance of this Work 

Assignment. 

c) If a conflict of interest is identified during performance under this Work 

Assignment, the Contractor shall immediately make a full disclosure in writing to 

the CO. The disclosure shall include a description of action which the Contractor 
 

 

 

 Peer Review Handbook Page F-5 

has taken or proposes to take, after consultation with the CO, to avoid, mitigate, 

or neutralize the conflict of interest. 

d) After selecting the peer reviewers but before starting the peer review, submit to 

EPA documentation that shows that the contractor has determined if the peer 

review candidates: 

1) have a conflict of interest or a situation that could create the appearance of 

a lack of impartiality in relation to the work product, and 

2) have had or presently have a financial relationship with EPA. 

This documentation summarizes for EPA the Contractor’s efforts to 

identify and propose resolution of these concerns with peer review candidates. 

EXPENDITURE OF FUNDS/HOURS 

In addition to the requirements of the contract, the contractor shall notify both the Project Officer 

and the Work Assignment Manager when 75% of funds or hours for this Work Assignment have 

been expended. 

INFORMATION COLLECTION 

Any other provision of this Work Assignment notwithstanding, the contractor shall not proceed 

with any information collection where the same or similar information will be collected from ten 

or more public respondents until written approval is received from the Contracting Officer. This 

approval will cite an approval number from the Office of Management and Budget as required 

by the Paperwork Reduction Act (PRA). 

Only Federal agencies and their employees are exempt from the PRA definition of "public 

respondent." State agencies and their employees are classified as "public respondents." 

Soliciting similar information applies to any collection method, i.e., written, oral, electronic, etc., 

and utilizing any approach, i.e., surveys, phone calls, focus groups, TQM, etc. The PRA applies 

equally to "willing participants" and participation that is mandated by law. 

Any question of applicability of the PRA shall be resolved by submitting a complete description 

of the circumstances in a written request to the Contracting Officer. No collection shall be 

undertaken until the Contracting Officer provides written notice to the Contractor as to the 

applicability of the PRA. If the PRA is determined to be applicable, the Contractor shall not 

initiate any collection until the requisite approval is received. 

The General Services Administration (GSA), under FIRMR Bulletin B-2 administers the 

Interagency Reports Management Program as derived from 44 USC Chapters 29 and 31. All 
 Peer Review Handbook Page F-6 

work performed under this Work Assignment involving Federal interagency reporting must be 

done in full compliance with these GSA procedures. 

CONFIDENTIAL BUSINESS INFORMATION 

If this Work Assignment requires use of RCRA Confidential Business Information (CBI), the 

contract must specifically authorize the contractor to have access to RCRA CBI and the 

contractor shall abide by all RCRA CBI requirements and stipulations found in the RCRA CBI 

Security Manual and in the contract. The contractor shall identify in the Work Plan budget all 

estimated costs for dealing with CBI requirements. All CBI must be returned to EPA as soon as 

it is no longer needed under this Work Assignment or before the expiration of the Work 

Assignment, whichever occurs first. 

PRINTING AND DUPLICATION 

The contractor is prohibited from performing any printing under the Government Printing and 

Binding Regulations. Duplication is allowed to the extent it does not exceed 5,000 impressions 

of a single-page document or 25,000 impressions of a multiple-page stand-alone document, is 

limited to one color (black) copies, and does not exceed the maximum image size of 10 3/4 by 14 

1/4 inches. For all duplication jobs in excess of 5,000 impressions, the EPA WAM will 

determine in advance if the work can be performed more cost effectively and under the job or 

time constraints at the EPA Print Shop. If the total number of photocopies for this Work 

Assignment exceeds 5,000 impressions, the contractor shall identify in their Work Plan the 

photocopying costs by task and deliverable. 

WORK ASSIGNMENT/WORK PLAN BUDGETS 

The contractor shall not exceed either the dollar or PL hour budget contained in the approved 

Work Plan. In addition, on Quick Response Tasks (QRTs) the contractor shall not exceed the PL 

hour budget of the QRT. 

TECHNICAL DIRECTION 

The Designated Work Assignment Manager (WAM) on this Work Assignment is authorized to 

provide technical direction to the extent allowed under EPAAR (1552.237-71) (APR 1984) 

(DEVIATION). Other than the Designated WAM, only the Project Officer and the Contracting 

Officer are authorized to provide technical direction. 

Technical direction includes: 

(1) Direction to the contractor which assists the contractor in accomplishing the 

Statement of Work; 

(2) Comments on and approval/acceptance of reports or other deliverables. 
 Peer Review Handbook Page F-7 

Technical direction must be within the contract and the Work Assignment statement of work. 

The Project Officer and the WAM do not have the authority to issue technical direction which 

(1) institutes additional work outside the scope of either the contract or this Work 

Assignment; 

(2) constitutes a change as defined in the "Changes" clause; 

(3) causes an increase or decrease in the estimated cost of the contract or Work 

Assignment; 

(4) alters the period of performance or deliverable due dates; or 

(5) changes any of the other express terms or conditions of the contract or Work 

Assignment. 

Technical direction will be issued in writing or confirmed in writing within five (5) calendar 

days after verbal issuance. The technical direction memorandum will be provided to the 

contractor and copies will be forwarded to the Contracting Officer and the Project Officer. If the 

contractor has not received written confirmation within five (5) calendar days of an oral 

issuance, the contractor must notify the Project Officer. 

INHERENTLY GOVERNMENTAL FUNCTIONS 

The contractor shall not perform any inherently governmental functions (IGF) under this Work 

Assignment. If during the course of developing the plan of work, through receipt of technical 

direction, or in carrying out the assignment any portion of the effort is considered to possibly be 

an inherently governmental function, the contractor must immediately notify the Project Officer 

and the Contracting Officer. 

OCCUPATIONAL HEALTH AND SAFETY 

Facility site visits conducted under a Work Assignment that include on-site inspections or 

sampling must be conducted in full compliance with the Department of Labor, Occupational 

Safety, and Health Administration rules under 29 CFR Part 1910 and EPA Order 1440 

(Occupational Health and Safety Manual). 
 Peer Review Handbook Page F-8 

TRAVEL COSTS 

The contractor shall follow the requirements of Subpart 31.2 of the FAR and the Federal 

regulations in incurring allowable travel costs under this Work Assignment, and correspondingly 

must at all times seek and obtain Government rates whenever available and observe current 

subsistence ceilings. 

QUICK RESPONSE TASKS 

Each Quick Response Task (QRT) shall be confirmed in writing and approved by the Project 

Officer. The contractor shall respond by letter to the Project Officer with copies to the WAM 

and the CO within two working days, giving a brief description of the plan of work, including 

best estimate of hours (by P-level) and a break-out of costs to accomplish the task. 

No task shall exceed a duration of 30 calendar days from start date to completion date. The level 

of effort for each task shall be limited to a maximum of 250 labor hours. 

Quick Response Task Requests do not change the dollar or professional labor hour budgets of a 

Work Assignment. 
 

 

 

 Peer Review Handbook Page F-9 

STATEMENT OF WORK - EXAMPLE 2 - External Peer Review of Protozoa Method 

Development Criteria Document 

Period of Performance: Work Plan Approval to August 1, 1997 

Work Assignment Manager: Sally Q. Government Employee 

 Office of Water

 U.S. Environmental 

Protection Agency 

LOE: 196 hours 

SOW: 2.4 

BACKGROUND INFORMATION: 

The United States Environmental Protection Agency (EPA), Office of Water is charged with 

protecting public health and the environment from adverse exposure to chemicals and microbials 

in water media, such as ambient and drinking waters, wastewater/sewage sludge and sediments. 

In support of this mission OW’s Office of Science and Technology (OST) develops health 

standards, health criteria, health advisories, and technical guidance documents for water and 

water-related media. Under this work assignment, documents prepared by OST are to undergo 

peer review. 

Peer review is an important component of the scientific process. It provides a focused, objective 

evaluation of a research proposal, publication, risk assessment, health advisory, guidance or 

other document submitted for review. The criticism, suggestions and new ideas provided by the 

peer reviewers stimulate creative thought, strengthens the reviewed document and confer 

credibility on the product. Comprehensive, objective peer reviews leads to good science and 

product acceptance within the scientific community. 

Under this work assignment, the contractor will receive one document (Protozoa Method 

Development Criteria Document) for peer review which is related to human health and 

ecological effects. 

STATEMENT OF WORK: 

Task 1.	 The contractor shall develop a work plan to address all tasks in this work 

assignment. The work plan shall describe the steps that will be taken by the 

contractor to provide for peer review, including selection of peer reviewer 

candidates with appropriate expertise, determining absence of conflict of interest, 

document and reference distribution, establishing schedules, preparing the peer 

review report, and submittal of the peer review package. Curriculum vitae for all 

persons assigned to complete this work assignment shall be provided. All P levels, 
 Peer Review Handbook Page F-10 

hours and total costs for each task will be provided and costs greater than $100.00 

shall be itemized in detail. 

Task 2.	 The contractor shall select a group of peer reviewers and determine their 

availability for the task and absence of conflict of interest, and establish a schedule 

for the peer review. The contractor is directed to ensure that none of the conflicts 

disclosed is so direct and substantial as to rule out a particular reviewer. Three peer 

reviewers shall participate in the review. No single peer reviewer may charge more 

than 40 hours to this task. It is fully acceptable for peer reviewers to commit to less 

than 40 hours. The peer review will be conducted for the Protozoa Method 

Development Criteria Document. Reviewers selected by and working for the 

contractor shall be approved by the EPA Project Officer in writing prior to their 

beginning work. Minimally, all peer reviewers shall be accomplished in protozoan 

methods for sample recovery and analysis from water. Approval submissions shall 

include the reviewers’ names and curriculum vitae. 

Task 3.	 The contractor shall arrange for the selected peer reviewers to review the EPA 

document. Prepare the charge to the peer reviewers based on technical direction 

received from the EPA WAM. Provide the peer reviewers with copies of the 

candidate report and all relevant references and instruct the selected peer reviewers 

to undertake the review. The WAM will provide the contractor with the final 

version of the document to be reviewed. 

Task 4.	 The contractor shall monitor peer reviewers’ progress to ensure timely completion. 

The contractor shall collate peer review comments, and organize the comments in 

the peer review “for comments” document. The contractor shall provide the peer 

review document and all materials submitted by the peer reviewers to the EPA 

WAM. 

SCHEDULE AND DELIVERABLES: 

Task 1. (Work Plan) 15 days after receipt of work assignment 

Task 2. 1 week after work plan approval 

Task 3. 1 week after selection of peer reviewers 

Task 4. 1 week after receiving comments from the peer reviewers 

TRAVEL: No travel is anticipated under this work assignment. Any travel directly chargeable 

to this work assignment must be submitted and approved by the Project Officer. 
 Peer Review Handbook Page F-11 

STATEMENT OF WORK - EXAMPLE 3 - Technical Support for Peer Review Services 

INTRODUCTION 

Background 

The mission of the United States Environmental Protection Agency’s Office of Water (OW) is to 

protect public health and the environment from adverse effects of pollutants (e.g., chemicals and 

microorganisms) in media such as ambient water, drinking water, wastewater, sewage sludge and 

sediments. In fulfillment of this mission, OW’s Office of Science and Technology (OST) 

develops effluent guidelines, human health and ecological criteria, health advisories, directs the 

national program for adoption of Water Quality Standards, develops prevention and remediation 

strategies to reduce the risk to human health and aquatic life resulting from contaminated fish 

and sediment, provides technical support to the Total Maximum Daily Load program, analytical 

methods for detection and quantification of pollutants in the environment and assesses, 

remediates or otherwise manages and communicates the risks associated with exposure to 

contaminants and microbial pathogens. 

In January 1993, responding to recommendations in the report “Safeguarding the Future: 

Credible Science, Credible Decisions,” Administrator William Reilly issued an Agency-wide 

policy for peer review. Administrator Carol Browner confirmed and reissued the policy on June 

7, 1994 and instituted an Agency-wide implementation program. The Agency policy guides the 

involvement of peers in the Agency’s development of products. This involvement may be 

through peer consultation or peer review of products. The goal of the Peer Review Policy is to 

implement a comprehensive Agency-wide program to ensure that major scientific and technical 

work products receive critical scrutiny from scientific and technical experts as part of the overall 

decision making process. 

Purpose 

The purpose of this contract is to provide peer consultation and peer review support services to 

OST consistent with the Agency Peer Review Policy. (Other offices within the Office of Water 

can utilize this contract for efforts within this Statement of Work, if the OST Project Officer 

determines capacity is sufficient, based upon OST projections and requirements). The purpose of 

peer review is to uncover any technical problems or unresolved issues for use in revising a 

preliminary product so that the final work product will reflect sound technical information and 

analyses. These processes enhance the quality of and provide for objective review and 

evaluation of scientific or technical work by knowledgeable experts who have no vested interest 

in related policy or implementation issues. 

Scientific and technical work products identified for peer review may address one or more of the 

following: 

o Support major regulatory decisions or policy/guidance of major impact 
 

 

 

 

 Peer Review Handbook Page F-12 

o Establish a significant precedent, model or methodology 

o Address a controversial issue 

o Focus on a significant emerging issue 

o Have significant cross-Agency/interagency implications 

o Involve significant investment of Agency resources 

o Consider an innovative approach for a previously defined 

problem/process/methodology 

o Satisfy a statutory or other legal mandate for peer review. 

o Other products deemed appropriate by OST 

The contractor is responsible for preparing reports on peer review evaluations and shall provide 

support for all stages or portions of the peer review process. The term peer review shall mean 

both peer consultation and peer review of products. Peer review will include initial research 

protocol or work plan, preliminary review of an interim draft, review of a draft final work 

product and final evaluation of the work product. All peer review procedures, products, and 

responsibilities shall be in accordance with the latest revision of the “U.S. Environmental 

Protection Agency, Peer Review Handbook.” 

Statement of Work 

1.0 Administration and Management 

1.1 Candidates for peer review include scientific, engineering and economic documents, 

methodologies, strategies or positions that are used to support a research agenda, regulatory 

program, policy position or other OST decision. OST has several general categories of scientific 

and technical work products which are subject to peer review: Criteria Methodologies/Criteria 

and Advisories; Engineering and Technical Studies; Reports to Congress; Technical Guidance 

and Methodologies; Economic Evaluations; Statistical Analyses; Environmental Models; 

Technical Publications, Products, and Approaches; Analytical Methods; and Policies and 

Strategies. The contractor shall provide the necessary professional and technical personnel for 

review of the following types of products: 

o Toxicological Assessments 

o Exposure Assessments 

o Analytic Methods 

o Statistical Analyses 

o Dose-Response Assessments 

o Microbial Risk Assessments 

o Environmental Sampling Methods 

o Monte Carlo Analyses 

o Risk Assessment Guidelines 

o Engineering and Technology Analyses 

o Economic Analyses 

o Technology Analyses 
 Peer Review Handbook Page F-13 

1.2 The contractor shall appoint a person to serve as the Program Manager for this contract. 

The Program Manager shall be the main point of contact for the contract; shall be responsible for 

the performance of work under this contract, and shall be designated a key person in accordance 

with the Key Personnel Clause in EPAAR 1552.237-72(APR 1984). The contractor shall 

manage all aspects of this contract and shall be solely responsible for the direction and selection 

of all contractor, subcontractor and consultant personnel who may be involved in the 

performance of the contract. 

1.3 The contractor shall furnish the facilities, materials, equipment and necessary 

professional, technical and support personnel for performance of work required under this 

contract. The contractor shall develop and institute internal management controls that will 

ensure that, in the performance of this contract, the contractor will not provide personal services 

nor perform inherently governmental functions. For these purposes, personal services are 

defined in EPA Order 1900.1 dated October 31, 1985, and inherently governmental functions are 

defined in EPA Order 1900.2 dated October 22, 1990, and OFPP Policy Letter 92-1, dated 

December 1, 1992. 

2.0 Peer Review Support 

2.1 Selection of Peer Reviewers 

The contractor shall provide Peer Review Services. Peer reviewers will be selected for 

independence and scientific/technical expertise. The scientific/technical peer reviewers shall be 

recognized nationally or internationally in their field and have a general knowledge of 

environmental science issues as well as the specific knowledge, expertise or experience as 

required by the Work Assignment. An independent peer reviewer is an expert who was not 

associated with the generation of the specific work product either directly by substantial 

contribution to its development or indirectly by consultation during the development of the 

specific product (see Note below). Knowledge about the subject area is not equivalent to 

expertise in the subject matter. For Agency decision, a multi-disciplinary group of experts 

corresponding to the disciplines that contribute to complex Agency decisions is often necessary 

for a full and complete peer review. For example, a risk assessment that relies on both animal 

and human data often requires experts in both areas for a complete review. The Project Officer 

(PO)/Work Assignment Manager (WAM) will provide the contractor with the document to be 

reviewed. The PO/WAM will, when available, provide the contractor with names of known 

experts (listed in alphabetical order) in the required disciplines for the contractor to make final 

selection. 

With submission of the work plan or within the timeframe identified in the Work Assignment, 

the contractor shall identify a group of candidate peer reviewers, either from the pool provided 

by EPA, from contractor in-house staff experts, or from a list independently identified by the 

contractor (or a combination) and determine their availability for the task. The number of peer 

reviewers required for each review will be identified in the work assignment. The contractor 

shall submit its selection of peer reviewers, along with their curriculum vitae to the EPA 
 

 

 Peer Review Handbook Page F-14 

PO/WAM. The Agency will review the proposed candidates for consistency with the 

requirements of the work assignment , based either on qualifications, conflicts of interest, or past 

direct involvement with the work under review, however, EPA shall refrain from suggesting 

particular individuals to replace such candidates. Once the reviewers are confirmed, the 

PO/WAM will provide the contractor with the final version of the charge and copies of relevant 

reference materials. 

The contractor shall not be involved in the review of documents prepared by the contractor, or its 

subcontractor or consultants. It is the responsibility of the contractor to ensure that all peer 

reviews are conducted in a manner to avoid all actual, potential or apparent conflicts of interest. 

The contractor shall obtain and provide written certification from all peer reviewers prior to the 

commencement of work that no personal or organizational conflicts of interest exist. 

It is the Agency’s policy to have panels of experts that represent diverse constituencies, e.g. 

industry, academia, public interest groups, Federal (non-EPA) Agencies, State and Local 

governments, so long as the disciplinary expertise requirements of the panel are also met. The 

contractor shall attempt in all cases to fulfill this criteria. Cultural diversity is also to be sought, 

however, disciplinary expertise is the number one criterion for selection. 

Note: Peer reviewers of a product shall be independent experts which have not been 

substantially involved in the development of the product. In special cases where the number of 

experts in a disciplinary area is very small or not available at the specifically designated time; the 

contractor may select a peer reviewer who has been a peer consultant or prior reviewer of the 

product. If this selection is necessary, the contractor shall submit documentation of the prior 

involvement in the development process of the product by the potential reviewer, along with a 

control plan to avoid bias or lack of impartiality, for EPA CO (Contracting Officer) and Project 

Officer review and consent. If special circumstances exist where the expertise is so narrow that 

another peer reviewer isn’t available, the EPA CO/PO, along with the Peer Review Leader, will 

determine the appropriate involvement and controls for the potential reviewer, so that no 

perception of bias or conflict exists. 

2.2 Information and Charge to Peer Reviewers 

EPA will provide to the contractor written technical direction identifying the charge to be given 

to the peer reviewers. The peer reviewers shall consider EPA-supplied and EPA-specified 

documents along with EPA-provided instructions on the exact nature of the assignment (as 

outlined in the EPA Work Assignment). The charge to peer reviewers will usually make two 

general requests. First, it will focus the review by presenting specific questions and concerns 

that the Agency expects the reviewers to address. Secondly, it invites general comments on the 

entire work product. The contractor shall provide the charge to the peer reviewers along with the 

product to be peer reviewed and all reference materials, and instruct the peer reviewers to 

undertake the review. Where appropriate, raw data and/or data in summary form will be provided 

and reviewers will be expected to rely on their assessment of such data in performing their 

review. Each peer reviewer must be informed of the need for confidentiality with regard to the 
 Peer Review Handbook Page F-15 

release of Agency products that are stamped as “DRAFT” or “DRAFT - Do Not Cite, Quote or 

Release.” The peer reviewer and the contractor will agree that the draft shall not be given to 

others who are not under contractor direction without the written approval of the PO/WAM. 

Any requests for such drafts should be forwarded to the EPA Contracting Officer for action. 

2.3 Peer Review Report 

The contractor shall monitor peer reviewers’ progress to ensure timely completion. All peer 

review comments shall be submitted to the EPA PO/WAM for the particular work assignment. 

Peer reviewers will be required to submit with their evaluations or comments a description of the 

procedures used to arrive at their recommendations; a summary of their findings; a list of sources 

relied upon; and make clear and substantiate the methods and considerations upon which their 

recommendations are based. When a document assigned under a task order requires more than 

one peer reviewer, the contractor shall summarize all review comments and present this 

summary in an integrated manner that facilitates an overall evaluation (e.g., an annotated copy) , 

and shall collate them into a peer review report. The contractor shall be tasked with the 

compilation, categorization, and summarization of the comments. The peer review report shall 

be submitted to EPA in a specified format by a specified deadline. The format and deadline will 

be provided in the work assignment issued for the work. The contractor shall submit to the EPA 

PO/WAM the peer review report and all materials submitted by the peer reviewers. EPA shall 

evaluate and analyze all peer review comments and recommendations to ensure technical 

soundness and adherence to Agency policy. Agency officials should document their evaluations 

of the quality and validity of the peer review. 

2.4 Peer Review Meetings 

The contractor shall discuss peer review comments with the EPA PO, WAM, and/or other peer 

reviewers in telephone conferences, peer review workshops, and/or other meetings. Telephone 

conferences, workshops and meetings shall be conducted in accordance with work assignment 

specifications, which may require taping of and transcription of any proceedings. The scientific 

expertise required shall include the broad array of the scientific disciplines relevant to the 

specific document under development or review. In instances when a document or issue paper is 

reviewed, the contractor may be required to meet and document widely varying topics which will 

be defined specifically by the work assignment. The contractor shall organize and provide 

support in arranging and conducting peer review meetings. The contractor shall obtain all postmeeting comments, collect and compile all comments and suggested document revision, and 

return all required information/documentation to the PO/WAM. Occasionally, searches of 

recent literature and hard copy or microfiche retrieval of germane papers may be required. 
 Peer Review Handbook Page F-16 

2.5 Peer Review Meeting Support 

Peer Review meeting logistical support includes bringing together appropriate experts to address 

various issues and concerns raised by EPA for selected products. The contractor may be 

required to (if specified in the work assignment): 

1)	 prepare and distribute presentation materials, including pre-conference information , 

agenda, pre-registration forms, fact sheets, overheads, slide presentations, etc.; 

2)	 arrange for meeting space when government space is not available, reserve meeting 

rooms, copying and audiovisual equipment, furniture/room setups, microphones, and 

related services to accommodate meeting needs; 

3)	 provide for, prepare, and convey all materials such as name badges, table tents, visual 

aids, displays, etc., needed for pre-conference and onsite activities; 

4) inspect the conference site in accordance with expected facilities and services; 

5) handle registration, distribution of meeting materials, and attendance list; 

6) provide on-site recorder and/or facilitator; 

7) arrange reporting , transcription and/or note taking services, typing, reproduction, and 

related support services as required by meeting needs, including rapid photocopying for 

important materials to be copied at meetings; 

8) operate audiovisual equipment, microphones, other presentation related support 

equipment, and/or assist participants in operating such equipment; 

9) handle close-down of the site, reviewing all bills for accuracy, and completing 

payments for equipment rental, etc. 

10)	 A draft and final report may be prepared, in accordance with the requirements as 

specified in the work assignment, and in accordance with the specifications of 

Statement of Work Parts 2.3 and 2.4 herein. 

11)	 If required, distribute draft proceeding summaries to the participants for comment and 

make final revisions to the document. 

The length of the meeting, expected attendance, seating arrangements and audiovisual equipment 

needs will be specifically identified in the work assignment issued by the CO. 

The contractor shall arrange travel only in accordance with the authority and limitations of the 

Section H Clause entitled “Approval of Contractor Travel”, i.e., use of contract funds to 

reimburse travel is strictly limited to logistical support for peer reviewers who directly contribute 

to the requirements of specific contract performance related to the statement of work of the 

contract which supports the mission of OST. In providing logistical support within the scope, 

the contractor may claim transportation, lodging and subsistence for the personnel of the 

contractor (including subcontractors and consultants with a contractual agreement with the 

prime), who, by reason of their role within the scope of the Work Assignment are required to 

attend the meeting. The contractor may be expected to reserve a block of lodging rooms for all 

reviewers and participants in a meeting. However, those participants who are Federal employees 

must then make their own room reservations, travel arrangements, and travel funding 

arrangements.