2664 Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 

as Red, or it would not be categorized 

at Severity Level I, 

(3) The licensee submits a letter of 

intent by December 31, 2005, stating its 

intent to transition to 10 CFR 50.48(c). 

After December 31, 2005, as 

addressed in (3) above, this enforcement 

discretion for implementation of 

corrective actions for existing identified 

noncompliances will not be available 

and the requirements of 10 CFR 50.48(b) 

(and any other requirements in fire 

protection license conditions) will be 

enforced in accordance with normal 

enforcement practices. 

Dated at Rockville, MD, this 11th day of 

January, 2005. 

For the Nuclear Regulatory Commission. 

Annette L. Vietti-Cook, 

Secretary of the Commission. 

[FR Doc. 05–887 Filed 1–13–05; 8:45 am] 

BILLING CODE 7590–01–P 

NUCLEAR REGULATORY 

COMMISSION 

Sunshine Act Meeting Notice 

AGENCY: Nuclear Regulatory 

Commission. 

DATE: Week of January 17, 2005. 

PLACE: Commissioners’ Conference 

Room, 11555 Rockville Pike, Rockville, 

Maryland. 

STATUS: Public.


ADDITIONAL MATTERS TO BE CONSIDERED: 


Week of January 17, 2005 

Tuesday, January 18, 2005 

9:55 a.m. Affirmation Session (Public 

Meeting) (Tentative). 

a. System Energy Resources Inc. 

(Early Site Permit for Grand Gulf 

Nuclear Site), Docket Number 52–009, 

Appeal by National Association for the 

Advancement of Colored People— 

Claiborne County, Mississippi Branch, 

Nuclear Information Service, Public 

Citizen, and Mississippi Chapter of the 

Sierra Club from LBP–04–19. 

(Tentative). 

b. Louisiana Energy Services, L.P. 

(National Enrichment Facility) 

(Tentative). 

*The schedule for Commission 

meetings is subject to change on short 

notice. To verify the status of meetings 

call (recording)—(301) 415–1292. 

Contact person for more information: 

Dave Gamberoni, (301) 415–1651. 

* * * * * 

The NRC Commission Meeting 

Schedule can be found on the Internet 

at: http://www.nrc.gov/what-we-do/ 

policy-making/schedule.html. 

* * * * * 

The NRC provides reasonable 

accommodation to individuals with 

disabilities where appropriate. If you 

need a reasonable accommodation to 

participate in these public meetings, or 

need this meeting notice or the 

transcript or other information from the 

public meetings in another format (e.g. 

braille, large print), please notify the 

NRC’s Disability Program Coordinator, 

August Spector, at (301) 415–7080, 

TDD: (301) 415–2100, or by e-mail at 

aks@nrc.gov. Determinations on 

requests for reasonable accommodation 

will be made on a case-by-case basis. 

* * * * * 

This notice is distributed by mail to 

several hundred subscribers; if you no 

longer wish to receive it, or would like 

to be added to the distribution, please 

contact the Office of the Secretary, 

Washington, DC 20555 (301) 415–1969). 

In addition, distribution of this meeting 

notice over the Internet system is 

available. If you are interested in 

receiving this Commission meeting 

schedule electronically, please send an 

electronic message to dkw@nrc.gov. 

Dated: January 11, 2005. 

Dave Gamberoni, 

Office of the Secretary. 

[FR Doc. 05–890 Filed 1–12–05; 9:32 am] 

BILLING CODE 7590–01–M 

OFFICE OF MANAGEMENT AND 

BUDGET 

Final Information Quality Bulletin for 

Peer Review 

AGENCY: Office of Management and 

Budget, Executive Office of the 

President. 

ACTION: Final bulletin. 

SUMMARY: On December 16, 2004, the 

Office of Management and Budget 

(OMB), in consultation with the Office 

of Science and Technology Policy 

(OSTP), issued its Final Information 

Quality Bulletin for Peer Review to the 

heads of departments and agencies 

(available at http:// 

www.whitehouse.gov/omb/memoranda/ 

fy2005/m05–03.html). This new 

guidance is designed to realize the 

benefits of meaningful peer review of 

the most important science 

disseminated by the Federal 

Government. It is part of an ongoing 

effort to improve the quality, objectivity, 

utility, and integrity of information 

disseminated by the Federal 

Government to the public. This final 

bulletin has benefited from an extensive 

stakeholder process. OMB originally 

requested comment on its ‘‘Proposed 

Bulletin on Peer Review and 

Information Quality,’’ published in the 

Federal Register on September 15, 2003. 

OMB received 187 public comments 

during the comment period (available at 

http://www.whitehouse.gov/omb/ 

inforeg/2003iq/iq_list.html). In addition, 

to improve the draft Bulletin, OMB 

encouraged federal agencies to sponsor 

a public workshop at the National 

Academy of Sciences (NAS). The NAS 

workshop (November 18, 2003, at the 

National Academies in Washington, DC) 

attracted several hundred participants, 

including leaders in the scientific 

community (available at http:// 

www7.nationalacademies.org/stl/ 

STL_Peer_Review_Agenda.html). OMB 

also participated in outreach activities 

with major scientific organizations and 

societies that had expressed specific 

interest in the draft Bulletin. A formal 

interagency review of the draft Bulletin, 

resulting in detailed comments from 

numerous Federal departments and 

agencies, was undertaken in 

collaboration with the White House 

Office of Science and Technology 

Policy. In light of the substantial interest 

in the Bulletin, including a wide range 

of constructive criticisms of the initial 

draft, OMB decided to issue a revised 

draft for further comment. This revised 

draft was published in the Federal 

Register on April 28, 2004, and solicited 

a second round of public comment. The 

revised draft stimulated a much smaller 

number of comments (57) (available at: 

http://www.whitehouse.gov/omb/ 

inforeg/peer2004/list_peer2004.html). 

OMB’s response to the additional 

criticisms, suggestions, and refinements 

offered for consideration is available at: 

http://www.whitehouse.gov/omb/ 

inforeg/peer2004/peer_response.pdf. 

The final Bulletin includes refinements 

that strike a balance among the diverse 

perspectives expressed during the 

comment period. Part I of the 

SUPPLEMENTARY INFORMATION below 

provides background. Part II provides 

the text of the final Bulletin. 

DATES: The requirements of this 

Bulletin, with the exception of those in 

Section V (Peer Review Planning), apply 

to information disseminated on or after 

June 16, 2005. However, they do not 

apply to information for which an 

agency has already provided a draft 

report and an associated charge to peer 

reviewers. The requirements in Section 

V regarding ‘‘highly influential 

scientific assessments’’ are effective 

June 16, 2005. The requirements in 

Section V regarding ‘‘influential 

scientific information’’ are effective 

December 16, 2005. 

Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 2665 

FOR FURTHER INFORMATION CONTACT: Dr. 

Margo Schwab, Office of Information 

and Regulatory Affairs, Office of 

Management and Budget, 725 17th 

Street, NW., New Executive Office 

Building, Room 10201, Washington, DC 

20503. Telephone (202) 395–5647 or 

email: OMB_peer_review@omb.eop.gov. 

SUPPLEMENTARY INFORMATION: 

Introduction 

This Bulletin establishes that 

important scientific information shall be 

peer reviewed by qualified specialists 

before it is disseminated by the Federal 

government. We published a proposed 

Bulletin on September 15, 2003. Based 

on public comments, we published a 

revised proposal for additional 

comment on April 28, 2004. We are now 

finalizing the April version, with minor 

revisions responsive to the public’s 

comments. 

The purpose of the Bulletin is to 

enhance the quality and credibility of 

the government’s scientific information. 

We recognize that different types of peer 

review are appropriate for different 

types of information. Under this 

Bulletin, agencies are granted broad 

discretion to weigh the benefits and 

costs of using a particular peer review 

mechanism for a specific information 

product. The selection of an appropriate 

peer review mechanism for scientific 

information is left to the agency’s 

discretion. Various types of information 

are exempted from the requirements of 

this Bulletin, including time-sensitive 

health and safety determinations, in 

order to ensure that peer review does 

not unduly delay the release of urgent 

findings. 

This Bulletin also applies stricter 

minimum requirements for the peer 

review of highly influential scientific 

assessments, which are a subset of 

influential scientific information. A 

scientific assessment is an evaluation of 

a body of scientific or technical 

knowledge that typically synthesizes 

multiple factual inputs, data, models, 

assumptions, and/or applies best 

professional judgment to bridge 

uncertainties in the available 

information. To ensure that the Bulletin 

is not too costly or rigid, these 

requirements for more intensive peer 

review apply only to the more important 

scientific assessments disseminated by 

the Federal government. 

Even for these highly influential 

scientific assessments, the Bulletin 

leaves significant discretion to the 

agency formulating the peer review 

plan. In general, an agency conducting 

a peer review of a highly influential 

scientific assessment must ensure that 

the peer review process is transparent 

by making available to the public the 

written charge to the peer reviewers, the 

peer reviewers’ names, the peer 

reviewers’ report(s), and the agency’s 

response to the peer reviewers’ report(s). 

The agency selecting peer reviewers 

must ensure that the reviewers possess 

the necessary expertise. In addition, the 

agency must address reviewers’ 

potential conflicts of interest (including 

those stemming from ties to regulated 

businesses and other stakeholders) and 

independence from the agency. This 

Bulletin requires agencies to adopt or 

adapt the committee selection policies 

employed by the National Academy of 

Sciences (NAS) 1 when selecting peer 

reviewers who are not government 

employees. Those that are government 

employees are subject to federal ethics 

requirements. The use of a transparent 

process, coupled with the selection of 

qualified and independent peer 

reviewers, should improve the quality of 

government science while promoting 

public confidence in the integrity of the 

government’s scientific products. 

Peer Review 

Peer review is one of the important 

procedures used to ensure that the 

quality of published information meets 

the standards of the scientific and 

technical community. It is a form of 

deliberation involving an exchange of 

judgments about the appropriateness of 

methods and the strength of the author’s 

inferences.2 Peer review involves the 

review of a draft product for quality by 

specialists in the field who were not 

involved in producing the draft. 

The peer reviewer’s report is an 

evaluation or critique that is used by the 

authors of the draft to improve the 

product. Peer review typically evaluates 

the clarity of hypotheses, the validity of 

the research design, the quality of data 

collection procedures, the robustness of 

the methods employed, the 

appropriateness of the methods for the 

hypotheses being tested, the extent to 

which the conclusions follow from the 

analysis, and the strengths and 

limitations of the overall product. 

Peer review has diverse purposes. 

Editors of scientific journals use 

reviewer comments to help determine 

whether a draft scientific article is of 

sufficient quality, importance, and 

interest to a field of study to justify 

1 National Academy of Sciences, ‘‘Policy and 

Procedures on Committee Composition and Balance 

and Conflicts of Interest for Committees Used in the 

Development of Reports,’’ May 2003: Available at: 

http://www.nationalacademies.org/coi/index.html. 

2 Carnegie Commission on Science, Technology, 

and Government, Risk and the Environment: 

Improving Regulatory Decision Making, Carnegie 

Commission, New York, 1993: 75. 

publication. Research funding 

organizations often use peer review to 

evaluate research proposals. In addition, 

some Federal agencies make use of peer 

review to obtain evaluations of draft 

information that contains important 

scientific determinations. 

Peer review should not be confused 

with public comment and other 

stakeholder processes. The selection of 

participants in a peer review is based on 

expertise, with due consideration of 

independence and conflict of interest. 

Furthermore, notice-and-comment 

procedures for agency rulemaking do 

not provide an adequate substitute for 

peer review, as some experts— 

especially those most knowledgeable in 

a field—may not file public comments 

with Federal agencies. 

The critique provided by a peer 

review often suggests ways to clarify 

assumptions, findings, and conclusions. 

For instance, peer reviews can filter out 

biases and identify oversights, 

omissions, and inconsistencies.3 Peer 

review also may encourage authors to 

more fully acknowledge limitations and 

uncertainties. In some cases, reviewers 

might recommend major changes to the 

draft, such as refinement of hypotheses, 

reconsideration of research design, 

modifications of data collection or 

analysis methods, or alternative 

conclusions. However, peer review does 

not always lead to specific 

modifications in the draft product. In 

some cases, a draft is in excellent shape 

prior to being submitted for review. In 

others, the authors do not concur with 

changes suggested by one or more 

reviewers. 

Peer review may take a variety of 

forms, depending upon the nature and 

importance of the product. For example, 

the reviewers may represent one 

scientific discipline or a variety of 

disciplines; the number of reviewers 

may range from a few to more than a 

dozen; the names of each reviewer may 

be disclosed publicly or may remain 

anonymous (e.g., to encourage candor); 

the reviewers may be blinded to the 

authors of the report or the names of the 

authors may be disclosed to the 

reviewers; the reviewers may prepare 

individual reports or a panel of 

reviewers may be constituted to produce 

a collaborative report; panels may do 

their work electronically or they may 

meet together in person to discuss and 

prepare their evaluations; and reviewers 

may be compensated for their work or 

they may donate their time as a 

3 William W. Lowrance, Modern Science and 

Human Values, Oxford University Press, New York, 

NY 1985: 85. 

2666 Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 

contribution to science or public 

service. 

For large, complex reports, different 

reviewers may be assigned to different 

chapters or topics. Such reports may be 

reviewed in stages, sometimes with 

confidential reviews that precede a 

public process of panel review. As part 

of government-sponsored peer review, 

there may be opportunity for written 

and/or oral public comments on the 

draft product. 

The results of peer review are often 

only one of the criteria used to make 

decisions about journal publication, 

grant funding, and information 

dissemination. For instance, the editors 

of scientific journals (rather than the 

peer reviewers) make final decisions 

about a manuscript’s appropriateness 

for publication based on a variety of 

considerations. In research-funding 

decisions, the reports of peer reviewers 

often play an important role, but the 

final decisions about funding are often 

made by accountable officials based on 

a variety of considerations. Similarly, 

when a government agency sponsors 

peer review of its own draft documents, 

the peer review reports are an important 

factor in information dissemination 

decisions but rarely are the sole 

consideration. Agencies are not 

expected to cede their discretion with 

regard to dissemination or use of 

information to peer reviewers; 

accountable agency officials must make 

the final decisions. 

The Need for Stronger Peer Review 

Policies 

There are a multiplicity of science 

advisory procedures used at Federal 

agencies and across the wide variety of 

scientific products prepared by 

agencies.4 In response to congressional 

inquiry, the U.S. General Accounting 

Office (now the Government 

Accountability Office) documented the 

variability in both the definition and 

implementation of peer review across 

agencies.5 The Carnegie Commission on 

Science, Technology and Government 6 

has highlighted the importance of 

‘‘internal’’ scientific advice (within the 

agency) and ‘‘external’’ advice (through 

scientific advisory boards and other 

mechanisms). 

A wide variety of authorities have 

argued that peer review practices at 

4 Sheila Jasanoff, The Fifth Branch: Science 

Advisors as Policy Makers, Harvard University 

Press, Boston, 1990. 5 U.S. General Accounting Office, Federal 

Research: Peer Review Practices at Federal Agencies 

Vary, GAO/RCED–99–99, Washington, DC, 1999. 6 Carnegie Commission on Science, Technology, 

and Government, Risk and the Environment: 

Improving Regulatory Decision Making, Carnegie 

Commission, New York, 1993: 90. 

federal agencies need to be 

strengthened.7 Some arguments focus 

on specific types of scientific products 

(e.g., assessments of health, safety and 

environmental hazards).8 The 

Congressional/Presidential Commission 

on Risk Assessment and Risk 

Management suggests that ‘‘peer review 

of economic and social science 

information should have as high a 

priority as peer review of health, 

ecological, and engineering 9 information.’’ 

Some agencies have formal peer 

review policies, while others do not. 

Even agencies that have such policies 

do not always follow them prior to the 

release of important scientific products. 

Prior to the development of this 

Bulletin, there were no governmentwide standards concerning when peer 

review is required and, if required, what 

type of peer review processes are 

appropriate. No formal interagency 

mechanism existed to foster crossagency sharing of experiences with peer 

review practices and policies. Despite 

the importance of peer review for the 

credibility of agency scientific products, 

the public lacked a consistent way to 

determine when an important scientific 

information product is being developed 

by an agency, the type of peer review 

planned for that product, or whether 

there would be an opportunity to 

provide comments and data to the 

reviewers. 

This Bulletin establishes minimum 

standards for when peer review is 

7 National Academy of Sciences, Peer Review in 

the Department of Energy—Office of Science and 

Technology, Interim Report, National Academy 

Press, Washington, DC, 1997; National Academy of 

Sciences, Peer Review in Environmental Technology 

Development: The Department of Energy—Office of 

Science and Technology, National Academy Press, 

Washington, DC, 1998; National Academy of 

Sciences, Strengthening Science at the U.S. 

Environmental Protection Agency: ResearchManagement and Peer-Review Practices, National 

Academy Press, Washington, DC, 2000; U.S. 

General Accounting Office, EPA’s Science Advisory 

Board Panels: Improved Policies and Procedures 

Needed to Ensure Independence and Balance, 

GAO–01–536, Washington, DC, 2001; U.S. 

Environmental Protection Agency, Office of 

Inspector General, Pilot Study: Science in Support 

of Rulemaking 2003–P–00003, Washington, DC, 

2002; Carnegie Commission on Science, 

Technology, and Government, In the National 

Interest: The Federal Government in the Reform of 

K–12 Math and Science Education, Carnegie 

Commission, New York, 1991; U.S. General 

Accounting Office, Endangered Species Program: 

Information on How Funds Are Allocated and What 

Activities are Emphasized, GAO–02–581, 

Washington, DC, 2002. 

8 National Research Council, Science and 

Judgment in Risk Assessment, National Academy 

Press, Washington, DC, 1994. 

9 Presidential/Congressional Commission on Risk 

Assessment and Risk Management, Risk 

Commission Report, Volume 2, Risk Assessment 

and Risk Management in Regulatory DecisionMaking, 1997:103. 

required for scientific information and 

the types of peer review that should be 

considered by agencies in different 

circumstances. It also establishes a 

transparent process for public 

disclosure of peer review planning, 

including a Web-accessible description 

of the peer review plan that the agency 

has developed for each of its 

forthcoming influential scientific 

disseminations. 

Legal Authority for the Bulletin 

This Bulletin is issued under the 

Information Quality Act and OMB’s 

general authorities to oversee the quality 

of agency information, analyses, and 

regulatory actions. In the Information 

Quality Act, Congress directed OMB to 

issue guidelines to ‘‘provide policy and 

procedural guidance to Federal agencies 

for ensuring and maximizing the 

quality, objectivity, utility and integrity 

of information’’ disseminated by Federal 

agencies. Public Law No. 106–554, 

§ 515(a). The Information Quality Act 

was developed as a supplement to the 

Paperwork Reduction Act, 44 U.S.C. 

3501 et seq., which requires OMB, 

among other things, to ‘‘develop and 

oversee the implementation of policies, 

principles, standards, and guidelines to 

* * * apply to Federal agency 

dissemination of public information.’’ In 

addition, Executive Order 12866, 58 FR 

51,735 (Oct. 4, 1993), establishes that 

OIRA is ‘‘the repository of expertise 

concerning regulatory issues,’’ and it 

directs OMB to provide guidance to the 

agencies on regulatory planning. E.O. 

12866, § 2(b). The Order also requires 

that ‘‘[e]ach agency shall base its 

decisions on the best reasonably 

obtainable scientific, technical, 

economic, or other information.’’ E.O. 

12866, § 1(b)(7). Finally, OMB has 

authority in certain circumstances to 

manage the agencies under the purview 

of the President’s Constitutional 

authority to supervise the unitary 

Executive Branch. All of these 

authorities support this Bulletin. 

The Requirements of This Bulletin 

This Bulletin addresses peer review of 

scientific information disseminations 

that contain findings or conclusions that 

represent the official position of one or 

more agencies of the Federal 

government. 

Section I: Definitions 

Section I provides definitions that are 

central to this Bulletin. Several terms 

are identical to or based on those used 

in OMB’s government-wide information 

quality guidelines, 67 FR 8452 (Feb. 22, 

2002), and the Paperwork Reduction 

Act, 44 U.S.C. 3501 et seq. 

Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 2667 

The term ‘‘Administrator’’ means the 

Administrator of the Office of 

Information and Regulatory Affairs in 

the Office of Management and Budget 

(OIRA). 

The term ‘‘agency’’ has the same 

meaning as in the Paperwork Reduction 

Act, 44 U.S.C. 3502(1). 

The term ‘‘Information Quality Act’’ 

means Section 515 of Public Law 106– 

554 (Pub. L. No. 106–554, § 515, 114 

Stat. 2763, 2763A–153–154 (2000)). 

The term ‘‘dissemination’’ means 

agency initiated or sponsored 

distribution of information to the 

public. Dissemination does not include 

distribution limited to government 

employees or agency contractors or 

grantees; intra-or inter-agency use or 

sharing of government information; or 

responses to requests for agency records 

under the Freedom of Information Act, 

the Privacy Act, the Federal Advisory 

Committee Act, the Government 

Performance and Results Act, or similar 

laws. This definition also excludes 

distribution limited to correspondence 

with individuals or persons, press 

releases, archival records, public filings, 

subpoenas and adjudicative processes. 

In the context of this Bulletin, the 

definition of ‘‘dissemination’’ modifies 

the definition in OMB’s governmentwide information quality guidelines to 

address the need for peer review prior 

to official dissemination of the 

information product. Accordingly, 

under this Bulletin, ‘‘dissemination’’ 

also excludes information distributed 

for peer review in compliance with this 

Bulletin or shared confidentially with 

scientific colleagues, provided that the 

distributing agency includes an 

appropriate and clear disclaimer on the 

information, as explained more fully 

below. Finally, the Bulletin does not 

directly cover information supplied to 

the government by third parties (e.g., 

studies by private consultants, 

companies and private, non-profit 

organizations, or research institutions 

such as universities). However, if an 

agency plans to disseminate information 

supplied by a third party (e.g., using this 

information as the basis for an agency’s 

factual determination that a particular 

behavior causes a disease), the 

requirements of the Bulletin apply, if 

the dissemination is ‘‘influential’’. 

In cases where a draft report or other 

information is released by an agency 

solely for purposes of peer review, a 

question may arise as to whether the 

draft report constitutes an official 

‘‘dissemination’’ under informationquality guidelines. Section I instructs 

agencies to make this clear by 

presenting the following disclaimer in 

the report: 

This information is distributed solely for the 

purpose of pre-dissemination peer review 

under applicable information quality 

guidelines. It has not been formally 

disseminated by [the agency]. It does not 

represent and should not be construed to 

represent any agency determination or 

policy. 

In cases where the information is 

highly relevant to specific policy or 

regulatory deliberations, this disclaimer 

shall appear on each page of a draft 

report. Agencies also shall discourage 

state, local, international and private 

organizations from using information in 

draft reports that are undergoing peer 

review. Draft influential scientific 

information presented at scientific 

meetings or shared confidentially with 

colleagues for scientific input prior to 

peer review shall include the 

disclaimer: ‘‘The Findings and 

Conclusions in This Report 

(Presentation) Have Not Been Formally 

Disseminated by [The Agency] and 

Should Not Be Construed to Represent 

Any Agency Determination or Policy.’’ 

An information product is not 

covered by the Bulletin unless it 

represents an official view of one or 

more departments or agencies of the 

Federal government. Accordingly, for 

the purposes of this Bulletin, 

‘‘dissemination’’ excludes research 

produced by government-funded 

scientists (e.g., those supported 

extramurally or intramurally by Federal 

agencies or those working in state or 

local governments with Federal support) 

if that information is not represented as 

the views of a department or agency 

(i.e., they are not official government 

disseminations). For influential 

scientific information that does not have 

the imprimatur of the Federal 

government, scientists employed by the 

Federal government are required to 

include in their information product a 

clear disclaimer that ‘‘the findings and 

conclusions in this report are those of 

the author(s) and do not necessarily 

represent the views of the funding 

agency.’’ A similar disclaimer is advised 

for non-government employees who 

publish government-funded research. 

For the purposes of the peer review 

Bulletin, the term ‘‘scientific 

information’’ means factual inputs, data, 

models, analyses, technical information, 

or scientific assessments related to such 

disciplines as the behavioral and social 

sciences, public health and medical 

sciences, life and earth sciences, 

engineering, or physical sciences. This 

includes any communication or 

representation of knowledge such as 

facts or data, in any medium or form, 

including textual, numerical, graphic, 

cartographic, narrative, or audiovisual 

forms. This definition includes 

information that an agency disseminates 

from a Web page, but does not include 

the provision of hyperlinks on a Web 

page to information that others 

disseminate. This definition excludes 

opinions, where the agency’s 

presentation makes clear that an 

individual’s opinion, rather than a 

statement of fact or of the agency’s 

findings and conclusions, is being 

offered. 

The term ‘‘influential scientific 

information’’ means scientific 

information the agency reasonably can 

determine will have or does have a clear 

and substantial impact on important 

public policies or private sector 

decisions. In the term ‘‘influential 

scientific information,’’ the term 

‘‘influential’’ should be interpreted 

consistently with OMB’s governmentwide information quality guidelines and 

the information quality guidelines of the 

agency. Information dissemination can 

have a significant economic impact even 

if it is not part of a rulemaking. For 

instance, the economic viability of a 

technology can be influenced by the 

government’s characterization of its 

attributes. Alternatively, the Federal 

government’s assessment of risk can 

directly or indirectly influence the 

response actions of state and local 

agencies or international bodies. 

One type of scientific information is 

a scientific assessment. For the purposes 

of this Bulletin, the term ‘‘scientific 

assessment’’ means an evaluation of a 

body of scientific or technical 

knowledge, which typically synthesizes 

multiple factual inputs, data, models, 

assumptions, and/or applies best 

professional judgment to bridge 

uncertainties in the available 

information. These assessments include, 

but are not limited to, state-of-science 

reports; technology assessments; weightof-evidence analyses; meta-analyses; 

health, safety, or ecological risk 

assessments; toxicological 

characterizations of substances; 

integrated assessment models; hazard 

determinations; or exposure 

assessments. Such assessments often 

draw upon knowledge from multiple 

disciplines. Typically, the data and 

models used in scientific assessments 

have already been subject to some form 

of peer review (e.g., refereed journal 

peer review or peer review under 

Section II of this Bulletin). 

Section II: Peer Review of Influential 

Scientific Information 

Section II requires each agency to 

subject ‘‘influential’’ scientific 

information to peer review prior to 

dissemination. For dissemination of 

2668 Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 

influential scientific information, 

Section II provides agencies broad 

discretion in determining what type of 

peer review is appropriate and what 

procedures should be employed to 

select appropriate reviewers. Agencies 

are directed to chose a peer review 

mechanism that is adequate, giving due 

consideration to the novelty and 

complexity of the science to be 

reviewed, the relevance of the 

information to decision making, the 

extent of prior peer reviews, and the 

expected benefits and costs of 

additional review. 

The National Academy of Public 

Administration suggests that the 

intensity of peer review should be 

commensurate with the significance of 

the information being disseminated and 

the likely implications for policy 

decisions.10 Furthermore, agencies need 

to consider tradeoffs between depth of 

peer review and timeliness.11 More 

rigorous peer review is necessary for 

information that is based on novel 

methods or presents complex challenges 

for interpretation. Furthermore, the 

need for rigorous peer review is greater 

when the information contains 

precedent-setting methods or models, 

presents conclusions that are likely to 

change prevailing practices, or is likely 

to affect policy decisions that have a 

significant impact. 

This tradeoff can be considered in a 

benefit-cost framework. The costs of 

peer review include both the direct 

costs of the peer review activity and 

those stemming from potential delay in 

government and private actions that can 

result from peer review. The benefits of 

peer review are equally clear: the 

insights offered by peer reviewers may 

lead to policy with more benefits and/ 

or fewer costs. In addition to 

contributing to strong science, peer 

review, if performed fairly and 

rigorously, can build consensus among 

stakeholders and reduce the temptation 

for courts and legislators to secondguess or overturn agency actions.12 

While it will not always be easy for 

agencies to quantify the benefits and 

costs of peer review, agencies are 

10 National Academy of Public Administration, 

Setting Priorities, Getting Results: A New Direction 

for EPA, National Academy Press, Washington, DC, 

1995:23. 

11 Presidential/Congressional Commission on 

Risk Assessment and Risk Management, Risk 

Commission Report, 1997. 

12 Mark R. Powell, Science at EPA: Information in 

the Regulatory Process, Resources for the Future, 

Washington, DC, 1999: 148, 176; Sheila Jasanoff, 

The Fifth Branch: Science Advisors as Policy 

Makers, Harvard University Press, Boston, 1990: 

242. 

encouraged to approach peer review 

from a benefit-cost perspective. 

Regardless of the peer review 

mechanism chosen, agencies should 

strive to ensure that their peer review 

practices are characterized by both 

scientific integrity and process integrity. 

‘‘Scientific integrity,’’ in the context of 

peer review, refers to such issues as 

‘‘expertise and balance of the panel 

members; the identification of the 

scientific issues and clarity of the charge 

to the panel; the quality, focus and 

depth of the discussion of the issues by 

the panel; the rationale and 

supportability of the panel’s findings; 

and the accuracy and clarity of the 

panel report.’’ ‘‘Process integrity’’ 

includes such issues as ‘‘transparency 

and openness, avoidance of real or 

perceived conflicts of interest, a 

workable process for public comment 

and involvement,’’ and adherence to 

defined procedures.13 

When deciding what type of peer 

review mechanism is appropriate for a 

specific information product, agencies 

will need to consider at least the 

following issues: Individual versus 

panel review; timing; scope of the 

review; selection of reviewers; 

disclosure and attribution; public 

participation; disposition of reviewer 

comments; and adequacy of prior peer 

review. 

Individual Versus Panel Review 

Letter reviews by several experts 

generally will be more expeditious than 

convening a panel of experts. Individual 

letter reviews are more appropriate 

when a draft document covers only one 

discipline or when premature disclosure 

of a sensitive report to a public panel 

could cause harm to government or 

private interests. When time and 

resources warrant, panels are preferable, 

as they tend to be more deliberative 

than individual letter reviews and the 

reviewers can learn from each other. 

There are also multi-stage processes in 

which confidential letter reviews are 

conducted prior to release of a draft 

document for public notice and 

comment, followed by a formal panel 

review. These more rigorous and 

expensive processes are particularly 

valuable for highly complex, 

multidisciplinary, and more important 

documents, especially those that are 

novel or precedent-setting. 

Timing of Peer Review 

As a general rule, it is most useful to 

consult with peers early in the process 

13 ILSI Risk Sciences Institute, ‘‘Policies and 

Procedures: Model Peer Review Center of 

Excellence,’’ 2002: 4. Available at http://rsi.ilsi.org/ 

file/Policies&Procedures.pdf. 

of producing information. For example, 

in the context of risk assessments, it is 

valuable to have the choice of input data 

and the specification of the model 

reviewed by peers before the agency 

invests time and resources in 

implementing the model and 

interpreting the results. ‘‘Early’’ peer 

review occurs in time to ‘‘focus 

attention on data inadequacies in time 

for corrections. 

When an information product is a 

critical component of rule-making, it is 

important to obtain peer review before 

the agency announces its regulatory 

options so that any technical corrections 

can be made before the agency becomes 

invested in a specific approach or the 

positions of interest groups have 

hardened. If review occurs too late, it is 

unlikely to contribute to the course of a 

rulemaking. Furthermore, investing in a 

more rigorous peer review early in the 

process ‘‘may provide net benefit by 

reducing the prospect of challenges to a 

regulation that later may trigger time 

consuming and resource-draining 14 litigation.’’ 

Scope of the Review 

The ‘‘charge’’ contains the 

instructions to the peer reviewers 

regarding the objective of the peer 

review and the specific advice sought. 

The importance of the information, 

which shapes the goal of the peer 

review, influences the charge. For 

instance, the goal of the review might be 

to determine the utility of a body of 

literature for drawing certain 

conclusions about the feasibility of a 

technology or the safety of a product. In 

this context, an agency might ask 

reviewers to determine the relevance of 

conclusions drawn in one context for 

other contexts (e.g., different exposure 

conditions or patient populations). 

The charge to the reviewers should be 

determined in advance of the selection 

of the reviewers. In drafting the charge, 

it is important to remember the 

strengths and limitations of peer review. 

Peer review is most powerful when the 

charge is specific and steers the 

reviewers to specific technical questions 

while also directing reviewers to offer a 

broad evaluation of the overall product. 

Uncertainty is inherent in science, 

and in many cases individual studies do 

not produce conclusive evidence. Thus, 

when an agency generates a scientific 

14 Fred Anderson, Mary Ann Chirba Martin, E 

Donald Elliott, Cynthia Farina, Ernest Gellhorn, 

John D. Graham, C. Boyden Gray, Jeffrey Holmstead, 

Ronald M. Levin, Lars Noah, Katherine Rhyne, 

Jonathan Baert Wiener, ‘‘Regulatory Improvement 

Legislation: Risk Assessment, Cost-Benefit Analysis, 

and Judicial Review,’’ Duke Environmental Law and 

Policy Forum, Fall 2000, vol. XI (1): 132. 

Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 2669 

assessment, it is presenting its scientific 

judgment about the accumulated 

evidence rather than scientific fact.15 

Specialists attempt to reach a consensus 

by weighing the accumulated evidence. 

Peer reviewers can make an important 

contribution by distinguishing scientific 

facts from professional judgments. 

Furthermore, where appropriate, 

reviewers should be asked to provide 

advice on the reasonableness of 

judgments made from the scientific 

evidence. However, the charge should 

make clear that the reviewers are not to 

provide advice on the policy (e.g., the 

amount of uncertainty that is acceptable 

or the amount of precaution that should 

be embedded in an analysis). Such 

considerations are the purview of the 

government.16 

The charge should ask that peer 

reviewers ensure that scientific 

uncertainties are clearly identified and 

characterized. Since not all 

uncertainties have an equal effect on the 

conclusions drawn, reviewers should be 

asked to ensure that the potential 

implications of the uncertainties for the 

technical conclusions drawn are clear. 

In addition, peer reviewers might be 

asked to consider value-of-information 

analyses that identify whether more 

research is likely to decrease key 

uncertainties.17 Value-of-information 

analysis was suggested for this purpose 

in the report of the Presidential/ 

Congressional Commission on Risk 

Assessment and Risk Management.18 A 

description of additional research that 

would appreciably influence the 

conclusions of the assessment can help 

an agency assess and target subsequent 

efforts. 

Selection of Reviewers 

Expertise. The most important factor 

in selecting reviewers is expertise: 

ensuring that the selected reviewer has 

the knowledge, experience, and skills 

necessary to perform the review. 

Agencies shall ensure that, in cases 

where the document being reviewed 

spans a variety of scientific disciplines 

or areas of technical expertise, reviewers 

who represent the necessary spectrum 

of knowledge are chosen. For instance, 

expertise in applied mathematics and 

15 Mark R. Powell, Science at EPA: Information in 

the Regulatory Process, Resources for the Future, 

Washington, DC, 1999: 139. 

16 Ibid. 

17 Granger Morgan and Max Henrion, ‘‘The Value 

of Knowing How Little You Know,’’ Uncertainty: A 

Guide to Dealing with Uncertainty in Quantitative 

Risk and Policy Analysis, Cambridge University 

Press, 1990: 307. 

18 Presidential/Congressional Commission on 

Risk Assessment and Risk Management, Risk 

Commission Report, 1997, Volume 1: 39, Volume 2: 

91. 

statistics is essential in the review of 

models, thereby allowing an audit of 

calculations and claims of significance 

and robustness based on the numeric 

data.19 For some reviews, evaluation of 

biological plausibility is as important as 

statistical modeling. Agencies shall 

consider requesting that the public, 

including scientific and professional 

societies, nominate potential reviewers. 

Balance. While expertise is the 

primary consideration, reviewers should 

also be selected to represent a diversity 

of scientific perspectives relevant to the 

subject. On most controversial issues, 

there exists a range of respected 

scientific viewpoints regarding 

interpretation of the available literature. 

Inviting reviewers with competing 

views on the science may lead to a 

sharper, more focused peer review. 

Indeed, as a final layer of review, some 

organizations (e.g., the National 

Academy of Sciences) specifically 

recruit reviewers with strong opinions 

to test the scientific strength and 

balance of their reports. The NAS policy 

on committee composition and 

balance 20 highlights important 

considerations associated with 

perspective, bias, and objectivity. 

Independence. In its narrowest sense, 

independence in a reviewer means that 

the reviewer was not involved in 

producing the draft document to be 

reviewed. However, for peer review of 

some documents, a broader view of 

independence is necessary to assure 

credibility of the process. Reviewers are 

generally not employed by the agency or 

office producing the document. As the 

National Academy of Sciences has 

stated, ‘‘external experts often can be 

more open, frank, and challenging to the 

status quo than internal reviewers, who 

may feel constrained by organizational 

concerns.’’ 21 The Carnegie Commission 

on Science, Technology, and 

Government notes that ‘‘external science 

advisory boards serve a critically 

important function in providing 

regulatory agencies with expert advice 

on a range of issues.’’ 22 However, the 

choice of reviewers requires a case-by19 William W. Lowrance, Modern Science and 

Human Values, Oxford University Press, New York, 

NY 1985: 86. 

20 National Academy of Sciences, ‘‘Policy and 

Procedures on Committee Composition and Balance 

and Conflicts of Interest for Committees Used in the 

Development of Reports,’’ May 2003: Available at: 

http://www.nationalacademies.org/coi/index.html. 

21 National Research Council, Peer Review in 

Environmental Technology Development Programs: 

The Department of Energy’s Office of Science and 

Technology, National Academy Press, Washington, 

DC, 1998: 3. 

22 Carnegie Commission on Science, Technology, 

and Government, Risk and the Environment: 

Improving Regulatory Decision Making, Carnegie 

Commission, New York, 1993: 90. 

case analysis. Reviewers employed by 

other Federal and state agencies may 

possess unique or indispensable 

expertise. 

A related issue is whether 

government-funded scientists in 

universities and consulting firms have 

sufficient independence from the 

federal agencies that support their work 

to be appropriate peer reviewers for 

those agencies.23 This concern can be 

mitigated in situations where the 

scientist initiates the hypothesis to be 

tested or the method to be developed, 

which effectively creates a buffer 

between the scientist and the agency. 

When an agency awards grants through 

a competitive process that includes peer 

review, the agency’s potential to 

influence the scientist’s research is 

limited. As such, when a scientist is 

awarded a government research grant 

through an investigator-initiated, peerreviewed competition, there generally 

should be no question as to that 

scientist’s ability to offer independent 

scientific advice to the agency on other 

projects. This contrasts, for example, to 

a situation in which a scientist has a 

consulting or contractual arrangement 

with the agency or office sponsoring a 

peer review. Likewise, when the agency 

and a researcher work together (e.g., 

through a cooperative agreement) to 

design or implement a study, there is 

less independence from the agency. 

Furthermore, if a scientist has 

repeatedly served as a reviewer for the 

same agency, some may question 

whether that scientist is sufficiently 

independent from the agency to be 

employed as a peer reviewer on agencysponsored projects. 

As the foregoing suggests, 

independence poses a complex set of 

questions that must be considered by 

agencies when peer reviewers are 

selected. In general, agencies shall make 

an effort to rotate peer review 

responsibilities across the available pool 

of qualified reviewers, recognizing that 

in some cases repeated service by the 

same reviewer is needed because of 

essential expertise. 

Some agencies have built entire 

organizations to provide independent 

scientific advice while other agencies 

tend to employ ad hoc scientific panels 

on specific issues. Respect for the 

independence of reviewers may be 

enhanced if an agency collects names of 

potential reviewers (based on 

considerations of expertise and 

reputation for objectivity) from the 

23 Lars Noah, ‘‘Scientific ‘Republicanism’: Expert 

Peer Review and the Quest for Regulatory 

Deliberation, Emory Law Journal, Atlanta, Fall 

2000:1066. 

2670 Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 

public, including scientific or 

professional societies. The Department 

of Energy’s use of the American Society 

of Mechanical Engineers to identify 

potential peer reviewers from a variety 

of different scientific societies provides 

an example of how professional 

societies can assist in the development 

of an independent peer review panel.24 

Conflict of Interest. The National 

Academy of Sciences defines ‘‘conflict 

of interest’’ as any financial or other 

interest that conflicts with the service of 

an individual on the review panel 

because it could impair the individual’s 

objectivity or could create an unfair 

competitive advantage for a person or 

organization.25 This standard provides a 

useful benchmark for agencies to 

consider in selecting peer reviewers. 

Agencies shall make a special effort to 

examine prospective reviewers’ 

potential financial conflicts, including 

significant investments, consulting 

arrangements, employer affiliations and 

grants/contracts. Financial ties of 

potential reviewers to regulated entities 

(e.g., businesses), other stakeholders, 

and regulatory agencies shall be 

scrutinized when the information being 

reviewed is likely to be relevant to 

regulatory policy. The inquiry into 

potential conflicts goes beyond financial 

investments and business relationships 

and includes work as an expert witness, 

consulting arrangements, honoraria and 

sources of grants and contracts. To 

evaluate any real or perceived conflicts 

of interest with potential reviewers and 

questions regarding the independence of 

reviewers, agencies are referred to 

federal ethics requirements, applicable 

standards issued by the Office of 

Government Ethics, and the prevailing 

practices of the National Academy of 

Sciences. Specifically, peer reviewers 

who are Federal employees (including 

special government employees) are 

subject to Federal requirements 

governing conflicts of interest. See, e.g., 

18 U.S.C. 208; 5 CFR part 2635 (2004). 

With respect to reviewers who are not 

Federal employees, agencies shall adopt 

or adapt the NAS policy for committee 

selection with respect to evaluating 

conflicts of interest.26 Both the NAS and 

the Federal government recognize that 

under certain circumstances some 

24 American Society for Mechanical Engineers, 

Assessment of Technologies Supported by the 

Office of Science and Technology, Department of 

Energy: Results of the Peer Review for Fiscal Year 

2002, ASME Technical Publishing, Danvers, MA, 

2003. 

25 National Academy of Sciences, ‘‘Policy and 

Procedures on Committee Composition and Balance 

and Conflicts of Interest for Committees Used in the 

Development of Reports,’’ May 2003: Available at: 

http://www.nationalacademies.org/coi/index.html. 

26 Ibid. 

conflict may be unavoidable in order to 

obtain the necessary expertise. See, e.g., 

18 U.S.C. 208(b)(3); 5 U.S.C. App. 15 

(governing NAS committees). To 

improve the transparency of the process, 

when an agency determines that it is 

necessary to use a reviewer with a real 

or perceived conflict of interest, the 

agency should consider publicly 

disclosing those conflicts. In such 

situations, the agency shall inform 

potential reviewers of such disclosure at 

the time they are recruited. 

Disclosure and Attribution: Anonymous 

Versus Identified 

Peer reviewers must have a clear 

understanding of how their comments 

will be conveyed to the authors of the 

document and to the public. When peer 

review of government reports is 

considered, the case for transparency is 

stronger, particularly when the report 

addresses an issue with significant 

ramifications for the public and private 

sectors. The public may not have 

confidence in the peer review process 

when the names and affiliations of the 

peer reviewers are unknown. Without 

access to the comments of reviewers, the 

public is incapable of determining 

whether the government has seriously 

considered the comments of reviewers 

and made appropriate revisions. 

Disclosure of the slate of reviewers and 

the substance of their comments can 

strengthen public confidence in the peer 

review process. It is common at many 

journals and research funding agencies 

to disclose annually the slate of 

reviewers. Moreover, the National 

Academy of Sciences now discloses the 

names of its peer reviewers, without 

disclosing the substance of their 

comments. The science advisory 

committees to regulatory agencies 

typically disclose at least a summary of 

the comments of reviewers as well as 

their names and affiliations. 

For agency-sponsored peer review 

conducted under Sections II and III, this 

Bulletin strikes a compromise by 

requiring disclosure of the identity of 

the reviewers, but not public attribution 

of specific comments to specific 

reviewers. The agency has considerable 

discretion in the implementation of this 

compromise (e.g., summarizing the 

views of reviewers as a group or 

disclosing individual reviewer 

comments without attribution). 

Whatever approach is employed, the 

agency must inform reviewers in 

advance of how it intends to address 

this issue. Information about a reviewer 

retrieved from a record filed by the 

reviewer’s name or other identifier may 

be disclosed only as permitted by the 

conditions of disclosure enumerated in 

the Privacy Act, 5 U.S.C. 552a as 

amended, and as interpreted in OMB 

implementing guidance, 40 FR 28,948 

(July 9, 1975). 

Public Participation 

Public comments can be important in 

shaping expert deliberations. Agencies 

may decide that peer review should 

precede an opportunity for public 

comment to ensure that the public 

receives the most scientifically strong 

product (rather than one that may 

change substantially as a result of peer 

reviewer suggestions). However, there 

are situations in which public 

participation in peer review is an 

important aspect of obtaining a highquality product through a credible 

process. Agencies, however, should 

avoid open-ended comment periods, 

which may delay completion of peer 

reviews and complicate the completion 

of the final work product. 

Public participation can take a variety 

of forms, including opportunities to 

provide oral comments before a peer 

review panel or requests to provide 

written comments to the peer reviewers. 

Another option is for agencies to 

publish a ‘‘request for comment’’ or 

other notice in which they solicit public 

comment before a panel of peer 

reviewers performs its work. 

Disposition of Reviewer Comments 

A peer review is considered 

completed once the agency considers 

and addresses the reviewers’ comments. 

All reviewer comments should be given 

consideration and be incorporated 

where relevant and valid. For instance, 

in the context of risk assessments, the 

National Academy of Sciences 

recommends that peer review include a 

written evaluation made available for 

public inspection.27 In cases where 

there is a public panel, the agency 

should plan publication of the peer 

review report(s) and the agency’s 

response to peer reviewer comments. 

In addition, the credibility of the final 

scientific report is likely to be enhanced 

if the public understands how the 

agency addressed the specific concerns 

raised by the peer reviewers. 

Accordingly, agencies should consider 

preparing a written response to the peer 

review report explaining: The agency’s 

agreement or disagreement, the actions 

the agency has undertaken or will 

undertake in response to the report, and 

(if applicable) the reasons the agency 

believes those actions satisfy any key 

27 National Research Council, Risk Assessment in 

the Federal Government: Managing the Process, 

National Academy Press, Washington, DC, 1983. 

Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 2671 

concerns or recommendations in the 

report. 

Adequacy of Prior Peer Review 

In light of the broad range of 

information covered by Section II, 

agencies are directed to choose a peer 

review mechanism that is adequate, 

giving due consideration to the novelty 

and complexity of the science to be 

reviewed, the relevance of the 

information to decision making, the 

extent of prior peer reviews, and the 

expected benefits and costs of 

additional review. 

Publication in a refereed scientific 

journal may mean that adequate peer 

review has been performed. However, 

the intensity of peer review is highly 

variable across journals. There will be 

cases in which an agency determines 

that a more rigorous or transparent 

review process is necessary. For 

instance, an agency may determine a 

particular journal review process did 

not address questions (e.g., the extent of 

uncertainty inherent in a finding) that 

the agency determines should be 

addressed before disseminating that 

information. As such, prior peer review 

and publication is not by itself sufficient 

grounds for determining that no further 

review is necessary. 

Section III: Peer Review of Highly 

Influential Scientific Assessments 

Whereas Section II leaves most of the 

considerations regarding the form of the 

peer review to the agency’s discretion, 

Section III requires a more rigorous form 

of peer review for highly influential 

scientific assessments. The 

requirements of Section II of this 

Bulletin apply to Section III, but Section 

III has some additional requirements, 

which are discussed below. In planning 

a peer review under Section III, agencies 

typically will have to devote greater 

resources and attention to the issues 

discussed in Section II, i.e., individual 

versus panel review; timing; scope of 

the review; selection of reviewers; 

disclosure and attribution; public 

participation; and disposition of 

reviewer comments. 

A scientific assessment is considered 

‘‘highly influential’’ if the agency or the 

OIRA Administrator determines that the 

dissemination could have a potential 

impact of more than $500 million in any 

one year on either the public or private 

sector or that the dissemination is novel, 

controversial, or precedent-setting, or 

has significant interagency interest. One 

of the ways information can exert 

economic impact is through the costs or 

benefits of a regulation based on the 

disseminated information. The 

qualitative aspect of this definition may 

be most useful in cases where it is 

difficult for an agency to predict the 

potential economic effect of 

dissemination. In the context of this 

Bulletin, it may be either the approach 

used in the assessment or the 

interpretation of the information itself 

that is novel or precedent-setting. Peer 

review can be valuable in establishing 

the bounds of the scientific debate when 

methods or interpretations are a source 

of controversy among interested parties. 

If information is covered by Section III, 

an agency is required to adhere to the 

peer review procedures specified in 

Section III. 

Section III(2) clarifies that the 

principal findings, conclusions and 

recommendations in official reports of 

the National Academy of Sciences that 

fall under this Section are generally 

presumed not to require additional peer 

review. All other highly influential 

scientific assessments require a review 

that meets the requirements of Section 

III of this Bulletin. 

With regard to the selection of 

reviewers, Section III(3)(a) emphasizes 

consideration of expertise and balance. 

As discussed in Section II, expertise 

refers to the required knowledge, 

experience and skills required to 

perform the review whereas balance 

refers to the need for diversity in 

scientific perspective and disciplines. 

We emphasize that the term ‘‘balance’’ 

here refers not to balancing of 

stakeholder or political interests but 

rather to a broad and diverse 

representation of respected perspectives 

and intellectual traditions within the 

scientific community, as discussed in 

the NAS policy on committee 

composition and balance.28 

Section III(3)(b) instructs agencies to 

consider barring participation by 

scientists with a conflict of interest. The 

conflict of interest standards for 

Sections II and III of the Bulletin are 

identical. As discussed under Section II, 

those peer reviewers who are Federal 

employees, including Special 

Government Employees, are subject to 

applicable statutory and regulatory 

standards for Federal employees. For 

non-government employees, agencies 

shall adopt or adapt the NAS policy for 

committee member selection with 

respect to evaluating conflicts of 

interest. 

Section III(3)(c) instructs agencies to 

ensure that reviewers are independent 

of the agency sponsoring the review. 

Scientists employed by the sponsoring 

28 National Academy of Sciences, ‘‘Policy and 

Procedures on Committee Composition and Balance 

and Conflicts of Interest for Committees Used in the 

Development of Reports,’’ May 2003: Available at: 

http://www.nationalacademies.org/coi/index.html. 

agency are not permitted to serve as 

reviewers for highly influential 

scientific assessments. This does not 

preclude Special Government 

Employees, such as academics 

appointed to advisory committees, from 

serving as peer reviewers. The only 

exception to this ban would be the rare 

situation in which a scientist from a 

different agency of a Cabinet-level 

department than the agency that is 

disseminating the scientific assessment 

has expertise, experience and skills that 

are essential but cannot be obtained 

elsewhere. In evaluating the need for 

this exception, agencies shall use the 

NAS criteria for assessing the 

appropriateness of using employees of 

sponsors (e.g., the government scientist 

must not have had any part in the 

development or prior review of the 

scientific information and must not hold 

a position of managerial or policy 

responsibility). 

We also considered whether a 

reviewer can be independent of the 

agency if that reviewer receives a 

substantial amount of research funding 

from the agency sponsoring the review. 

Research grants that were awarded to 

the scientist based on investigatorinitiated, competitive, peer-reviewed 

proposals do not generally raise issues 

of independence. However, significant 

consulting and contractual relationships 

with the agency may raise issues of 

independence or conflict, depending 

upon the situation. 

Section III(3)(d) addresses concerns 

regarding repeated use of the same 

reviewer in multiple assessments. Such 

repeated use should be avoided unless 

a particular reviewer’s expertise is 

essential. Agencies should rotate 

membership across the available pool of 

qualified reviewers. Similarly, when 

using standing panels of scientific 

advisors, it is suggested that the agency 

rotate membership among qualified 

scientists in order to obtain fresh 

perspectives and reinforce the reality 

and perception of independence from 

the agency. 

Section III(4) requires agencies to 

provide reviewers with sufficient 

background information, including 

access to key studies, data and models, 

to perform their role as peer reviewers. 

In this respect, the peer review 

envisioned in Section III is more 

rigorous than some forms of journal peer 

review, where the reviewer is often not 

provided access to underlying data or 

models. Reviewers shall be informed of 

applicable access, objectivity, 

reproducibility and other quality 

standards under Federal information 

quality laws. 

2672 Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 

Section III(5) addresses opportunity 

for public participation in peer review, 

and provides that the agency shall, 

wherever possible, provide for public 

participation. In some cases, an 

assessment may be so sensitive that it is 

critical that the agency’s assessment 

achieve a high level of quality before it 

is publicized. In those situations, a 

rigorous yet confidential peer review 

process may be appropriate, prior to 

public release of the assessment. If an 

agency decides to make a draft 

assessment publicly available at the 

onset of a peer review process, the 

agency shall, whenever possible, 

provide a vehicle for the public to 

provide written comments, make an oral 

presentation before the peer reviewers, 

or both. When written public comments 

are received, the agency shall ensure 

that peer reviewers receive copies of 

comments that address significant 

scientific issues with ample time to 

consider them in their review. To avoid 

undue delay of agency activities, the 

agency shall specify time limits for 

public participation throughout the peer 

review process. 

Section III(6) requires that agencies 

instruct reviewers to prepare a peer 

review report that describes the nature 

and scope of their review and their 

findings and conclusions. The report 

shall disclose the name of each peer 

reviewer and a brief description of his 

or her organizational affiliation, 

credentials and relevant experiences. 

The peer review report should either 

summarize the views of the group as a 

whole (including any dissenting views) 

or include a verbatim copy of the 

comments of the individual reviewers 

(with or without attribution of specific 

views to specific names). The agency 

shall also prepare a written response to 

the peer review report, indicating 

whether the agency agrees with the 

reviewers and what actions the agency 

has taken or plans to take to address the 

points made by reviewers. The agency is 

required to disseminate the peer review 

report and the agency’s response to the 

report on the agency’s Web site, 

including all the materials related to the 

peer review such as the charge 

statement, peer review report, and 

agency response to the review. If the 

scientific information is used to support 

a final rule then, where practicable, the 

peer review report shall be made 

available to the public with enough time 

for the public to consider the 

implications of the peer review report 

for the rule being considered. 

Section III(7) authorizes but does not 

require an agency to commission an 

entity independent of the agency to 

select peer reviewers and/or manage the 

peer review process in accordance with 

this Bulletin. The entity may be a 

scientific or professional society, a firm 

specializing in peer review, or a nonprofit organization with experience in 

peer review. 

Section IV: Alternative Procedures 

Peer review as described in this 

Bulletin is only one of many procedures 

that agencies can employ to ensure an 

appropriate degree of pre-dissemination 

quality of influential scientific 

information. For example, Congress has 

assigned the NAS a special role in 

advising the Federal government on 

scientific and technical issues. The 

procedures of the NAS are generally 

quite rigorous, and thus agencies should 

presume that major findings, 

conclusions, and recommendations of 

NAS reports meet the performance 

standards of this Bulletin. 

As an alternative to complying with 

Sections II and III of this Bulletin, an 

agency may instead (1) rely on scientific 

information produced by the National 

Academy of Sciences, (2) commission 

the National Academy of Sciences to 

peer review an agency draft scientific 

information product, or (3) employ an 

alternative procedure or set of 

procedures, specifically approved by the 

OIRA Administrator in consultation 

with the Office of Science and 

Technology Policy (OSTP), that ensures 

that the scientific information product 

meets applicable information-quality 

standards. 

An example of an alternative 

procedure is to commission a respected 

third party other than the NAS (e.g., the 

Health Effects Institute or the National 

Commission on Radiation Protection 

and Measurement) to conduct an 

assessment or series of related 

assessments. Another example of an 

alternative set of procedures is the 

three-part process used by the National 

Institutes of Health (NIH) to generate 

scientific guidance. Under that process, 

a scientific proposal or white paper is 

generated by a working group composed 

of external, independent scientific 

experts; that paper is then forwarded to 

a separate external scientific council, 

which then makes recommendations to 

the agency. The agency, in turn, decides 

whether to adopt and/or modify the 

proposal. For large science agencies that 

have diverse research portfolios and do 

not have significant regulatory 

responsibilities, such as NIH, an 

acceptable alternative would be to allow 

scientists from one part of the agency 

(for example, an NIH institute) to 

participate in the review of documents 

prepared by another part of the agency, 

as long as the head of the agency 

confirms in writing that each of the 

reviewers meets the NAS criteria 

relating to the appropriateness of using 

employees of sponsors (e.g., the 

government scientist must not have had 

any part in the development or prior 

review of the scientific information and 

must not hold a position of managerial 

or policy responsibility). The purpose of 

Section IV is to encourage these types of 

innovation in the methods used to 

ensure pre-dissemination quality 

control of influential scientific 

information. 

The mere existence of a public 

comment process (e.g., notice-andcomment procedures under the 

Administrative Procedure Act) does not 

constitute adequate peer review or an 

‘‘alternative process,’’ because it does 

not assure that qualified, impartial 

specialists in relevant fields have 

performed a critical evaluation of the 

agency’s draft product.29 

Section V: Peer Review Planning 

Section V requires agencies to begin a 

systematic process of peer review 

planning for influential scientific 

information (including highly 

influential scientific assessments) that 

the agency plans to disseminate in the 

foreseeable future. A key feature of this 

planning process is a Web-accessible 

listing of forthcoming influential 

scientific disseminations (i.e., an 

agenda) that is regularly updated by the 

agency. By making these plans publicly 

available, agencies will be able to gauge 

the extent of public interest in the peer 

review process for influential scientific 

information, including highly 

influential scientific assessments. These 

Web-accessible agendas can also be 

used by the public to monitor agency 

compliance with this Bulletin. 

Each entry on the agenda shall 

include a preliminary title of the 

planned report, a short paragraph 

describing the subject and purpose of 

the planned report, and an agency 

contact person. The agency shall 

provide its prediction regarding whether 

the dissemination will be ‘‘influential 

scientific information’’ or a ‘‘highly 

influential scientific assessment,’’ as the 

designation can influence the type of 

peer review to be undertaken. The 

agency shall discuss the timing of the 

peer review, as well as the use of any 

deferrals. Agencies shall include entries 

in the agenda for influential scientific 

information, including highly 

influential scientific assessments, for 

which the Bulletin’s requirements have 

29 William W. Lowrance, Modern Science and 

Human Values, Oxford University Press, New York, 

NY 1985: 86. 

Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 2673 

been deferred or waived. If the agency, 

in consultation with the OIRA 

Administrator, has determined that it is 

appropriate to use a Section IV 

‘‘alternative procedure’’ for a specific 

dissemination, a description of that 

alternative procedure shall be included 

in the agenda. 

Furthermore, for each entry on the 

agenda, the agency shall describe the 

peer review plan. Each peer review plan 

shall include: (i) A paragraph including 

the title, subject and purpose of the 

planned report, as well as an agency 

contact to whom inquiries may be 

directed to learn the specifics of the 

plan; (ii) whether the dissemination is 

likely to be influential scientific 

information or a highly influential 

scientific assessment; (iii) the timing of 

the review (including deferrals); (iv) 

whether the review will be conducted 

through a panel or individual letters (or 

whether an alternative procedure will 

be exercised); (v) whether there will be 

opportunities for the public to comment 

on the work product to be peer 

reviewed, and if so, how and when 

these opportunities will be provided; 

(vi) whether the agency will provide 

significant and relevant public 

comments to the peer reviewers before 

they conduct their review; (vii) the 

anticipated number of reviewers (3 or 

fewer; 4–10; or more than 10); (viii) a 

succinct description of the primary 

disciplines or expertise needed in the 

review; (ix) whether reviewers will be 

selected by the agency or by a 

designated outside organization; and (x) 

whether the public, including scientific 

or professional societies, will be asked 

to nominate potential peer reviewers. 

The agency shall provide a link from the 

agenda to each document made public 

pursuant to this Bulletin. Agencies shall 

link their peer review agendas to the 

U.S. Government’s official Web portal: 

firstgov at http://www.FirstGov.gov. 

Agencies should update their peer 

review agendas at least every six 

months. However, in some cases— 

particularly for highly influential 

scientific assessments and other 

particularly important information— 

more frequent updates of existing 

entries on the agenda, or the addition of 

new entries to the agenda, may be 

warranted. When new entries are added 

to the agenda of forthcoming reports and 

other information, the public should be 

provided with sufficient time to 

comment on the agency’s peer review 

plan for that report or product. Agencies 

shall consider public comments on the 

peer review plan. Agencies are 

encouraged to offer a listserve or similar 

mechanism for members of the public 

who would like to be notified by email 

each time an agency’s peer review 

agenda has been updated. 

The peer review planning 

requirements of this Bulletin are 

designed to be implemented in phases. 

Specifically, the planning requirements 

of the Bulletin will go into effect for 

documents subject to Section III of the 

Bulletin (highly influential scientific 

assessments) six months after 

publication. However, the planning 

requirements for documents subject to 

Section II of the Bulletin do not go into 

effect until one year after publication. It 

is expected that agency experience with 

the planning requirements of the 

Bulletin for the smaller scope of 

documents encompassed in Section III 

will be used to inform implementation 

of these planning requirements for the 

larger scope of documents covered 

under Section II. 

Section VI: Annual Report 

Each agency shall prepare an annual 

report that summarizes key decisions 

made pursuant to this Bulletin. In 

particular, each agency should provide 

to OIRA the following: (1) The number 

of peer reviews conducted subject to the 

Bulletin (i.e., for influential scientific 

information and highly influential 

scientific assessments); (2) the number 

of times alternative procedures were 

invoked; (3) the number of times 

waivers or deferrals were invoked (and 

in the case of deferrals, the length of 

time elapsed between the deferral and 

the peer review); (4) any decision to 

appoint a reviewer pursuant to any 

exception to the applicable 

independence or conflict of interest 

standards of the Bulletin, including 

determinations by the Secretary or 

Deputy Secretary pursuant to Section 

III(3)(c); (5) the number of peer review 

panels that were conducted in public 

and the number that allowed public 

comment; (6) the number of public 

comments provided on the agency’s 

peer review plans; and (7) the number 

of peer reviewers that the agency used 

that were recommended by professional 

societies. 

Section VII: Certification in the 

Administrative Record 

If an agency relies on influential 

scientific information or a highly 

influential scientific assessment subject 

to the requirements of this Bulletin in 

support of a regulatory action, the 

agency shall include in the 

administrative record for that action a 

certification that explains how the 

agency has complied with the 

requirements of this Bulletin and the 

Information Quality Act. Relevant 

materials are to be placed in the 

administrative record. 

Section VIII: Safeguards, Deferrals, and 

Waivers 

Section VIII recognizes that 

individuals serving as peer reviewers 

have a privacy interest in information 

about themselves that the government 

maintains and retrieves by name or 

identifier from a system of records. To 

the extent information about a reviewer 

(name, credential, affiliation) will be 

disclosed along with his/her comments 

or analysis, the agency must comply 

with the requirements of the Privacy 

Act, 5 U.S.C. 552a, as amended, and 

OMB Circular A–130, Appendix I, 61 FR 

6428 (February 20, 1996) to establish 

appropriate routine uses in a published 

System of Records Notice. Furthermore, 

the peer review must be conducted in a 

manner that respects confidential 

business information as well as 

intellectual property. 

Section VIII also allows for a deferral 

or waiver of the requirements of the 

Bulletin where necessary. Specifically, 

the agency head may waive or defer 

some or all of the peer review 

requirements of Sections II or III of this 

Bulletin if there is a compelling 

rationale for waiver or deferral. Waivers 

will seldom be warranted under this 

provision because the Bulletin already 

provides significant safety valves, such 

as: The exemptions provided in Section 

IX, including the exemption for timesensitive health and safety information; 

the authorization for alternative 

procedures in Section IV; and the 

overall flexibility provided for peer 

reviews of influential scientific 

information under Section II. 

Nonetheless, we have included this 

waiver and deferral provision to ensure 

needed flexibility in unusual and 

compelling situations not otherwise 

covered by the exemptions to the 

Bulletin, such as situations where 

unavoidable legal deadlines prevent full 

compliance with the Bulletin before 

information is disseminated. Deadlines 

found in consent decrees agreed to by 

agencies after the Bulletin is issued will 

not ordinarily warrant waiver of the 

Bulletin’s requirements because those 

deadlines should be negotiated to 

permit time for all required procedures, 

including peer review. In addition, 

when an agency is unavoidably up 

against a deadline, deferral of some or 

all requirements of the Bulletin (as 

opposed to outright waiver of all of 

them) is the most appropriate 

accommodation between the need to 

satisfy immovable deadlines and the 

need to undertake proper peer review. If 

the agency head defers any of the peer 

2674 Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 

review requirements prior to 

dissemination, peer review should be 

conducted as soon as practicable 

thereafter. 

Section IX: Exemptions 

There are a variety of situations where 

agencies need not conduct peer review 

under this Bulletin. These include, for 

example, disseminations of sensitive 

information related to certain national 

security, foreign affairs, or negotiations 

involving international treaties and 

trade where compliance with this 

Bulletin would interfere with the need 

for secrecy or promptness. 

This Bulletin does not cover official 

disseminations that arise in 

adjudications and permit proceedings, 

unless the agency determines that peer 

review is practical and appropriate and 

that the influential dissemination is 

scientifically or technically novel (i.e., a 

major change in accepted practice) or 

likely to have precedent-setting 

influence on future adjudications or 

permit proceedings. This exclusion is 

intended to cover, among other things, 

licensing, approval and registration 

processes for specific product 

development activities as well as sitespecific activities. The determination as 

to whether peer review is practical and 

appropriate is left to the discretion of 

the agency. While this Bulletin is not 

broadly applicable to adjudications, 

agencies are encouraged to hold peer 

reviews of scientific assessments 

supporting adjudications to the same 

technical standards as peer reviews 

covered by the Bulletin, including 

transparency and disclosure of the data 

and models underlying the assessments. 

Protections apply to confidential 

business information. 

The Bulletin does not cover timesensitive health and safety 

disseminations, for example, a 

dissemination based primarily on data 

from a recent clinical trial that was 

adequately peer reviewed before the 

trial began. For this purpose, ‘‘health’’ 

includes public health, or plant or 

animal infectious diseases. 

This Bulletin covers original data and 

formal analytic models used by agencies 

in Regulatory Impact Analyses (RIAs). 

However, the RIA documents 

themselves are already reviewed 

through an interagency review process 

under E.O. 12866 that involves 

application of the principles and 

methods defined in OMB Circular A–4. 

In that respect, RIAs are excluded from 

coverage by this Bulletin, although 

agencies are encouraged to have RIAs 

reviewed by peers within the 

government for adequacy and 

completeness. 

The Bulletin does not cover 

accounting, budget, actuarial, and 

financial information including that 

which is generated or used by agencies 

that focus on interest rates, banking, 

currency, securities, commodities, 

futures, or taxes. 

Routine statistical information 

released by Federal statistical agencies 

(e.g., periodic demographic and 

economic statistics) and analyses of 

these data to compute standard 

indicators and trends (e.g., 

unemployment and poverty rates) is 

excluded from this Bulletin. 

The Bulletin does not cover 

information disseminated in connection 

with routine rules that materially alter 

entitlements, grants, user fees, or loan 

programs, or the rights and obligations 

of recipients thereof. 

If information is disseminated 

pursuant to an exemption to this 

Bulletin, subsequent disseminations are 

not automatically exempted. For 

example, if influential scientific 

information is first disseminated in the 

course of an exempt agency 

adjudication, but is later disseminated 

in the context of a non-exempt 

rulemaking, the subsequent 

dissemination will be subject to the 

requirements of this Bulletin even 

though the first dissemination was not. 

Section X: OIRA and OSTP 

Responsibilities 

OIRA, in consultation with OSTP, is 

responsible for overseeing agency 

implementation of this Bulletin. In 

order to foster learning about peer 

review practices across agencies, OIRA 

and OSTP shall form an interagency 

workgroup on peer review that meets 

regularly, discusses progress and 

challenges, and recommends 

improvements to peer review practices. 

Section XI: Effective Date and Existing 

Law 

The requirements of this Bulletin, 

with the exception of Section V, apply 

to information disseminated on or after 

six months after publication of this 

Bulletin. However, the Bulletin does not 

apply to information that is already 

being addressed by an agency-initiated 

peer review process (e.g., a draft is 

already being reviewed by a formal 

scientific advisory committee 

established by the agency). An existing 

peer review mechanism mandated by 

law should be implemented by the 

agency in a manner as consistent as 

possible with the practices and 

procedures outlined in this Bulletin. 

The requirements of Section V apply to 

‘‘highly influential scientific 

assessments,’’ as designated in Section 

III of the Bulletin, within six months of 

publication of the final Bulletin. The 

requirements in Section V apply to 

documents subject to Section II of the 

Bulletin one year after publication of the 

final Bulletin. 

Section XII: Judicial Review 

This Bulletin is intended to improve 

the internal management of the 

Executive Branch and is not intended 

to, and does not, create any right or 

benefit, substantive or procedural, 

enforceable at law or in equity, against 

the United States, its agencies or other 

entities, its officers or employees, or any 

other person. 

Bulletin for Peer Review 

I. Definitions 

For purposes of this Bulletin— 

1. The term ‘‘Administrator’’ means 

the Administrator of the Office of 

Information and Regulatory Affairs in 

the Office of Management and Budget 

(OIRA); 

2. The term ‘‘agency’’ has the same 

meaning as in the Paperwork Reduction 

Act, 44 U.S.C. 3502(1); 

3. The term ‘‘dissemination’’ means 

agency initiated or sponsored 

distribution of information to the public 

(see 5 CFR 1320.3(d) (definition of 

‘‘Conduct or Sponsor’’)). Dissemination 

does not include distribution limited to 

government employees or agency 

contractors or grantees; intra- or interagency use or sharing of government 

information; or responses to requests for 

agency records under the Freedom of 

Information Act, the Privacy Act, the 

Federal Advisory Committee Act, the 

Government Performance and Results 

Act or similar law. This definition also 

excludes distribution limited to 

correspondence with individuals or 

persons, press releases, archival records, 

public filings, subpoenas and 

adjudicative processes. The term 

‘‘dissemination’’ also excludes 

information distributed for peer review 

in compliance with this Bulletin, 

provided that the distributing agency 

includes a clear disclaimer on the 

information as follows: ‘‘This 

information is distributed solely for the 

purpose of pre-dissemination peer 

review under applicable information 

quality guidelines. It has not been 

formally disseminated by [the agency]. 

It does not represent and should not be 

construed to represent any agency 

determination or policy.’’ For the 

purposes of this Bulletin, 

‘‘dissemination’’ excludes research 

produced by government-funded 

scientists (e.g., those supported 

extramurally or intramurally by Federal 

Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 2675 

agencies or those working in state or 

local governments with Federal support) 

if that information does not represent 

the views of an agency. To qualify for 

this exemption, the information should 

display a clear disclaimer that ‘‘the 

findings and conclusions in this report 

are those of the author(s) and do not 

necessarily represent the views of the 

funding agency’’; 

4. The term ‘‘Information Quality 

Act’’ means Section 515 of Public Law 

106–554 (Pub. L. No. 106–554, § 515, 

114 Stat. 2763, 2763A–153–154 (2000)); 

5. The term ‘‘scientific information’’ 

means factual inputs, data, models, 

analyses, technical information, or 

scientific assessments based on the 

behavioral and social sciences, public 

health and medical sciences, life and 

earth sciences, engineering, or physical 

sciences. This includes any 

communication or representation of 

knowledge such as facts or data, in any 

medium or form, including textual, 

numerical, graphic, cartographic, 

narrative, or audiovisual forms. This 

definition includes information that an 

agency disseminates from a Web page, 

but does not include the provision of 

hyperlinks to information that others 

disseminate. This definition does not 

include opinions, where the agency’s 

presentation makes clear that what is 

being offered is someone’s opinion 

rather than fact or the agency’s views; 

6. The term ‘‘influential scientific 

information’’ means scientific 

information the agency reasonably can 

determine will have or does have a clear 

and substantial impact on important 

public policies or private sector 

decisions; and 

7. The term ‘‘scientific assessment’’ 

means an evaluation of a body of 

scientific or technical knowledge, which 

typically synthesizes multiple factual 

inputs, data, models, assumptions, and/ 

or applies best professional judgment to 

bridge uncertainties in the available 

information. These assessments include, 

but are not limited to, state-of-science 

reports; technology assessments; weightof-evidence analyses; meta-analyses; 

health, safety, or ecological risk 

assessments; toxicological 

characterizations of substances; 

integrated assessment models; hazard 

determinations; or exposure 

assessments. 

II. Peer Review of Influential Scientific 

Information 

1. In General: To the extent permitted 

by law, each agency shall conduct a 

peer review on all influential scientific 

information that the agency intends to 

disseminate. Peer reviewers shall be 

charged with reviewing scientific and 

technical matters, leaving policy 

determinations for the agency. 

Reviewers shall be informed of 

applicable access, objectivity, 

reproducibility and other quality 

standards under the Federal laws 

governing information access and 

quality. 

2. Adequacy of Prior Peer Review: For 

information subject to this section of the 

Bulletin, agencies need not have further 

peer review conducted on information 

that has already been subjected to 

adequate peer review. In determining 

whether prior peer review is adequate, 

agencies shall give due consideration to 

the novelty and complexity of the 

science to be reviewed, the importance 

of the information to decision making, 

the extent of prior peer reviews, and the 

expected benefits and costs of 

additional review. Principal findings, 

conclusions and recommendations in 

official reports of the National Academy 

of Sciences are generally presumed to 

have been adequately peer reviewed. 

3. Selection of Reviewers: a. Expertise 

and Balance: Peer reviewers shall be 

selected based on expertise, experience 

and skills, including specialists from 

multiple disciplines, as necessary. The 

group of reviewers shall be sufficiently 

broad and diverse to fairly represent the 

relevant scientific and technical 

perspectives and fields of knowledge. 

Agencies shall consider requesting that 

the public, including scientific and 

professional societies, nominate 

potential reviewers. 

b. Conflicts: The agency—or the entity 

selecting the peer reviewers—shall (i) 

ensure that those reviewers serving as 

federal employees (including special 

government employees) comply with 

applicable Federal ethics requirements; 

(ii) in selecting peer reviewers who are 

not government employees, adopt or 

adapt the National Academy of Sciences 

policy for committee selection with 

respect to evaluating the potential for 

conflicts (e.g., those arising from 

investments; agency, employer, and 

business affiliations; grants, contracts 

and consulting income). For scientific 

information relevant to specific 

regulations, the agency shall examine a 

reviewer’s financial ties to regulated 

entities (e.g., businesses), other 

stakeholders, and the agency. 

c. Independence: Peer reviewers shall 

not have participated in development of 

the work product. Agencies are 

encouraged to rotate membership on 

standing panels across the pool of 

qualified reviewers. Research grants that 

were awarded to scientists based on 

investigator-initiated, competitive, peerreviewed proposals generally do not 

raise issues as to independence or 

conflicts. 

4. Choice of Peer Review Mechanism: 

The choice of a peer review mechanism 

(for example, letter reviews or ad hoc 

panels) for influential scientific 

information shall be based on the 

novelty and complexity of the 

information to be reviewed, the 

importance of the information to 

decision making, the extent of prior peer 

review, and the expected benefits and 

costs of review, as well as the factors 

regarding transparency described in 

II(5). 

5. Transparency: The agency—or 

entity managing the peer review—shall 

instruct peer reviewers to prepare a 

report that describes the nature of their 

review and their findings and 

conclusions. The peer review report 

shall either (a) include a verbatim copy 

of each reviewer’s comments (either 

with or without specific attributions) or 

(b) represent the views of the group as 

a whole, including any disparate and 

dissenting views. The agency shall 

disclose the names of the reviewers and 

their organizational affiliations in the 

report. Reviewers shall be notified in 

advance regarding the extent of 

disclosure and attribution planned by 

the agency. The agency shall 

disseminate the final peer review report 

on the agency’s Web site along with all 

materials related to the peer review (any 

charge statement, the peer review 

report, and any agency response). The 

peer review report shall be discussed in 

the preamble to any related rulemaking 

and included in the administrative 

record for any related agency action. 

6. Management of Peer Review 

Process and Reviewer Selection: The 

agency may commission independent 

entities to manage the peer review 

process, including the selection of peer 

reviewers, in accordance with this 

Bulletin. 

III. Additional Peer Review 

Requirements for Highly Influential 

Scientific Assessments 

1. Applicability: This section applies 

to influential scientific information that 

the agency or the Administrator 

determines to be a scientific assessment 

that: 

(i) Could have a potential impact of 

more than $500 million in any year, or 

(ii) Is novel, controversial, or 

precedent-setting or has significant 

interagency interest. 

2. In General: To the extent permitted 

by law, each agency shall conduct peer 

reviews on all information subject to 

this Section. The peer reviews shall 

satisfy the requirements of Section II of 

this Bulletin, as well as the additional 

2676 Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 

requirements found in this Section. 

Principal findings, conclusions and 

recommendations in official reports of 

the National Academy of Sciences that 

fall under this Section are generally 

presumed not to require additional peer 

review. 

3. Selection of Reviewers: a. Expertise 

and Balance: Peer reviewers shall be 

selected based on expertise, experience 

and skills, including specialists from 

multiple disciplines, as necessary. The 

group of reviewers shall be sufficiently 

broad and diverse to fairly represent the 

relevant scientific and technical 

perspectives and fields of knowledge. 

Agencies shall consider requesting that 

the public, including scientific and 

professional societies, nominate 

potential reviewers. 

b. Conflicts: The agency—or the entity 

selecting the peer reviewers—shall (i) 

ensure that those reviewers serving as 

Federal employees (including special 

government employees) comply with 

applicable Federal ethics requirements; 

(ii) in selecting peer reviewers who are 

not government employees, adopt or 

adapt the National Academy of 

Sciences’ policy for committee selection 

with respect to evaluating the potential 

for conflicts (e.g., those arising from 

investments; agency, employer, and 

business affiliations; grants, contracts 

and consulting income). For scientific 

assessments relevant to specific 

regulations, a reviewer’s financial ties to 

regulated entities (e.g., businesses), 

other stakeholders, and the agency shall 

be examined. 

c. Independence: In addition to the 

requirements of Section II (3)(c), which 

shall apply to all reviews conducted 

under Section III, the agency—or entity 

selecting the reviewers—shall bar 

participation of scientists employed by 

the sponsoring agency unless the 

reviewer is employed only for the 

purpose of conducting the peer review 

(i.e., special government employees). 

The only exception to this bar would be 

the rare case where the agency 

determines, using the criteria developed 

by NAS for evaluating use of 

‘‘employees of sponsors,’’ that a premier 

government scientist is (a) not in a 

position of management or policy 

responsibility and (b) possesses 

essential expertise that cannot be 

obtained elsewhere. Furthermore, to be 

eligible for this exception, the scientist 

must be employed by a different agency 

of the Cabinet-level department than the 

agency that is disseminating the 

scientific information. The agency’s 

determination shall be documented in 

writing and approved, on a nondelegable basis, by the Secretary or 

Deputy Secretary of the department 

prior to the scientist’s appointment. 

d. Rotation: Agencies shall avoid 

repeated use of the same reviewer on 

multiple assessments unless his or her 

participation is essential and cannot be 

obtained elsewhere. 

4. Information Access: The agency— 

or entity managing the peer review— 

shall provide the reviewers with 

sufficient information—including 

background information about key 

studies or models—to enable them to 

understand the data, analytic 

procedures, and assumptions used to 

support the key findings or conclusions 

of the draft assessment. 

5. Opportunity for Public 

Participation: Whenever feasible and 

appropriate, the agency shall make the 

draft scientific assessment available to 

the public for comment at the same time 

it is submitted for peer review (or 

during the peer review process) and 

sponsor a public meeting where oral 

presentations on scientific issues can be 

made to the peer reviewers by interested 

members of the public. When 

employing a public comment process as 

part of the peer review, the agency shall, 

whenever practical, provide peer 

reviewers with access to public 

comments that address significant 

scientific or technical issues. To ensure 

that public participation does not 

unduly delay agency activities, the 

agency shall clearly specify time limits 

for public participation throughout the 

peer review process. 

6. Transparency: In addition to the 

requirements specified in II(5), which 

shall apply to all reviews conducted 

under Section III, the peer review report 

shall include the charge to the reviewers 

and a short paragraph on both the 

credentials and relevant experiences of 

each peer reviewer. The agency shall 

prepare a written response to the peer 

review report explaining (a) the agency’s 

agreement or disagreement with the 

views expressed in the report, (b) the 

actions the agency has undertaken or 

will undertake in response to the report, 

and (c) the reasons the agency believes 

those actions satisfy the key concerns 

stated in the report (if applicable). The 

agency shall disseminate its response to 

the peer review report on the agency’s 

Web site with the related material 

specified in Section II(5). 

7. Management of Peer Review 

Process and Reviewer Selection: The 

agency may commission independent 

entities to manage the peer review 

process, including the selection of peer 

reviewers, in accordance with this 

Bulletin. 

IV. Alternative Procedures 

As an alternative to complying with 

Sections II and III of this Bulletin, an 

agency may instead: (i) Rely on the 

principal findings, conclusions and 

recommendations of a report produced 

by the National Academy of Sciences; 

(ii) commission the National Academy 

of Sciences to peer review an agency’s 

draft scientific information; or (iii) 

employ an alternative scientific 

procedure or process, specifically 

approved by the Administrator in 

consultation with the Office of Science 

and Technology Policy (OSTP), that 

ensures the agency’s scientific 

information satisfies applicable 

information quality standards. The 

alternative procedure(s) may be applied 

to a designated report or group of 

reports. 

V. Peer Review Planning 

1. Peer Review Agenda: Each agency 

shall post on its Web site, and update 

at least every six months, an agenda of 

peer review plans. The agenda shall 

describe all planned and ongoing 

influential scientific information subject 

to this Bulletin. The agency shall 

provide a link from the agenda to each 

document that has been made public 

pursuant to this Bulletin. Agencies are 

encouraged to offer a listserve or similar 

mechanism to alert interested members 

of the public when entries are added or 

updated. 

2. Peer Review Plans: For each entry 

on the agenda the agency shall describe 

the peer review plan. Each peer review 

plan shall include: (i) A paragraph 

including the title, subject and purpose 

of the planned report, as well as an 

agency contact to whom inquiries may 

be directed to learn the specifics of the 

plan; (ii) whether the dissemination is 

likely to be influential scientific 

information or a highly influential 

scientific assessment; (iii) the timing of 

the review (including deferrals); (iv) 

whether the review will be conducted 

through a panel or individual letters (or 

whether an alternative procedure will 

be employed); (v) whether there will be 

opportunities for the public to comment 

on the work product to be peer 

reviewed, and if so, how and when 

these opportunities will be provided; 

(vi) whether the agency will provide 

significant and relevant public 

comments to the peer reviewers before 

they conduct their review; (vii) the 

anticipated number of reviewers (3 or 

fewer; 4–10; or more than 10); (viii) a 

succinct description of the primary 

disciplines or expertise needed in the 

review; (ix) whether reviewers will be 

selected by the agency or by a 

Federal Register / Vol. 70, No. 10 / Friday, January 14, 2005 / Notices 2677 

designated outside organization; and (x) 

whether the public, including scientific 

or professional societies, will be asked 

to nominate potential peer reviewers. 

3. Public Comment: Agencies shall 

establish a mechanism for allowing the 

public to comment on the adequacy of 

the peer review plans. Agencies shall 

consider public comments on peer 

review plans. 

VI. Annual Reports 

Each agency shall provide to OIRA, by 

December 15 of each year, a summary of 

the peer reviews conducted by the 

agency during the fiscal year. The report 

should include the following: (1) The 

number of peer reviews conducted 

subject to the Bulletin (i.e., for 

influential scientific information and 

highly influential scientific 

assessments); (2) the number of times 

alternative procedures were invoked; (3) 

the number of times waivers or deferrals 

were invoked (and in the case of 

deferrals, the length of time elapsed 

between the deferral and the peer 

review); (4) any decision to appoint a 

reviewer pursuant to any exception to 

the applicable independence or conflict 

of interest standards of the Bulletin, 

including determinations by the 

Secretary pursuant to Section III(3)(c); 

(5) the number of peer review panels 

that were conducted in public and the 

number that allowed public comment; 

(6) the number of public comments 

provided on the agency’s peer review 

plans; and (7) the number of peer 

reviewers that the agency used that were 

recommended by professional societies. 

VII. Certification in the Administrative 

Record 

If an agency relies on influential 

scientific information or a highly 

influential scientific assessment subject 

to this Bulletin to support a regulatory 

action, it shall include in the 

administrative record for that action a 

certification explaining how the agency 

has complied with the requirements of 

this Bulletin and the applicable 

information quality guidelines. Relevant 

materials shall be placed in the 

administrative record. 

VIII. Safeguards, Deferrals, and Waivers

1. Privacy: To the extent information 

about a reviewer (name, credentials, 

affiliation) will be disclosed along with 

his/her comments or analysis, the 

agency shall comply with the 

requirements of the Privacy Act, 5 

U.S.C. 522a as amended, and OMB 

Circular A–130, Appendix I, 61 FR 6428 

(February 20, 1996) to establish 

appropriate routine uses in a published 

System of Records Notice. 

2. Confidentiality: Peer review shall 

be conducted in a manner that respects 

(i) confidential business information 

and (ii) intellectual property. 

3. Deferral and Waiver: The agency 

head may waive or defer some or all of 

the peer review requirements of 

Sections II and III of this Bulletin where 

warranted by a compelling rationale. If 

the agency head defers the peer review 

requirements prior to dissemination, 

peer review shall be conducted as soon 

as practicable. 

IX. Exemptions 

Agencies need not have peer review 

conducted on information that is: 

1. Related to certain national security, 

foreign affairs, or negotiations involving 

international trade or treaties where 

compliance with this Bulletin would 

interfere with the need for secrecy or 

promptness; 

2. Disseminated in the course of an 

individual agency adjudication or 

permit proceeding (including a 

registration, approval, licensing, sitespecific determination), unless the 

agency determines that peer review is 

practical and appropriate and that the 

influential dissemination is 

scientifically or technically novel or 

likely to have precedent-setting 

influence on future adjudications and/or 

permit proceedings; 

3. A health or safety dissemination 

where the agency determines that the 

dissemination is time-sensitive (e.g., 

findings based primarily on data from a 

recent clinical trial that was adequately 

peer reviewed before the trial began); 

4. An agency regulatory impact 

analysis or regulatory flexibility analysis 

subject to interagency review under 

Executive Order 12866, except for 

underlying data and analytical models 

used; 

5. Routine statistical information 

released by federal statistical agencies 

(e.g., periodic demographic and 

economic statistics) and analyses of 

these data to compute standard 

indicators and trends (e.g., 

unemployment and poverty rates); 

6. Accounting, budget, actuarial, and 

financial information, including that 

which is generated or used by agencies 

that focus on interest rates, banking, 

currency, securities, commodities, 

futures, or taxes; or 

7. Information disseminated in 

connection with routine rules that 

materially alter entitlements, grants, 

user fees, or loan programs, or the rights 

and obligations of recipients thereof. 

X. Responsibilities of OIRA and OSTP 

OIRA, in consultation with OSTP, 

shall be responsible for overseeing 

implementation of this Bulletin. An 

interagency group, chaired by OSTP and 

OIRA, shall meet periodically to foster 

better understanding about peer review 

practices and to assess progress in 

implementing this Bulletin. 

XI. Effective Date and Existing Law 

The requirements of this Bulletin, 

with the exception of those in Section 

V (Peer Review Planning), apply to 

information disseminated on or after six 

months following publication of this 

Bulletin, except that they do not apply 

to information for which an agency has 

already provided a draft report and an 

associated charge to peer reviewers. Any 

existing peer review mechanisms 

mandated by law shall be employed in 

a manner as consistent as possible with 

the practices and procedures laid out 

herein. The requirements in Section V 

apply to ‘‘highly influential scientific 

assessments,’’ as designated in Section 

III of this Bulletin, within six months of 

publication of this Bulletin. The 

requirements in Section V apply to 

documents subject to Section II of this 

Bulletin one year after publication of 

this Bulletin. 

XII. Judicial Review 

This Bulletin is intended to improve 

the internal management of the 

executive branch, and is not intended 

to, and does not, create any right or 

benefit, substantive or procedural, 

enforceable at law or in equity, against 

the United States, its agencies or other 

entities, its officers or employees, or any 

other person. 

John D. Graham, 

Administrator, Office of Information and 

Regulatory Affairs. 

[FR Doc. 05–769 Filed 1–13–05; 8:45 am] 

BILLING CODE 3110–01–P 

OVERSEAS PRIVATE INVESTMENT 

CORPORATION 

Sunshine Act Meeting; Board of 

Directors 

TIME AND DATE: Thursday, January 27, 

2005, 9:30 a.m. (open portion); 9:45 a.m. 

(closed portion). 

PLACE: Offices of the Corporation, 

Twelfth Floor Board Room, 1100 New 

York Avenue, NW., Washington, DC. 

STATUS: Meeting open to the public from 

9:30 a.m. to 9:45 a.m.; closed portion 

will commence at 9:45 a.m. (approx.). 

MATTERS TO BE CONSIDERED: 

1. President’s Report. 

2. Approval of November 10, 2004 

Minutes (open portion).