Appendix C

NRC Report Development Procedures

1) The National Academies: Our Study Process – Ensuring Independent, Objective

Advice

2) The National Academies: Getting to Know the Committee Process

3) Report Review Committee Guidelines for Review: Consensus Reports

118

119

121

141
submission of information by outside parties; 3) reviews of

the scientific literature, and 4) the investigations of the

committee members and staff. In all cases, efforts are

made to solicit input from individuals who have been

directly involved in, or who have special knowledge of, the

problem under consideration. 

In accordance with federal law and with few exceptions,

information-gathering meetings of the committee are open

to the public, and any written materials provided to the committee by individuals who are not officials, agents, or

employees of the National Academies are maintained in a

public access file that is available for examination. 

The committee deliberates in meetings closed to the public in order to develop draft findings and recommendations

free from outside influences. The public is provided with

brief summaries of these meetings that include the list of

committee members present. All analyses and drafts of the

report remain confidential.

STAGE 4. Report Review

As a final check on the quality and objectivity of the study, 

all National Academies reports—whether products of 

studies, summaries of workshop proceedings, or other 

documents—must undergo a rigorous, independent external review by experts whose comments are provided 

anonymously to the committee members. The National

Academies recruit independent experts with a range of

views and perspectives to review and comment on the draft

report prepared by the committee. 

The review process is structured to ensure that each report

addresses its approved study charge and does not go beyond

it, that the findings are supported by the scientific evidence

and arguments presented, that the exposition and organization are effective, and that the report is impartial and objective. 

Each committee must respond to, but need not agree

with, reviewer comments in a detailed “response to review”

that is examined by one or two independent report review

“monitors” responsible for ensuring that the report review

criteria have been satisfied. After all committee members

and appropriate National Academies officials have signed

off on the final report, it is transmitted to the sponsor of the

study and is released to the public. Sponsors are not given

an opportunity to suggest changes in reports. The names

and affiliations of the report reviewers are made public

when the report is released.

STAGE 3. Committee Meetings,

Information Gathering, Deliberations,

and Drafting the Report

Study committees typically gather information through: 1)

meetings that are open to the public and that are announced

in advance through the National Academies Web site; 2) the

HOW THE PUBLIC CAN FOLLOW 

AND PROVIDE INPUT TO STUDIES

The Current Projects System was established

with a link from the National Academies homepage, www.national-academies.org, to

make it easy for members of the general 

public with interest in the subject to follow the

progress of a study and submit comments.

The system offers separate views by subject

and by project title. 

Reports of the National Academies are

available from the National Academies Press,

500 Fifth Street, NW, Washington, DC 20001 

1-800-624-6242 • www.nap.edu.

interest, no individual can be appointed to serve (or continue to serve) on a committee of the institution used in the

development of reports if the individual has a conflict of 

interest that is relevant to the functions to be performed. For

more information, see the National Academies’ Web site at

www.national-academies.org.

Other considerations. Membership in the three Academies

(NAS, NAE, IOM) and previous involvement in National

Academies studies are taken into account in committee 

selection. The inclusion of women, minorities, and young professionals are additional considerations.

Specific steps in the committee selection and approval

process are as follows:

 Staff solicit an extensive number of suggestions for

potential committee members from a wide range of

sources, then recommend a slate of nominees. 

 Nominees are reviewed and approved at several levels within the National Academies; a provisional slate is then

approved by the President of the National Academy of

Sciences, who is also the Chair of the National Research

Council. 

 The provisional committee list is posted for public 

comment in the Current Projects System on the Web 

(http://www4.national-academies.org/cp.nsf). 

 The provisional committee members complete background information and conflict of interest disclosure

forms. 

 The committee balance and conflict of interest discussion

is held at the first committee meeting. 

 Any conflicts of interest or issues of committee balance

and expertise are investigated; changes to the committee

are proposed and finalized. 

 Committee is formally approved. 

 Committee members continue to be screened for conflict

of interest throughout the life of the committee.

Ensuring Independent,

Objective Advice

OUR STUDY

PROCESS

NATIONALACADEMIES

THE

National Academy of Sciences

National Academy of Engineering

Institute of Medicine

National Research Council

119
Project

starts

when

funding is

received

Provisional

slate

approved by

NAS

President

Provisional

committee

posted 

for public

comment

via Current

Projects

System

Committee 

balance and 

expertise 

evaluated; 

any conflicts 

of interest are 

investigated

Final 

committee 

formally 

approved

Monitoring for 

potential conflicts 

of interest 

continues

STEPS TAKEN TO ENSURE

INDEPENDENCE AND OBJECTIVITY

The reports of the National Academies are viewed as being

valuable and credible because of the institution’s reputation for providing independent, objective, and non-partisan

advice with high standards of scientific and technical quality. Checks and balances are applied at every step in the

study process to protect the integrity of the reports and to

maintain public confidence in them. The study process can

be broken down into four major stages: 1) defining the

study; 2) committee selection and approval; 3) committee

meetings, information gathering, deliberations, and drafting

of the report; and 4) report review.

STAGE 1. Defining the Study

Before the committee selection process begins, National

Academies’ staff and members of their boards work with

sponsors to determine the specific set of questions to be

addressed by the study in a formal “statement of task,” as

well as the duration and cost of the study. The statement of

task defines and bounds the scope of the study, and it

serves as the basis for determining the expertise and the

balance of perspectives needed on the committee. 

The statement of task, work plan, and budget must be

approved by the Executive Committee of the National

Research Council Governing Board. This review often results

in changes to the proposed task and work plan. On occasion,

it results in turning down studies that the institution believes are

inappropriately framed or not within its purview. 

STAGE 2. Committee Selection and

Approval

Selection of appropriate committee members, individually

and collectively, is essential for the success of a study. All

committee members serve as individual experts, not as

representatives of organizations or interest groups. Each

member is expected to contribute to the project on the

basis of his or her own expertise and good judgment. A

committee is not finally approved until a thorough balance

and conflict of interest discussion is held at the first meeting, and any issues raised in that discussion or by the public are investigated and addressed. 

Careful steps are taken to convene committees that meet

the following criteria:

An appropriate range of expertise for the task. The

committee must include experts with the specific expertise

and experience needed to address the study’s statement of

task. One of the strengths of the National Academies is the

tradition of bringing together recognized experts from

diverse disciplines and backgrounds who might not otherwise collaborate. These diverse groups are encouraged to

conceive new ways of thinking about a problem.

A balance of perspectives. Having the right expertise is

not sufficient for success. It is also essential to evaluate the

overall composition of the committee in terms of different

experiences and perspectives. The goal is to ensure that

the relevant points of view are, in the National Academies’

judgment, reasonably balanced so that the committee can

carry out its charge objectively and credibly. 

POINT OF VIEW IS DIFFERENT

FROM CONFLICT OF INTEREST

A point of view or bias is not necessarily a conflict

of interest. Committee members are expected to

have points of view, and the National Academies

attempt to balance these points of view in a way

deemed appropriate for the task. Committee

members are asked to consider respectfully the

viewpoints of other members, to reflect their own

views rather than be a representative of any

organization, and to base their scientific findings

and conclusions on the evidence. Each committee member has the right to issue a dissenting

opinion to the report if he or she disagrees with the

consensus of the other members.

NRC Governing

Board reviews and

approves study

scope and plan 

Committee’s

first meeting

Report is

released to

the sponsor

and the public

Committee and National

Academies sign off on

report

Full committee signs off on

draft report

Reviewers

comment 

on report 

DEFINING THE STUDY COMMITTEE SELECTION AND APPROVAL

COMMITTEE MEETINGS, INFORMATION GATHERING, DELIBERATIONS, AND DRAFTING REPORT

REPORT REVIEW→

Screened for conflicts of interest. All provisional committee members are screened in writing and in a confidential group discussion about possible conflicts of interest. For this purpose, a “conflict of interest” means any

financial or other interest which conflicts with the service

of the individual because it could significantly impair the

individual's objectivity or could create an unfair competitive advantage for any person or organization. The term

“conflict of interest” means something more than individual bias. There must be an interest, ordinarily financial, that

could be directly affected by the work of the committee.

Except for those rare situations in which the National

Academies determine that a conflict of interest is unavoidable and promptly and publicly disclose the conflict of

STAGE 1 STAGE 2 STAGE 3 STAGE 4

F

or more than 140 years, the National Academies

have been advising the nation on issues of science,

technology, and medicine. The 1863 Congressional

charter signed by President Lincoln authorized this

non-governmental institution to honor top scientists

with membership and to serve the nation whenever called

upon. Today the National Academies—National Academy

of Sciences, National Academy of Engineering,

Institute of Medicine, and National Research Council—

continue that dual mission.

Like no other organization, the National Academies can

enlist the nation’s foremost scientists, engineers, health professionals, and other experts to address the scientific and

technical aspects of society’s most pressing problems. Each

year, more than 6,000 of these experts are selected to serve

on hundreds of study committees that are convened to

answer specific sets of questions. All serve without pay.

Federal agencies are the primary financial sponsors of the

Academies’ work. Additional studies are funded by state

agencies, foundations, other private sponsors, and the

National Academies endowment. The Academies provide

independent advice; the external sponsors have no control

over the conduct of a study once the statement of task and

budget are finalized. Study committees gather information

from many sources in public meetings but they carry out

their deliberations in private in order to avoid political, special

interest, and sponsor influence. 

Through this careful study process, the National Academies

produce 200–300 authoritative reports each year. Recent

reports cover such topics as the obesity epidemic, the use

of forensics in the courtroom, invasive plants, underage

drinking, the Hubble Telescope, vaccine safety, the hydrogen

economy, transportation safety, climate change, and homeland security. Many reports influence policy decisions; some

are instrumental in enabling new research programs; others

provide program reviews.

120

The National Academy

of Sciences was established

by Congress more than a 

century ago to provide scientific and technological

advice to the nation. Over

the years, the Academy

has evolved to incorporate

four distinguished organizations the National

Academy of Sciences, the

National Academy of

Engineering, the Institute

ABOUT THIS UNIQUE INSTITUTION 2

THE STUDY PROCESS 5

Committee Selection 6

Bias and Conflict of Interest 6

Committee Deliberations 7

Studies of Risk Assessment 8

Institutional Oversight 8

Writing the Report 9

Role of Staff 9

Report Review 10

Confidentiality 10

Public Access 12

DISSEMINATION 14

Congressional Outreach 14

Academies’ News Office 15

National Academies Press 15

Communications Initiative 16

Academies’ Web Site 16

CONTENTS

USEFUL REFERENCES 1 7

122
ACADEMIES

ASKED TO SERVE . . .

THE NATIONAL

Y

SO YOU'VE BEEN 

ou have been invited to

work on a project at the National

Academies and may be wondering

exactly what your role is as a committee member. This document is a brief

introduction to the institution and is

designed to give you a sense of the

committee process. No two projects

are alike, of course, and people’s

experiences vary. But general policies

and procedures have been developed

to ensure that the time you spend as a

volunteer in service to the nation is

productive and rewarding.

123
ACADEMIES

THE NATIONAL

2

The National Academy of Sciences was established by Congress more than a 

century ago to provide scientific and technological advice to the nation. Over the

years, the Academy has evolved to incorporate four distinguished organizations

— the National Academy of Sciences, the National Academy of Engineering, the

Institute of Medicine, and the National Research Council. Known collectively as

the National Academies, they perform an unparalleled public service by bringing

together experts in all areas of scientific and technological endeavor. These

experts serve as volunteers to address critical national issues and give unbiased

advice to the federal government and the public. Most of this advice is provided

either by the National Research Council — the chief operating arm of the

Academy of Sciences and the Academy of Engineering — or by the Institute of

Medicine, which operates under the charter of the National Academy of Sciences

and according to Research Council rules.

ABOUT THIS UNIQUE INSTITUTION

JD TALASEK

124
The National Academies provide science and technology advice in several different forms: 

written reports reflecting the consensus reached by an expert study committee; symposia and

convocations engaging large audiences in discussion of national issues; proceedings from conferences and workshops; or “white papers” on policy issues of special interest. Each project is

conducted or overseen by a committee serving pro bono, whose members are selected for

their expertise on the subject.

This booklet focuses primarily on the study committee process, in which a committee of

approximately 10 to 20 members with a diverse range of expertise and perspectives is 

convened to address a particular question or set of questions. Study committees have been

assembled, for example, to address various aspects of the AIDS crisis, to consider the use of

forensics techniques in the courts, and to offer policy advice on setting priorities for federal

funding of research and development. Before a committee meets, one of the volunteer experts

is appointed to serve as the committee chair, and a member of the Academies’ staff is

assigned as the study director.

Each committee investigates the many facets of the problem described in its statement of task

and develops a report of its findings, conclusions, and recommendations, based on the 

available scientific evidence. This report is subjected to rigorous review by a second group 

of independent experts anonymous to the committee. After revisions are made by the committee to satisfy the institution’s review process, the report is transmitted to the agency or 

organization that sponsored the study, and then released to the public. Names and affiliations

of principal reviewers are made public when the report is released.

The National Academies and their operations are distinctive in several ways:

• Committee members serve without compensation, except for reimbursement of expenses. 

• Most projects originate from requests made by outside sponsors such as Congress, 

federal agencies, and foundations. However, volunteer members of the institution’s 

boards also develop ideas for studies that subsequently are funded by external sources. 

In addition, the Academies have been building their own endowments through private 

contributions so that they can address important issues of science and policy without 

government or foundation support. 

3

125
ACADEMIES

THE NATIONAL

4

• The institution has no research laboratories. In reaching conclusions and recommendations, study committees hear invited testimony and evaluate published research from the 

relevant scientific literature, as part of an educational process designed to enable the 

committee to reach consensus. The resulting reports often present a novel synthesis of 

ideas unique to the committee; these reports are highly regarded by the sponsoring 

agencies and the public for their thorough analysis and carefully supported recommendations. The reputation of the institution for objectivity, integrity, independence, and 

competence is one of its most valuable assets. For this reason, procedures designed to 

ensure excellence apply to each study undertaken.

• The National Academies do not compete in response to federal requests for proposals. 

Their one-of-a-kind service, not duplicated by other organizations, was reaffirmed in a 

January 1993 executive order from President Bush.

• The Academies are not subject to the Federal Advisory Committee Act in the way 

government agencies are; neither are they subject to the Freedom of Information Act. 

However, the Federal Advisory Committee Act Amendments of 1997 (Public Law 

105-153) require the Academies to provide opportunities for public access and 

involvement in the study process. Those requirements are discussed in the Public Access

section of this booklet.

126
5

The core of the institution’s work consists of studies usually of six months to two

years in duration and performed under contract to a sponsor or set of sponsors.

Each study is conducted by a committee selected expressly for that purpose. The

committee meets at intervals to consider its scope of work, to review the relevant

scientific evidence, and to develop its findings, conclusions, and recommendations. Once the report has been outlined by the committee, various sections often

are written by individual members. The writing process is guided by the committee

chair and aided by the committee staff, in particular by the study director.

The charge to the committee — developed before committee members are

selected — is the formal statement of the questions to be addressed by the

study. This statement defines the study’s scope and issues to be examined. If a

committee finds in the course of its work that this description is inadequate, the

charge can be formally modified through petition to the Executive Committee of

the National Research Council’s Governing Board. Such petitions are carefully

examined; a committee that is well-balanced for one purpose may not be 

appropriately constructed for a modified task.

THE STUDY PROCESS

JD TALASEK

127
ACADEMIES

COMMITTEE SELECTION

The search for candidates for committee membership is initiated by staff with input and oversight from the relevant boards. In defining the areas of expertise that should be represented 

on a committee and identifying individuals qualified to serve, the staff reviews scholarly 

literature and consults widely with the institution’s members and volunteers, knowledgeable

authorities, and professional associations. Sponsors may offer suggestions but do not select

committee members.

Committee members are chosen on the basis of their knowledge and experience in the various

aspects of the topic to be investigated, and after careful review are appointed by the chair of

the National Research Council, who also is the president of the National Academy of 

Sciences. The names, affiliations, and short biographies of committee members are posted 

for public comment in the “Current Projects” area of the institution’s Web site at 

<national-academies.org>.

BIAS AND CONFLICT OF INTEREST 

Committee members serve as individuals, not as representatives of organizations or interest

groups. Each person is selected on the basis of his or her expertise and good judgment, and

is expected to contribute accordingly to the study.

The credibility of a report can be called into question if the committee that produced it is perceived to be biased. Potential sources of bias and conflict of interest are significant issues that

are taken into consideration in the selection of committee members and are re-examined periodically throughout the study process.

Recognizing that each individual who is knowledgeable about a subject brings his or her own

biases and experiences to any study effort, the institution has adopted specific procedures to

achieve appropriate balance in the committee membership and to avoid conflicts of interest. At

the time of appointment, each committee member is required to list all professional, consulting,

and financial connections, as well as to describe pertinent intellectual positions and public

statements by filling out a confidential form, “Background Information and Confidential Conflict

of Interest Disclosure.” The committee appointment is not finalized until the institution completes a review of information regarding potential conflicts of interest and bias.

THE NATIONAL

6

128
As part of the process of becoming acquainted with each other and with the task before them,

committee members discuss this information in closed session at the beginning of their first

meeting and annually thereafter. The information also is reviewed by officials of the institution,

and if a potential conflict becomes apparent, the committee member may be asked to resign.

In exceptional circumstances, an individual may continue to serve on the committee if the 

conflict of interest is promptly and publicly disclosed, and the National Academies have determined that the conflict is unavoidable. To fulfill its legal requirement for such public disclosure,

the institution posts on its Web site a brief statement describing the unavoidable conflict. When

a question of balance arises, the usual procedure is to add members to the committee to

achieve the appropriate balance.

COMMITTEE DELIBERATIONS

A successful report is the result of a dynamic group process, requiring that committee members be open to new ideas and innovative solutions, and be willing to learn from one another.

Committees are expected to be evenhanded and to examine all evidence dispassionately.

Although all interested parties should be heard and their views given serious and respectful

consideration, one of the committee’s primary roles is to separate fact from opinion, analysis

from advocacy. Scientific standards are essential in evaluating all arguments and alternatives.

Most committees eventually issue a unanimous report of their conclusions. Members of committees should strive for consensus, but not at the cost of substantially weakening their analyses and conclusions. It may be more valuable in the long run to explain the rationale behind

areas of disagreement than to issue unanimous conclusions that are so limited that they fail to

contribute to a better understanding of the issue.

7

PAUL KENNEDY

129
ACADEMIES

STUDIES OF RISK ASSESSMENT

Studies involving the assessment of risk are among the most difficult and potentially 

controversial. Government agencies and others engaged in the management of hazardous 

conditions and substances often are faced with making policy decisions in the absence 

of conclusive scientific evidence. Many Research Council and Institute of Medicine studies

dealing with risk are requested by federal agencies seeking specific answers on which to 

base important policy decisions. In studies involving incomplete or inconclusive data, the 

collective experience and reasoned judgment of committee members become crucial factors 

in developing conclusions and recommendations.

It is critical for each committee working on a study of risk assessment to distinguish clearly

between conclusions based on scientific evidence and those based on informed judgment. 

In cases in which scientific proof is incomplete, special care is needed to explain how the 

committee arrived at its conclusions. Moreover, the assumptions used should be explicitly 

identified and justified. A conscientious effort to be clear in writing the report will help to avoid

the potential for misinterpretation when the report is published.

The institution has developed guidelines for committees charged with conducting 

risk-related projects (available on the Academies’ intranet site, the AcademyNet). These guidelines emphasize the special care that must be taken in assembling the committee, orienting

new committee members, conducting bias discussions, managing the consensus process,

handling any minority opinions, and writing the report.

INSTITUTIONAL OVERSIGHT

Every study is subject to oversight, from initial approval to public release of a final report.

Whether a study is requested by a government agency or a private organization, or is initiated within the institution itself, the study proposal first must be approved by the Executive

Committee of the Research Council’s Governing Board. This group carefully examines the

proposal and considers such factors as the importance and timeliness of the question,

whether there is an adequate base of scientific knowledge to support the study, the intended audience, the likely impact of the report, and the competence of the institution to

take on the task.

Oversight is provided throughout the duration of a study by the various supervisory entities

within the Research Council and the Institute of Medicine, as well as by the committee chair

and the staff member serving as study director. Their responsibilities are to ensure that the

THE NATIONAL

8

130
committee focuses on its stated tasks, that measures of quality control are enforced, and that

the study proceeds on schedule and within budget. Sponsors do not engage in oversight of

the studies.

WRITING THE REPORT

Completing the consensus-building process and writing a report that clearly presents the committee’s findings, conclusions, and recommendations may be the most challenging, yet rewarding aspects of serving on a study committee. The report may well become an important reference for those who formulate public policy. For this reason, the value of a carefully prepared

report cannot be overstated.

Although each committee may go about the drafting of its report differently, every report is the

collective product of a group process. A committee member often will draft a chapter or portion of the report, but the “author of record” is the entire committee, and the responsibility for

authorship lies with the committee as well. Individual authorship generally is not credited; the

report and all copyrights become the property of the National Academy of Sciences.

ROLE OF STAFF

Each committee is assisted in its work by highly qualified staff members who facilitate the work of

the committee during the conduct of the study. When committee and staff form a close professional partnership, the experience can be exhilarating for everyone involved.

Staff help to create the objective atmosphere in which the committee’s deliberations take

place. In addition, staff are responsible for ensuring that institutional procedures and practices

are followed throughout the study, and that the study stays on schedule and within budget.

Staff members assist with many aspects of assembling the report, including researching, writing, integrating portions written by others, and ensuring consistent style and format. However,

the conclusions and recommendations are those of the committee. Staff do not insert their

personal conclusions or recommendations into the report.

9

131
ACADEMIES

THE NATIONAL

10

REPORT REVIEW 

Like all good science, reports should be based on fact and rigorous analysis. All of the institution’s reports — whether products of studies, summaries of workshop proceedings, or abbreviated documents — must undergo an independent review by anonymous experts who were not

involved in the report’s preparation. 

Report review is an integral and constructive part of the study process. It is the final opportunity

for committees to test their reasoning, conclusions, and recommendations before release of the

report to the public.

The Report Review Committee (RRC) ensures that an independent review has been 

conducted, and that:

• the report addresses the approved study charge and does not go beyond it; 

• the findings are supported by the evidence and arguments presented; 

• the exposition and organization are effective; and 

• the tone of the report is impartial, and sensitive policy issues are treated with 

appropriate care. 

The report may not be transmitted to the sponsor or released to the public until review has

been completed to the satisfaction of the Report Review Committee. Details of the quality

standards followed by reviewers can be found in the RRC document Guidelines for Review:

Consensus Reports. Once the report is released, names and affiliations of principal reviewers

are made public.

CONFIDENTIALITY

During more than a century of service, the institution has earned a reputation for providing

independent, expert advice. Procedures and practices have evolved that protect committees

from outside pressures and thereby safeguard the credibility and integrity of their work. 

Committee meetings, particularly as the committee gathers information, are usually open to

interested individuals and the news media. However, meetings are closed when the committee

is deliberating to develop its findings and during discussion of financial and personnel matters.

Closed meetings are not open to the public or to any person who is not a committee 

member or an official, agent, or employee of the Academies.

132
Reports are the product of the institution, not of the committee alone. Committee deliberations,

drafts of the report in progress, and tentative conclusions all are confidential until a completed

report passes through review and receives sign-off by the Report Review Committee and by

the major unit responsible for the study. Committee members are expected to reject any

requests for early briefings or media interviews on the committee’s findings, and to treat 

committee deliberations and draft products as confidential.

A cardinal rule to keep in mind: Until the review process has been satisfactorily completed,

the document is not an official report of the National Academies. Conclusions and recommendations can change up to the final sign-off; premature briefings for sponsors or others outside

the institution may lock committees into a position not fully supported by the evidence. Early

briefings also damage the final report by subjecting the committee to the accusation that it

permitted the sponsor to preview and approve the conclusions and recommendations — a

serious charge that undermines the independence and integrity of both the committee and

the institution. In such cases, the hard work of the committee can be discredited, diminishing

the report’s value to the sponsor and to the nation.

Until the report is publicly released, committee members should limit public comment to 

the following:

• the scope of the project and what the committee has been asked to do; 

• the name of the sponsor and estimated cost of the study, if known; and 

• the makeup of the committee, including names and affiliations of 

committee members.

11

GREG HADLEY

133
ACADEMIES

This and other relevant information can be found on

the institution’s Web site at <national-academies.org>

and in its public access files. These files, which

include items such as materials presented to the

committee in data-gathering meetings open to the

public, are available for public review.

PUBLIC ACCESS

Legislation passed in late 1997 protects the

Academies from government control under the

Federal Advisory Committee Act. But in doing so, it

explicitly requires the Academies to ensure public

access to committee activities.

For example, any meeting of a committee at which

anyone other than committee members or officials,

agents, or employees of the institution is present —

whether in person, by telephone, or teleconference

— is considered a “data-gathering committee meeting.” Except as determined and approved in advance

by the Academies’ leadership, all data-gathering

committee meetings are open to the public. Datagathering meetings that involve committee consideration of classified, proprietary, or personal-privacy

information, however, exemplify situations in which

meetings are not open to the public.

To facilitate the process of informing the public about

a committee’s work and enabling interested individuals to attend open data-gathering sessions, an

advance announcement must be posted — preferably 14 days before the meeting — on the institution’s Web site.

Data-gathering committee meetings should be

regarded as on the record. Therefore, whether or not

THE NATIONAL

12

134
representatives of the news media are in attendance, the chair of the meeting should advise

everyone present of the nature and purpose of the

meeting. Statements of this type are necessary to

help ensure that participants and observers do not

misinterpret the purpose of the meeting, or prematurely interpret the discussion to be the positions of

individual participants, the committee, or the institution. For guidance on assembling introductory

remarks for the chair, see “Setting the Ground Rules

at an Open Meeting to Which the Public and Press

Have Been Invited as Observers,” available on the

AcademyNet.

To acquaint the public with the background of committee members, the chair should ask each member to state briefly, in open session at the first 

data-gathering committee meeting, those aspects

of his or her background, experience, expertise,

and previously stated positions that appear relevant

to the functions to be performed by the committee.

Committees also should create opportunities that

facilitate the gathering of as wide a range of views

as possible, such as having a session for public

comments at a data-gathering committee meeting

or soliciting comments in writing or via e-mail from

interested members of the public.

Within 10 days following a closed committee meeting, the Academies will post a brief summary of the

meeting on the Web, listing the committee members

present, the topics discussed, and materials made

available to the committee. This summary will not

disclose the substantive content, conclusions, 

recommendations, discussion of draft reports, or

any report review comments.

13

JD TALASEK

135
ACADEMIES

THE NATIONAL

14

Once a report has successfully completed review and been thoroughly edited, Academies

staff working with the Office of Congressional and Government Affairs and the Office of

News and Public Information develop a release timeline. At this stage it is critical to 

observe carefully the institution’s rules of confidentiality until the report is ready for release 

to the public.

CONGRESSIONAL OUTREACH

The Office of Congressional and Government Affairs (OCGA) is responsible for dissemination

and outreach to the congressional and executive branches of government. One component

of this is congressional and sponsor briefings. These briefings take place up to 24 hours

before the public release of a report. Dissemination of reports that are congressionally mandated involves specific guidelines for briefing members of Congress. For other reports, OCGA

informs concerned members of Congress and the appropriate congressional committees. 

OCGA also monitors congressional activities on issues that affect the National Academies and

helps committee members or staff prepare and submit testimony for Congress.

DISSEMINATION

136
ACADEMIES’ NEWS OFFICE

The Office of News and Public Information (ONPI) is the liaison between the Academies and

the news media and general public. Many reports of the National Academies are newsworthy,

and the media serve as an important channel for disseminating the content of reports.

For each report, ONPI works closely with study staff to develop a release plan that may include

a news release; public briefing; embargo arrangement; outreach to targeted media; and media

preparation for committee members to help them speak comfortably with journalists and handle potentially difficult inquiries.

Committee members or staff who receive a call from a reporter concerning a pending report

generally should feel free to respond, but in a limited way. Journalists may be provided with a

project’s statement of task, the committee roster, the identity of the sponsor(s), and the cost of

the study, as well as general information about the background and scope of the project.

Findings and recommendations, however, must be kept confidential until the report is released. 

ONPI should be informed of substantive conversations with the news media, especially if there

is any problem. Staff or committee members seeking guidance about media relations may 

consult with ONPI at any time. If a reporter is asking leading questions or inquiring about 

controversial issues, it often is best to delay answering until after conferring with ONPI.

ONPI also publicizes Academies reports through weekly and monthly electronic newsletters

and the institution’s magazine, The National Academies In Focus.

NATIONAL ACADEMIES PRESS

The National Academies Press (NAP) is the publisher

for the institution. NAP offers a wide range of services,

including publication planning, editing, printing, marketing, and distribution. It publishes nearly 200 committee

reports each year, ranging from pre-publication photocopies to full-color, high-quality trade books that are

marketed and sold around the world. The NAP Web

site at <www.nap.edu> makes all of these publications

available online — more than 3,000 books and 500,000

book pages in 2005. Most books published by NAP are

also offered as PDF files, many of which are free 

to the public. 

15

137
Current committee members are permitted free access to all available PDF files and should

contact study staff to make the necessary arrangements; they also receive a 25 percent 

discount on all books purchased from NAP for personal use.

COMMUNICATIONS INITIATIVE

The National Academies strive to disseminate the results of their work to a wide range of audiences. The Office of Communications was formed in 2001 to help the institution accomplish

this. Specifically, this office works in partnership with Academies program units to engage the

public with the institution’s work, create new products and services for different audiences, build

stronger relationships with existing and potential sponsors, and harness the Web as an effective

communication vehicle. Committees are encouraged to identify potential audiences for their

studies early in the process, discuss how to reach those audiences, and consider ways of

expanding public interest throughout the study process.

ACADEMIES’ WEB SITE

Information about the institution’s work is available on the Web at <national-academies.org>. 

A broad range of material can be found online, including current project information, news

releases, and full-text reports.

ACADEMIES

THE NATIONAL

16

138
The National Academy

of Sciences was established

by Congress more than a 

century ago to provide scientific and technological

advice to the nation. Over

the years, the Academy

has evolved to incorporate

four distinguished organizations the National

Academy of Sciences, the

National Academy of

Engineering, the Institute

ABOUT THIS UNIQUE INSTITUTION 2

THE STUDY PROCESS 5

Committee Selection 6

Bias and Conflict of Interest 6

Committee Deliberations 7

Studies of Risk Assessment 8

Institutional Oversight 8

Writing the Report 9

Role of Staff 9

Report Review 10

Confidentiality 10

Public Access 12

DISSEMINATION 14

Congressional Outreach 14

Academies’ News Office 15

National Academies Press 15

Communications Initiative 16

Academies’ Web Site 16

CONTENTS

USEFUL REFERENCES 1 7
140
CONSENSUS 

REPORTS

RepoRt Review Committee

GUIDELINES FOR REVIEW THE

NATIONALACADEMIES

may not be released to the sponsors or the public, nor 

may its findings be disclosed, until after the review process has been satisfactorily completed and all authors 

have approved the revised draft.

CONfIDENTIALITy AND ANONyMITy

To encourage reviewers to express their views freely, the 

review comments are treated as confidential documents 

and are given to the authors of the report with identifiers 

removed. After submitting their comments, reviewers are 

asked to return or destroy the draft manuscript and to 

refrain from disclosing their comments or the contents of 

the draft. The names and affiliations of participants in 

the review will be made public when the report is released (usually by acknowledgment in the printed report), but their comments remain confidential. Even after 

release of the report, reviewers should not divulge their 

comments or any changes made to the draft manuscript. 

These restrictions are imperative in safeguarding the integrity of the institutional review process.

SUPPORTINg EvIDENCE

The rationale for any findings, conclusions, and recommendations should be fully explained in the report. This 

explanation might include references to the literature, 

analysis of data, or a description of the pros and cons 

of the range of alternatives and the reasons for preferring a particular option. Failure to document conclusions 

and recommendations adequately is the most common 

shortcoming of draft reports. Of particular concern are 

recommendations calling for organizational changes or 

budgetary increases within government agencies or for 

additional work for the National Academies. In general, 

such recommendations should be avoided unless specifically called for in the study charge.

SUMMARIES AND ExECUTIvE SUMMARIES

Every study report must contain a Summary or an Executive Summary that clearly and concisely communicates 

the main message of the report to its intended audiences, 

including non-experts and readers who do not read the rest 

of the report. The Summary or Executive Summary may not 

distort or go beyond the content of the report. It must clearly 

identify the study charge, but need not include all conclusions and recommendations. An Executive Summary should 

typically be less than 800 words and not exceed 1,000 

words. It may appear alone or be followed by a Summary. A 

Summary should typically be less than 3,000 words and not 

exceed 5,000 words unless preceded by an Executive Summary (which may be called an “abstract” or “synopsis” when 

it precedes a Summary). 

CONSENSUS AND DISSENT

National Academies committees strive for consensus, but on 

rare occasion--despite extensive deliberations--one or more 

committee members may not concur with the views of the 

majority. Matters of disagreement should be addressed 

forthrightly in the report. As a final recourse, a committee 

member may choose to prepare a brief dissent (no more 

than 5,000 words) succinctly describing the issues of contention and the arguments in support of the minority view. This 

statement should be included as an appendix to the draft 

report, with reference to it in the introductory text and Table 

of Contents. A dissent may not address issues outside the 

study charge, misrepresent the majority’s views, or contain 

other inaccuracies. Any questions regarding the appropriateness of material included in a dissent shall be referred to 

the RRC chair. Although reviewers’ comments on the statement are given to its author for consideration, no formal 

written response is required.

141
REvIEw CRITERIA

National Academies reports cover a broad range of topics 

and appear in a variety of different forms. Although no 

rigid set of criteria is likely to be applicable to all reports, 

reviewers may find the following questions useful in formulating their comments. (A separate set of criteria is used for 

“letter reports” and other abbreviated documents.)

1. Is the charge clearly described in the report? 

 Are all aspects of the charge fully ad- 

 dressed? Do the authors go beyond their 

 charge or their expertise?

2. Are the conclusions and recommendations 

 adequately supported by evidence,analysis, 

 and argument? Are uncertainties or 

 incompleteness in the evidence explicitly 

 recognized? If any recommendations are 

 based on value judgments or the collective 

 opinions of the authors, is this acknowledged and 

 are adequate reasons given for reaching those 

 judgments? If the report is based on a sworkshop, 

 are findings and conclusions attributed to either an 

 individual or an NRC committee?

3. Are the data and analyses handled competently? 

 Are statistical methods applied appropriately?

4. Are sensitive policy issues treated with proper 

 care? For example, if the report contains 

 recommendations pertaining to the reorganization 

 of an agency or the creation of a new institutional 

 entity, are the advantages and disadvantages 

 of alternative options, including the status quo, 

 considered? consistent with other sections of 

 the report?

5. Are the report’s exposition and organization 

 effective? Is the title appropriate?

6. Is the report fair? Is its tone impartial and 

 devoid of special pleading?

7. Does the executive summary concisely and 

 accurately describe the key findings and 

 recommendations? Is it consistent with other 

 sections of the report?

8. Are signed papers or appendices, if any, 

 relevant to the charge? If the report relies on 

 signed papers to support consensus findings 

 or recommendations, do the papers meet 

 criterion 3 above?

9. What other significant improvements, if any, 

 might be made in the report?

In providing comments, reviewers are encouraged to distinguish issues they consider to be of general/major concern 

from other, less significant points.

PURPOSE

The purpose of review is to assist the authors in making 

their report as accurate and effective as possible and to 

ensure that they and the National Academies are creditably represented by the report published in both their 

names. Review not only fulfills the institutional obligation to 

exercise oversight, but also provides the authors with preliminary reactions from a diverse group of experts and, as 

a result, enhances the clarity, cogency, and credibility of 

the final document. Reviewers are asked to consider 

whether in their judgment the evidence and arguments 

presented are sound and the report is fully responsive to 

the study charge, not whether they concur with the findings.

PROCESS

The report review process is overseen by the Report 

Review Committee (RRC), made up of approximately 

30 members of the National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. The process is managed by the commission, 

board, or office responsible for institutional oversight of 

the project. This administrative unit, in consultation with 

the RRC, appoints a group of independent reviewers 

with diverse perspectives on key issues considered in 

the report. A draft report is sent to reviewers only after 

all authors have indicated that they are satisfied with its 

form and content. Reviewers receive the complete report (including front matter, preface, executive summary, and all appendices), along with the statement of 

task and this brochure. Reviewers are asked to provide 

written comments on any and all aspects of the draft 

report, but to pay particular attention to the review criteria set forth in the final section of this brochure. The authors are expected to consider all review comments and 

to provide written responses, which are evaluated by 

the monitor (appointed by the RRC) and/or review coordinator (appointed by the administrative unit). A report 

GUiDeLiNeS FoR tHe Review oF

NATIONAL ACADEMIES REPORTS

Any National Academies report (including meeting summaries, signed papers, letter reports, or other study products) 

must be reviewed by a diverse group of experts other than its 

authors before it may be released outside the institution. 

This independent, rigorous review is a hallmark that distinguishes the National Academies from many other organizations offering scientific and technical advice on issues of 

national importance.

THE NATIONAL

ACADEMIES

142