# Appendix C NRC Report Development Procedures

1) The National Academies: Our Study Process

– Ensuring Independent, Objective Advice

2) The National Academies: Getting to Know the Committee Process

3) Report Review Committee Guidelines for Review: C onsensus Reports

118 119

121

141sub mission of information by outside parties; 3) reviews of the scientific literature, and 4) the investigations of the committee members and staff. In all cases, efforts are made to solicit input from individuals who have been directly involved in, or who have special knowledge of, the problem under consideration.

In accordance with federal law and with few exceptions, information-gathering meetings of the committee are opento the public, and any written materials provided to the committee by individuals who are not officials, agents, or employees of the National Academies are maintained in a public access file that is available for examination.

The committee deliberates in meetings closed to the public in order to develop draft findings and recommendations free from outside influences. The public is provided with brief summaries of these meetings that include the list of committee members present. All analyses and drafts of the

r eport remain confidential.


### STAGE 4. Report Review

As a final check on the quality and objectivity of the study, all National Academies reports—whether products of studies, summaries of workshop proceedings, or otherdocuments—must undergo a rigorous, independent external review by experts whose comments are provided anonymously to the committee members. The National Academies recruit independent experts with a range of views and perspectives to review and comment on the draft

r eport prepared by the committee.

The review process is structured to ensure that each report addresses its approved study charge and does not go beyond it, that the findings are supported by the scientific evidenceand arguments presented, that the exposition and organization are effective, and that the report is impartial and objective.

Each committee must respond to, but need not agree with, reviewer comments in a detailed “response to review” that is examined by one or two independent report review “monitors” responsible for ensuring that the report review criteria have been satisfied. After all committee members and appropriate National Academies officials have signed off on the final report, it is transmitted to the sponsor of the study and is released to the public. Sponsors are not given an opportunity to suggest changes in reports. The names and affiliations of the report reviewers are made public when the report is released. 

### STAGE 3. Committee Meetings, Information Gathering, Deliberations, and Drafting the Report
Study committees typically gather information through: 1) meetings that are open to the public and that are announced in advance through the National Academies Web site; 2) the 

## HOW THE PUBLIC CAN FOLLOW AND PROVIDE INPUT TO STUDIES

The Current Projects System was establishedwith a link from the National Academies homepage, www.national-academies.org , to make it easy for members of the general public with interest in the subject to follow the progress of a study and submit comments. The system offers separate views by subject and by project title.

Reports of the National Academies are available from the National Academies Press, 500 Fifth Street, NW, Washington, DC 20001 1-800-624-6242 • www.nap.edu.interest, no individual can be appointed to serve (or continue to serve) on a committee of the institution used in the development of reports if the individual has a conflict of interest that is relevant to the functions to be performed. For more information, see the National Academies’ Web site at www.national-academies.org.

Other considerations. Membership in the three Academies (NAS, NAE, IOM) and previous involvement in National Academies studies are taken into account in committeeselection. The inclusion of women, minorities, and young professionals are additional considerations.

Specific steps in the committee selection and approval process are as follows:

 Staff solicit an extensive number of suggestions for potential committee members from a wide range of sources, then recommend a slate of nominees.

 Nominees are reviewed and approved at several levels within the National Academies; a provisional slate is then approved by the President of the National Academy of Sciences, who is also the Chair of the National Research Council.

 The provisional committee list is posted for public comment in the Current Projects System on the Web (http://www4.national-academies.org/cp.nsf).

 The provisional committee members complete background information and conflict of interest disclosure forms.

 The committee balance and conflict of interest discussion is held at the first committee meeting.

 Any conflicts of interest or issues of committee balance and expertise are investigated; changes to the committee are proposed and finalized.

 Committee is formally approved.

 Committee members continue to be screened for conflict of interest throughout the life of the committee. 

## Ensuring Independent, Objective Advice 

# OUR STUDY PROCESS



# NATIONALACADEMIESTHE



### National Academy of Sciences National Academy of Engineering Institute of Medicine National Research Council

119Project starts when funding is received Provisional slate approved by NAS President Provisional committee posted for public comment via Current Projects System Committee balance and expertise evaluated; any conflicts of interest are investigated Final committee formally approved Monitoring for potential conflicts of interest continues 

## STEPS TAKEN TO ENSURE INDEPENDENCE AND OBJECTIVITY

The reports of the National Academies are viewed as beingvaluable and credible because of the institution’s reputation for providing independent, objective, and non-partisanadvice with high standards of scientific and technical quality. Checks and balances are applied at every step in the study process to protect the integrity of the reports and to maintain public confidence in them. The study process can be broken down into four major stages: 1) defining the study; 2) committee selection and approval; 3) committee meetings, information gathering, deliberations, and drafting of the report; and 4) report review. 

### STAGE 1. Defining the Study

Before the committee selection process begins, National Academies’ staff and members of their boards work with sponsors to determine the specific set of questions to be addressed by the study in a formal “statement of task,” as well as the duration and cost of the study. The statement of task defines and bounds the scope of the study, and it serves as the basis for determining the expertise and the balance of perspectives needed on the committee.

The statement of task, work plan, and budget must be approved by the Executive Committee of the National Research Council Governing Board. This review often results in changes to the proposed task and work plan. On occasion, it results in turning down studies that the institution believes are inappropriately framed or not within its purview. 

### STAGE 2. Committee Selection and Approval
Selection of appropriate committee members, individually and collectively, is essential for the success of a study. All committee members serve as individual experts, not as

r epresentatives of organizations or interest groups. Each member is expected to contribute to the project on the basis of his or her own expertise and good judgment. A committee is not finally approved until a thorough balanceand conflict of interest discussion is held at the first meeting, and any issues raised in that discussion or by the public are investigated and addressed.

Careful steps are taken to convene committees that meet the following criteria:
An appropriate range of expertise for the task. The committee must include experts with the specific expertise and experience needed to address the study’s statement of task. One of the strengths of the National Academies is the tradition of bringing together recognized experts fromdiverse disciplines and backgrounds who might not otherwise collaborate. These diverse groups are encouraged to conceive new ways of thinking about a problem.

A balance of perspectives . Having the right expertise is not sufficient for success. It is also essential to evaluate the overall composition of the committee in terms of different experiences and perspectives. The goal is to ensure that the relevant points of view are, in the National Academies’ judgment, reasonably balanced so that the committee can carry out its charge objectively and credibly. 

## POINT OF VIEW IS DIFFERENT FROM CONFLICT OF INTEREST

A point of view or bias is not necessarily a conflict of interest. Committee members are expected to have points of view, and the National Academies attempt to balance these points of view in a way deemed appropriate for the task. Committee members are asked to consider respectfully the viewpoints of other members, to reflect their own views rather than be a representative of any organization, and to base their scientific findingsand conclusions on the evidence. Each committee member has the right to issue a dissenting opinion to the report if he or she disagrees with the consensus of the other members.

NRC Governing Board reviews and approves study scope and plan Committee’s first meeting Report is released to the sponsor and the public Committee and National Academies sign off on report

Full committee signs off on draft report Reviewers comment on report

DEFINING THE STUDY COMMITTEE SELECTION AND APPROVAL

COMMITTEE MEETINGS, INFORMATION GATHERING, DELIBERATIONS, AND DRAFTING REPORT REPORT REVIEW 

# → Screened for conflicts of interest. All provisional committee members are screened in writing and in a confidential group discussion about possible conflicts of interest. For this purpose, a “conflict of interest” means any financial or other interest which conflicts with the service of the individual because it could significantly impair theindividual's objectivity or could create an unfair competitive advantage for any person or organization. The term“conflict of interest” means something more than individual bias. There must be an interest, ordinarily financial, that could be directly affected by the work of the committee. Except for those rare situations in which the NationalAcademies determine that a conflict of interest is unavoidable and promptly and publicly disclose the conflict of

STAGE 1 STAGE 2 STAGE 4 STAGE 3 

# F or more than 140 years, the National Academies have been advising the nation on issues of science, technology, and medicine. The 1863 Congressional charter signed by President Lincoln authorized this non-governmental institution to honor top scientists with membership and to serve the nation whenever called upon. Today the National Academies— National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council

— continue that dual mission.

Like no other organization, the National Academies canenlist the nation’s foremost scientists, engineers, health professionals, and other experts to address the scientific and technical aspects of society’s most pressing problems. Each year, more than 6,000 of these experts are selected to serve on hundreds of study committees that are convened to answer specific sets of questions. All serve without pay.

Federal agencies are the primary financial sponsors of the Academies’ work. Additional studies are funded by state agencies, foundations, other private sponsors, and the National Academies endowment. The Academies provide independent advice; the external sponsors have no control over the conduct of a study once the statement of task and budget are finalized. Study committees gather information from many sources in public meetings but they carry out their deliberations in private in order to avoid political, special interest, and sponsor influence.

Through this careful study process, the National Academies produce 200–300 authoritative reports each year. Recent

r eports cover such topics as the obesity epidemic, the use of forensics in the courtroom, invasive plants, underage drinking, the Hubble Telescope, vaccine safety, the hydrogeneconomy, transportation safety, climate change, and homeland security. Many reports influence policy decisions; some are instrumental in enabling new research programs; others provide program reviews.

120

# GETTING TO KNOW THE COMMITTEE PROCESS

National Academy of Sciences National Academy of Engineering Institute of Medicine National Research Council 

# NATIONALTHEACADEMIES

121

# The National Academy of Sciences was established by Congress more than acentury ago to provide scientific and technological advice to the nation. Over the years, the Academy has evolved to incorporatefour distinguished organizations the National A cademy of Sciences, the National Academy of Engineering, the Institute Medicine, and the National Research Council. Known collectively as the National Academies, they perform an unparalleled public service by bringing together experts in all areas of scientific and technological endeavor. These experts serve as volunteers to addresscritical national issues and give unbiased advice to the federal government and the public. Most of this advice is provided either by the National Research Council the chief operating arm of the A cademy of Sciences and the Academy of Engineering or by the Institute of Medicine, which operates under the charter of the National Academy of Sciences and according to ResearchCouncil rules.The National Academies provide science and technology advice in several different forms: written reports reflecting theconsensus reached by an expert study committee; symposia and convocations engaging large audiences in discussion of national issues; proceedings from conferences and workshops; or “white papers” on The National Academy of Sciences was established by Congress more than a century ago to provide scientific and technological adviceto the nation. Over the years, the Academy has evolved to incorporate four distinguished organizations the National Academy of Sciences, the National Academy of Engineering, the Institute of by USEFUL REFERENCES ABOUT THIS UNIQUE INSTITUTION 2

THE STUDY PROCESS 5

Committee Selection 6

Bias and Conflict of Interest 6

Committee Deliberations 7

Studies of Risk Assessment 8
Institutional Oversight 8

W riting the Report 9

Role of Staff 9
Report Review 10
Confidentiality 10

Public Access 12

DISSEMINATION 14
Congressional Outreach 14

Academies’ News Office 15

National Academies Press 15

Communications Initiative 16

Academies’ Web Site 16

USEFUL REFERENCES 1 7CONTENTS Guidelines for Review: Consensus Reports

The National Academies Study Process: Ensuring Independent, Objective Advice

Roles of the Committee Chair

A Unique National Resource

Revised 2005

Copies of this booklet are available from the Office of News and Public Information

T

el.: 202-334-2138 E-mail: news@nas.edu
Fax: 202-334-2158 Internet: national-academies.org

122

# AC ADEMIESASKED TO SERVE . . .THE NATIONAL 

# Y SO YOU'VE BEEN

ou have been invited to

work on a project at the National

Academies and may be wondering

exactly what your role is as a committee member. This document is a brief

introduction to the institution and is

designed to give you a sense of the

committee process. No two projects

are alike, of course, and people’s

experiences vary. But general policies

and procedures have been developed

to ensure that the time you spend as a

volunteer in service to the nation is

productive and rewarding.

123

# AC ADEMIESTHE NATIONAL 

### 2

The National Academy of Sciences was established by Congress more than a century ago to provide scientific and technological advice to the nation. Over the years, the Academy has evolved to incorporate four distinguished organizations — the National Academy of Sciences, the National Academy of Engineering, the Institute of Medicine, and the National Research Council. Known collectively as the National Academies, they perform an unparalleled public service by bringing together experts in all areas of scientific and technological endeavor. These experts serve as volunteers to address critical national issues and give unbiased advice to the federal government and the public. Most of this advice is provided either by the National Research Council — the chief operating arm of the Academy of Sciences and the Academy of Engineering — or by the Institute of Medicine, which operates under the charter of the National Academy of Sciences and according to Research Council rules. 

## ABOUT THIS UNIQUE INSTITUTIONJD TALASEK

124The National Academies provide science and technology advice in several different forms: written reports reflecting the consensus reached by an expert study committee; symposia andconvocations engaging large audiences in discussion of national issues; proceedings from conferences and workshops; or “white papers” on policy issues of special interest. Each project is conducted or overseen by a committee serving pro bono, whose members are selected for their expertise on the subject.

This booklet focuses primarily on the study committee process, in which a committee of approximately 10 to 20 members with a diverse range of expertise and perspectives is convened to address a particular question or set of questions. Study committees have been assembled, for example, to address various aspects of the AIDS crisis, to consider the use of forensics techniques in the courts, and to offer policy advice on setting priorities for federal funding of research and development. Before a committee meets, one of the volunteer experts is appointed to serve as the committee chair, and a member of the Academies’ staff is assigned as the study director.

Each committee investigates the many facets of the problem described in its statement of task and develops a report of its findings, conclusions, and recommendations, based on the available scientific evidence. This report is subjected to rigorous review by a second groupof independent experts anonymous to the committee. After revisions are made by the committee to satisfy the institution’s review process, the report is transmitted to the agency or organization that sponsored the study, and then released to the public. Names and affiliations of principal reviewers are made public when the report is released.

The National Academies and their operations are distinctive in several ways:

• Committee members serve without compensation, except for reimbursement of expenses.

• Most projects originate from requests made by outside sponsors such as Congress, federal agencies, and foundations. However, volunteer members of the institution’s boards also develop ideas for studies that subsequently are funded by external sources. In addition, the Academies have been building their own endowments through private contributions so that they can address important issues of science and policy without government or foundation support. 

### 3

125

# AC ADEMIESTHE NATIONAL 

### 4 • The institution has no research laboratories. In reaching conclusions and recommendations, study committees hear invited testimony and evaluate published research from the

r elevant scientific literature, as part of an educational process designed to enable the committee to reach consensus. The resulting reports often present a novel synthesis of ideas unique to the committee; these reports are highly regarded by the sponsoringagencies and the public for their thorough analysis and carefully supported recommendations. The reputation of the institution for objectivity, integrity, independence, and competence is one of its most valuable assets. For this reason, procedures designed to ensure excellence apply to each study undertaken.

• The National Academies do not compete in response to federal requests for proposals. Their one-of-a-kind service, not duplicated by other organizations, was reaffirmed in a January 1993 executive order from President Bush.

• The Academies are not subject to the Federal Advisory Committee Act in the way government agencies are; neither are they subject to the Freedom of Information Act. However, the Federal Advisory Committee Act Amendments of 1997 (Public Law 105-153) require the Academies to provide opportunities for public access and involvement in the study process. Those requirements are discussed in the Public Access section of this booklet.

126

### 5 The core of the institution’s work consists of studies usually of six months to two years in duration and performed under contract to a sponsor or set of sponsors. Each study is conducted by a committee selected expressly for that purpose. The committee meets at intervals to consider its scope of work, to review the relevantscientific evidence, and to develop its findings, conclusions, and recommendations. Once the report has been outlined by the committee, various sections often are written by individual members. The writing process is guided by the committee chair and aided by the committee staff, in particular by the study director.

The charge to the committee — developed before committee members are selected — is the formal statement of the questions to be addressed by the study. This statement defines the study’s scope and issues to be examined. If a committee finds in the course of its work that this description is inadequate, the charge can be formally modified through petition to the Executive Committee of the National Research Council’s Governing Board. Such petitions are carefully examined; a committee that is well-balanced for one purpose may not be appropriately constructed for a modified task. 

## THE STUDY PROCESSJD TALASEK

127

# AC ADEMIES COMMITTEE SELECTIONThe search for candidates for committee membership is initiated by staff with input and oversight from the relevant boards. In defining the areas of expertise that should be represented on a committee and identifying individuals qualified to serve, the staff reviews scholarly literature and consults widely with the institution’s members and volunteers, knowledgeable authorities, and professional associations. Sponsors may offer suggestions but do not select committee members.

Committee members are chosen on the basis of their knowledge and experience in the various aspects of the topic to be investigated, and after careful review are appointed by the chair of the National Research Council, who also is the president of the National Academy of Sciences. The names, affiliations, and short biographies of committee members are posted for public comment in the “Current Projects” area of the institution’s Web site at <national-academies.org>.



### BIAS AND CONFLICT OF INTEREST Committee members serve as individuals, not as representatives of organizations or interest groups. Each person is selected on the basis of his or her expertise and good judgment, and is expected to contribute accordingly to the study.

The credibility of a report can be called into question if the committee that produced it is perceived to be biased. Potential sources of bias and conflict of interest are significant issues thatare taken into consideration in the selection of committee members and are re-examined periodically throughout the study process.

Recognizing that each individual who is knowledgeable about a subject brings his or her own biases and experiences to any study effort, the institution has adopted specific procedures to achieve appropriate balance in the committee membership and to avoid conflicts of interest. At the time of appointment, each committee member is required to list all professional, consulting, and financial connections, as well as to describe pertinent intellectual positions and public statements by filling out a confidential form, “Background Information and Confidential Conflictof Interest Disclosure.” The committee appointment is not finalized until the institution completes a review of information regarding potential conflicts of interest and bias.THE NATIONAL 

### 6

128As part of the process of becoming acquainted with each other and with the task before them, committee members discuss this information in closed session at the beginning of their first meeting and annually thereafter. The information also is reviewed by officials of the institution, and if a potential conflict becomes apparent, the committee member may be asked to resign. In exceptional circumstances, an individual may continue to serve on the committee if theconflict of interest is promptly and publicly disclosed, and the National Academies have determined that the conflict is unavoidable. To fulfill its legal requirement for such public disclosure, the institution posts on its Web site a brief statement describing the unavoidable conflict. When a question of balance arises, the usual procedure is to add members to the committee to achieve the appropriate balance.



### COMMITTEE DELIBERATIONSA successful report is the result of a dynamic group process, requiring that committee members be open to new ideas and innovative solutions, and be willing to learn from one another.

Committees are expected to be evenhanded and to examine all evidence dispassionately. Although all interested parties should be heard and their views given serious and respectful consideration, one of the committee’s primary roles is to separate fact from opinion, analysis from advocacy. Scientific standards are essential in evaluating all arguments and alternatives.

Most committees eventually issue a unanimous report of their conclusions. Members of committees should strive for consensus, but not at the cost of substantially weakening their analyses and conclusions. It may be more valuable in the long run to explain the rationale behind areas of disagreement than to issue unanimous conclusions that are so limited that they fail to contribute to a better understanding of the issue. 

### 7PAUL KENNEDY

129

# AC ADEMIES STUDIES OF RISK ASSESSMENT Studies involving the assessment of risk are among the most difficult and potentially controversial. Government agencies and others engaged in the management of hazardous conditions and substances often are faced with making policy decisions in the absence of conclusive scientific evidence. Many Research Council and Institute of Medicine studies dealing with risk are requested by federal agencies seeking specific answers on which to base important policy decisions. In studies involving incomplete or inconclusive data, the collective experience and reasoned judgment of committee members become crucial factors in developing conclusions and recommendations.

It is critical for each committee working on a study of risk assessment to distinguish clearly between conclusions based on scientific evidence and those based on informed judgment. In cases in which scientific proof is incomplete, special care is needed to explain how the committee arrived at its conclusions. Moreover, the assumptions used should be explicitly identified and justified. A conscientious effort to be clear in writing the report will help to avoid the potential for misinterpretation when the report is published.

The institution has developed guidelines for committees charged with conductingrisk-related projects (available on the Academies’ intranet site, the AcademyNet). These guidelines emphasize the special care that must be taken in assembling the committee, orienting new committee members, conducting bias discussions, managing the consensus process, handling any minority opinions, and writing the report.



### INSTITUTIONAL OVERSIGHT Every study is subject to oversight, from initial approval to public release of a final report.Whether a study is requested by a government agency or a private organization, or is initiated within the institution itself, the study proposal first must be approved by the Executive Committee of the Research Council’s Governing Board. This group carefully examines the proposal and considers such factors as the importance and timeliness of the question,whether there is an adequate base of scientific knowledge to support the study, the intended audience, the likely impact of the report, and the competence of the institution to take on the task.

Oversight is provided throughout the duration of a study by the various supervisory entities within the Research Council and the Institute of Medicine, as well as by the committee chair and the staff member serving as study director. Their responsibilities are to ensure that theTHE NATIONAL 

### 8

130committee focuses on its stated tasks, that measures of quality control are enforced, and that the study proceeds on schedule and within budget. Sponsors do not engage in oversight of the studies.



### WRITING THE REPORTCompleting the consensus-building process and writing a report that clearly presents the committee’s findings, conclusions, and recommendations may be the most challenging, yet rewarding aspects of serving on a study committee. The report may well become an important reference for those who formulate public policy. For this reason, the value of a carefully prepared

r eport cannot be overstated.

Although each committee may go about the drafting of its report differently, every report is thecollective product of a group process. A committee member often will draft a chapter or portion of the report, but the “author of record” is the entire committee, and the responsibility for authorship lies with the committee as well. Individual authorship generally is not credited; the

r eport and all copyrights become the property of the National Academy of Sciences.



### ROLE OF STAFF Each committee is assisted in its work by highly qualified staff members who facilitate the work ofthe committee during the conduct of the study. When committee and staff form a close professional partnership, the experi ence can be exhilarating for everyone involved.

Staff help to create the objective atmosphere in which the committee’s deliberations take place. In addition, staff are responsible for ensuring that institutional procedures and practices are followed throughout the study, and that the study stays on schedule and within budget.

Staff members assist with many aspects of assembling the report, including researching, writing, integrating portions written by others, and ensuring consistent style and format. However, the conclusions and recommendations are those of the committee. Staff do not insert their personal conclusions or recommendations into the report. 

### 9

131

# AC ADEMIESTHE NATIONAL 

### 10 REPORT REVIEWLike all good science, reports should be based on fact and rigorous analysis. All of the institution’s reports — whether products of studies, summaries of workshop proceedings, or abbreviated documents — must undergo an independent review by anonymous experts who were not involved in the report’s preparation.

Report review is an integral and constructive part of the study process. It is the final opportunity for committees to test their reasoning, conclusions, and recommendations before release of the

r eport to the public.

The Report Review Committee (RRC) ensures that an independent review has been conducted, and that:

• the report addresses the approved study charge and does not go beyond it; • the findings are supported by the evidence and arguments presented; • the exposition and organization are effective; and • the tone of the report is impartial, and sensitive policy issues are treated with appropriate care.

The report may not be transmitted to the sponsor or released to the public until review has been completed to the satisfaction of the Report Review Committee. Details of the quality standards followed by reviewers can be found in the RRC document Guidelines for Review: Consensus Reports . Once the report is released, names and affiliations of principal reviewers are made public.



### CONFIDENTIALITY During more than a century of service, the institution has earned a reputation for providing independent, expert advice. Procedures and practices have evolved that protect committees from outside pressures and thereby safeguard the credibility and integrity of their work.

Committee meetings, particularly as the committee gathers information, are usually open to interested individuals and the news media. However, meetings are closed when the committee is deliberating to develop its findings and during discussion of financial and personnel matters. Closed meetings are not open to the public or to any person who is not a committee member or an official, agent, or employee of the Academies.

132Reports are the product of the institution, not of the committee alone. Committee deliberations, drafts of the report in progress, and tentative conclusions all are confidential until a completed

r eport passes through review and receives sign-off by the Report Review Committee and by the major unit responsible for the study. Committee members are expected to reject any

r equests for early briefings or media interviews on the committee’s findings, and to treat committee deliberations and draft products as confidential.

A cardinal rule to keep in mind: Until the review process has been satisfactorily completed,the document is not an official report of the National Academies. Conclusions and recommendations can change up to the final sign-off; premature briefings for sponsors or others outside the institution may lock committees into a position not fully supported by the evidence. Early briefings also damage the final report by subjecting the committee to the accusation that it permitted the sponsor to preview and approve the conclusions and recommendations — a serious charge that undermines the independence and integrity of both the committee and the institution. In such cases, the hard work of the committee can be discredited, diminishing the report’s value to the sponsor and to the nation.

Until the report is publicly released, committee members should limit public comment to the following:

• the scope of the project and what the committee has been asked to do; • the name of the sponsor and estimated cost of the study, if known; and • the makeup of the committee, including names and affiliations of committee members. 

### 11GREG HADLEY

133

# AC ADEMIES This and other relevant information can be found on the institution’s Web site at <national-academies.org> and in its public access files. These files, which include items such as materials presented to the committee in data-gathering meetings open to the public, are available for public review.



### PUBLIC ACCESS Legislation passed in late 1997 protects the Academies from government control under the Federal Advisory Committee Act. But in doing so, it explicitly requires the Academies to ensure public access to committee activities.

For example, any meeting of a committee at which anyone other than committee members or officials, agents, or employees of the institution is present — whether in person, by telephone, or teleconference— is considered a “data-gathering committee meeting.” Except as determined and approved in advance by the Academies’ leadership, all data-gatheringcommittee meetings are open to the public. Datagathering meetings that involve committee consideration of classified, proprietary, or personal-privacy information, however, exemplify situations in which meetings are not open to the public.

To facilitate the process of informing the public abouta committee’s work and enabling interested individuals to attend open data-gathering sessions, anadvance announcement must be posted — preferably 14 days before the meeting — on the institution’s Web site.

Data-gathering committee meetings should be

r egarded as on the record. Therefore, whether or notTHE NATIONAL 

### 12

134r epresentatives of the news media are in attendance, the chair of the meeting should advise everyone present of the nature and purpose of the meeting. Statements of this type are necessary to help ensure that participants and observers do notmisinterpret the purpose of the meeting, or prematurely interpret the discussion to be the positions ofindividual participants, the committee, or the institution. For guidance on assembling introductory

r emarks for the chair, see “Setting the Ground Rules at an Open Meeting to Which the Public and Press Have Been Invited as Observers,” available on the AcademyNet.

To acquaint the public with the background of committee members, the chair should ask each member to state briefly, in open session at the first data-gathering committee meeting, those aspects of his or her background, experience, expertise, and previously stated positions that appear relevant to the functions to be performed by the committee. Committees also should create opportunities that facilitate the gathering of as wide a range of views as possible, such as having a session for public comments at a data-gathering committee meeting or soliciting comments in writing or via e-mail from interested members of the public.

W ithin 10 days following a closed committee meeting, the Academies will post a brief summary of the meeting on the Web, listing the committee members present, the topics discussed, and materials made available to the committee. This summary will not disclose the substantive content, conclusions,

r ecommendations, discussion of draft reports, or any report r eview comments. 

### 13JD TALASEK

135

# AC ADEMIESTHE NATIONAL 

### 14

Once a report has successfully completed review and been thoroughly edited, Academies staff working with the Office of Congressional and Government Affairs and the Office of News and Public Information develop a release timeline. At this stage it is critical to observe carefully the institution’s rules of confidentiality until the report is ready for release to the public.



### CONGRESSIONAL OUTREACH The Office of Congressional and Government Affairs (OCGA) is responsible for dissemination and outreach to the congressional and executive branches of government. One component of this is congressional and sponsor briefings. These briefings take place up to 24 hoursbefore the public release of a report. Dissemination of reports that are congressionally mandated involves specific guidelines for briefing members of Congress. For other reports, OCGA informs concerned members of Congress and the appropriate congressional committees.

OCGA also monitors congressional activities on issues that affect the National Academies and helps committee members or staff prepare and submit testimony for Congress. 

## DISSEMINATION

136

### AC ADEMIES’ NEWS OFFICE The Office of News and Public Information (ONPI) is the liaison between the Academies and the news media and general public. Many reports of the National Academies are newsworthy, and the media serve as an important channel for disseminating the content of reports.

For each report, ONPI works closely with study staff to develop a release plan that may include a news release; public briefing; embargo arrangement; outreach to targeted media; and mediapreparation for committee members to help them speak comfortably with journalists and handle potentially difficult inquiries.

Committee members or staff who receive a call from a reporter concerning a pending report generally should feel free to respond, but in a limited way. Journalists may be provided with a project’s statement of task, the committee roster, the identity of the sponsor(s), and the cost of the study, as well as general information about the background and scope of the project. Findings and recommendations, however, must be kept confidential until the report is released.

ONPI should be informed of substantive conversations with the news media, especially if there is any problem. Staff or committee members seeking guidance about media relations may consult with ONPI at any time. If a reporter is asking leading questions or inquiring about controversial issues, it often is best to delay answering until after conferring with ONPI.

ONPI also publicizes Academies reports through weekly and monthly electronic newsletters and the institution’s magazine, The National Academies In Focus

.



### NATIONAL ACADEMIES PRESS The National Academies Press (NAP) is the publisher for the institution. NAP offers a wide range of services,including publication planning, editing, printing, marketing, and distribution. It publishes nearly 200 committee

r eports each year, ranging from pre-publication photocopies to full-color, high-quality trade books that are marketed and sold around the world. The NAP Web site at <www.nap.edu> makes all of these publications available online — more than 3,000 books and 500,000 book pages in 2005. Most books published by NAP are also offered as PDF files, many of which are free to the public. 

### 15

137Current committee members are permitted free access to all available PDF files and should contact study staff to make the necessary arrangements; they also receive a 25 percent discount on all books purchased from NAP for personal use.



### COMMUNICATIONS INITIATIVEThe National Academies strive to disseminate the results of their work to a wide range of audiences. The Office of Communications was formed in 2001 to help the institution accomplish this. Specifically, this office works in partnership with Academies program units to engage the public with the institution’s work, create new products and services for different audiences, build stronger relationships with existing and potential sponsors, and harness the Web as an effective communication vehicle. Committees are encouraged to identify potential audiences for their studies early in the process, discuss how to reach those audiences, and consider ways of expanding public interest throughout the study process.



### AC ADEMIES’ WEB SITE Information about the institution’s work is available on the Web at <national-academies.org>. A broad range of material can be found online, including current project information, news

re leases, and full-text reports.



# AC ADEMIESTHE NATIONAL 

### 16

138

# The National Academy of Sciences was established by Congress more than acentury ago to provide scientific and technological advice to the nation. Over the years, the Academy has evolved to incorporatefour distinguished organizations the National A cademy of Sciences, the National Academy of Engineering, the Institute Medicine, and the National Research Council. Known collectively as the National Academies, they perform an unparalleled public service by bringing together experts in all areas of scientific and technological endeavor. These experts serve as volunteers to addresscritical national issues and give unbiased advice to the federal government and the public. Most of this advice is provided either by the National Research Council the chief operating arm of the A cademy of Sciences and the Academy of Engineering or by the Institute of Medicine, which operates under the charter of the National Academy of Sciences and according to ResearchCouncil rules.The National Academies provide science and technology advice in several different forms: written reports reflecting theconsensus reached by an expert study committee; symposia and convocations engaging large audiences in discussion of national issues; proceedings from conferences and workshops; or “white papers” on The National Academy of Sciences was established by Congress more than a century ago to provide scientific and technological adviceto the nation. Over the years, the Academy has evolved to incorporate four distinguished organizations the National Academy of Sciences, the National Academy of Engineering, the Institute of by USEFUL REFERENCES ABOUT THIS UNIQUE INSTITUTION 2

THE STUDY PROCESS 5

Committee Selection 6

Bias and Conflict of Interest 6

Committee Deliberations 7

Studies of Risk Assessment 8
Institutional Oversight 8

W riting the Report 9

Role of Staff 9
Report Review 10
Confidentiality 10

Public Access 12

DISSEMINATION 14
Congressional Outreach 14

Academies’ News Office 15

National Academies Press 15

Communications Initiative 16

Academies’ Web Site 16

USEFUL REFERENCES 1 7CONTENTS Guidelines for Review: Consensus Reports

The National Academies Study Process: Ensuring Independent, Objective Advice

Roles of the Committee Chair

A Unique National Resource

Revised 2005

Copies of this booklet are available from the Office of News and Public Information

T

el.: 202-334-2138 E-mail: news@nas.edu
Fax: 202-334-2158 Internet: national-academies.org

139

# GETTING TO KNOW THE COMMITTEE PROCESS

National Academy of Sciences National Academy of Engineering Institute of Medicine National Research Council 

# NATIONALTHEACADEMIES

140

# CONSENSUS REPORTS Repo R t Review Committee 

# GUIDELINES FOR REVIEW 

# NATIONALTHEACADEMIES may not be released to the sponsors or the public, normay its findings be disclosed, until after the review process has been satisfactorily completed and all authors have approved the revised draft.



### C ON f IDENTIALIT y AND A NON y MIT y To encourage reviewers to express their views freely, the review comments are treated as confidential documents and are given to the authors of the report with identifiers removed. After submitting their comments, reviewers are asked to return or destroy the draft manuscript and to refrain from disclosing their comments or the contents of the draft. The names and affiliations of participants inthe review will be made public when the report is released (usually by acknowledgment in the printed report), but their comments remain confidential. Even after release of the report, reviewers should not divulge their comments or any changes made to the draft manuscript.These restrictions are imperative in safeguarding the integrity of the institutional review process.



### S UPPORTIN g Ev IDENCEThe rationale for any findings, conclusions, and recommendations should be fully explained in the report. This explanation might include references to the literature, analysis of data, or a description of the pros and consof the range of alternatives and the reasons for preferring a particular option. Failure to document conclusions and recommendations adequately is the most common shortcoming of draft reports. Of particular concern are recommendations calling for organizational changes or budgetary increases within government agencies or for additional work for the National Academies. In general,such recommendations should be avoided unless specifically called for in the study charge.



### S UMMARIES AND Ex ECUTI v E S UMMARIESEvery study report must contain a Summary or an Executive Summary that clearly and concisely communicates the main message of the report to its intended audiences, including non-experts and readers who do not read the rest of the report. The Summary or Executive Summary may not distort or go beyond the content of the report. It must clearlyidentify the study charge, but need not include all conclusions and recommendations. An Executive Summary should typically be less than 800 words and not exceed 1,000 words. It may appear alone or be followed by a Summary. A Summary should typically be less than 3,000 words and notexceed 5,000 words unless preceded by an Executive Summary (which may be called an “abstract” or “synopsis” when it precedes a Summary).



### C ONSENSUS AND D ISSENT National Academies committees strive for consensus, but on rare occasion--despite extensive deliberations--one or more committee members may not concur with the views of the majority. Matters of disagreement should be addressed forthrightly in the report. As a final recourse, a committee member may choose to prepare a brief dissent (no morethan 5,000 words) succinctly describing the issues of contention and the arguments in support of the minority view. This statement should be included as an appendix to the draft report, with reference to it in the introductory text and Table of Contents. A dissent may not address issues outside the study charge, misrepresent the majority’s views, or containother inaccuracies. Any questions regarding the appropriateness of material included in a dissent shall be referred tothe RRC chair. Although reviewers’ comments on the statement are given to its author for consideration, no formal written response is required. 

### 4 4 3.85

141

### R E v IE w C RITERIA National Academies reports cover a broad range of topics and appear in a variety of different forms. Although no rigid set of criteria is likely to be applicable to all reports,reviewers may find the following questions useful in formulating their comments. (A separate set of criteria is used for “letter reports” and other abbreviated documents.)

1. Is the charge clearly described in the report? Are all aspects of the charge fully ad- dressed? Do the authors go beyond their charge or their expertise?

2. Are the conclusions and recommendations adequately supported by evidence,analysis, and argument? Are uncertainties or incompleteness in the evidence explicitly recognized? If any recommendations are based on value judgments or the collective opinions of the authors, is this acknowledged and are adequate reasons given for reaching those judgments? If the report is based on a sworkshop, are findings and conclus ions attributed to either an individual or an NRC committee? 3. Are the data and analyses handled competently? Are statistical methods applied appropriately?

4. Are sensitive policy issues treated with proper care? For example, if the report contains recommendations pertaining to the reorganization of an agency or the creation of a new institutional entity, are the advantages and disadvantages of alternative options, including the status quo, considered? consistent with other sections of the report?

5. Are the report’s exposition and organization effective? Is the title appropriate?

6. Is the report fair? Is its tone impartial and devoid of special pleading?

7. Does the executive summary concisely and accurately describe the key findings and recommendations? Is it consistent with other sections of the report?

8. Are signed papers or appendices, if any, relevant to the charge? If the report relies on signed papers to support consensus findings or recommendations, do the papers meet criterion 3 above?

9. What other significant improvements, if any, might be made in the report?
In providing comments, reviewers are encouraged to distinguish issues they consider to be of general/major concern from other, less significant points

. 

### P URPOSE The purpose of review is to assist the authors in making their report as accurate and effective as possible and toensure that they and the National Academies are creditably represented by the report published in both their names. Review not only fulfills the institutional obligation toexercise oversight, but also provides the authors with preliminary reactions from a diverse group of experts and, as a result, enhances the clarity, cogency, and credibility of the final document. Reviewers are asked to consider whether in their judgment the evidence and arguments presented are sound and the report is fully responsive tothe study charge, not whether they concur with the findings.



### P RO CESS The report review process is overseen by the Report Review Committee (RRC), made up of approximately30 members of the National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. The process is managed by the commission, board, or office responsible for institutional oversight of the project. This administrative unit, in consultation with the RRC, appoints a group of independent reviewers with diverse perspectives on key issues considered in the report. A draft report is sent to reviewers only after all authors have indicated that they are satisfied with itsform and content. Reviewers receive the complete report (including front matter, preface, executive summary, and all appendices), along with the statement of task and this brochure. Reviewers are asked to provide written comments on any and all aspects of the draftreport, but to pay particular attention to the review criteria set forth in the final section of this brochure. The authors are expected to consider all review comments and to provide written responses, which are evaluated bythe monitor (appointed by the RRC) and/or review coordinator (appointed by the administrative unit). A report 

## GU i D e L i N e S F o R t H e R eview o F 

# NATIONAL ACADEMIES REPORTS
Any National Academies report (including meeting summaries, signed papers, letter reports, or other study products) must be reviewed by a diverse group of experts other than its authors before it may be released outside the institution.This independent, rigorous review is a hallmark that distinguishes the National Academies from many other organizations offering scientific and technical advice on issues of national importance.NATIONAL THE



## ACADEMIES 4 4 3.85

142