Federal Research: The National Academy of Sciences and the Federal Advisory Committee Act
Released: 1998-11-25T00:00:00

Letter

Background

Founded in 1863 by congressional charter, the National Academy of Sciences has a long history of serving as a scientific adviser. The Academy, which has a total membership of 4,800, also serves as an honorary institution to recognize distinguished members of the scientific community. Among other activities, the Academy also organizes symposiums, manages scientific databases, and serves as a clearinghouse for research.

Throughout this report we use “Academy” to refer to the constituent members of the Academy complex: the National Academy of Sciences, the National Academy of Engineering, the Institute of Medicine, and the National Research Council. In 1916, the Academy formed the National Research Council to broaden its committee membership to include non-Academy members and to oversee the Academy’s advisory activities. In a 1998 report, the Academy reported that committee membership consists of 55 percent from academia, 24 percent from industry, 9 percent from nonprofit institutions, and 12 percent from different levels of government. The National Academy of Engineering and the Institute of Medicine were established in 1964 and 1970, respectively, to recognize distinguished members in these fields and to provide more specialized advice in these areas. The Academy is organized by study units, which produce reports in the following topic areas: transportation, health and safety, science, commerce, natural resources, defense, space, education, and international affairs. (See table 1.)

The Academy issued 1,331 committee reports from January 1993 to June 1997 and had an average annual budget of about $150 million. During those 5 years, most of its work was performed for the federal government, which provided the Academy with 87 percent of its revenue. (See fig. 1.) The Departments of Transportation, Energy, Health and Human Services, and the Army; the National Science Foundation; and the National Aeronautics and Space Administration have been its largest federal sponsors—amounting to 75 percent of the total revenues for 1993 to 1997. The Academy also advises state governments, private industry, and nonprofit institutions, but that work is limited by internal Academy guidelines. In addition, the Academy may use its endowment to fund self-initiated studies deemed critical by the Academy leadership.

The Federal Advisory Committee Act Amendments of 1997 addressed concerns over the openness of the Academy’s procedures. Prior to the amendments, the Academy’s committee procedures included some openness. A 1975 policy document stated that committee meetings where data would be gathered were to be open to the public with advance notice given. Announcements of scheduled open meetings were published monthly in a newsletter by the Academy’s Office of Information. However, the study unit heads determined which projects would have scheduled and announced open meetings. Executive meetings and working meetings, referred to as deliberative sessions, would “not normally be open to the public.” A 1995 proposed change to the Academy’s public access policy, among other things, further defined the types of meetings that could be closed and applied the policy uniformly across the Academy’s major study units. This proposal was under consideration at the time the amendments were enacted.

Reasons the Academy Sought Relief From the Federal Advisory Committee Act

According to Academy officials, the Academy had three main concerns that caused it to seek relief from the Federal Advisory Committee Act: (1) the erosion of independence if the Academy was under the influence of sponsoring agencies, (2) the inability to recruit committee members if committee deliberations were open to the public, and (3) the burden of administrative requirements that would render the Academy unresponsive to the government.

Paramount among these concerns was the Academy’s independence from the influence of sponsoring agencies. Under the act, a federal government officer or employee would have to chair or be present at every advisory committee meeting. This individual would have the power to adjourn the meeting “whenever he determines it to be in the public’s interest.” According to Academy officials, the Academy could lose sole authority in appointing committee members, and the Academy and committee members could be under pressure from a sponsoring agency to change a report during the drafting process.

Under the act and GSA regulations, advisory committee meetings, including deliberative meetings, would be open to the public. However, the Academy opposed opening its deliberative meetings to the public because it believed that such an action could stifle open debate and criticism of ideas in those meetings. The Academy was also concerned that the independence of the committees’ deliberations and the Academy’s review process would be jeopardized by attempts of sponsors and special interest groups to bring political pressure to bear. Academy officials said that closed committee deliberations are fundamental for ensuring the independence of their studies and the scientific quality of their reports. Moreover, they stated, if draft reports were available to the public, the first draft would become the enduring impressions of a report, regardless of any changes made later. In addition, the President of the Academy said that it could be more difficult to recruit potential committee members in the future if deliberations were open to the public.

We surveyed 12 current and former Academy committee members to obtain their views on whether or not they would serve on Academy committees if the deliberative meetings were open to the public. Two members said that they would serve, six said that their decision to serve would depend on the topic of study, and three said that they probably would not serve on a committee whose deliberations were open to the public. One member did not respond directly to the question but said that closed deliberative sessions encourage greater candor among the members.

In addition, these members generally echoed the Academy officials’ views regarding the need for closed deliberative sessions. The three members who responded that they would probably not serve said that open deliberations could seriously jeopardize the quality of the reports. Two members said that Academy study committees might be difficult to staff if deliberations were open to the public. Eleven out of 12 respondents indicated that the Academy should retain the ability to close committee deliberations.

Finally, the Academy was concerned that the amount of time and expense associated with implementing the act would render the Academy unresponsive to the government in general and to the Congress in particular. Of particular concern was the requirement under the act that each committee have a charter. Since the Academy is not a federal agency, the federal agency sponsoring the Academy study would prepare the charter and submit it for review by GSA. Academy officials estimated that the process would take between 6 and 12 months, on average, a length of time that an Academy official said would render the Academy unresponsive to the government’s requests for information. In addition, most of the Academy’s studies are funded by multiple agencies. Thus, the Academy was not certain which agency would be responsible for fulfilling the administrative requirements of the act. Academy officials also pointed out that applying the act to the Academy would more than double the number of committee charters that GSA would have to review each year.

Academy Committee Procedures

Prior to the enactment of the amendments of 1997, the Academy established a number of procedures for committee work that are intended to help ensure the integrity and the openness of committee activities. The procedures consist of the following phases: project formulation, committee selection, committee work, report review, and report release and dissemination. (See fig. 2.) According to Academy officials, the whole process can take anywhere from 4 months to 2 years (usually from 6 to 18 months).

Project Formulation

During the project formulation phase, the Academy assigns the project to a study unit. According to Academy guidance, the study unit is responsible for defining the scope of the project, leaving room for the committee to further define the study, and for developing the initial cost estimates. After the study unit approves the project, the Academy gives final approval for the project. Then a contract, grant, or cooperative agreement (depending on the sponsor) is drawn up and entered into with the agency. A permanent Academy staff member, referred to as the responsible staff officer, is assigned to organize and support the project. The staff officer is responsible for ensuring that institutional procedures and practices are followed throughout the study and that the study stays on schedule and within budget.

Committee Selection

According to the Academy’s documents, each project is conducted by a committee of subject matter experts who serve without compensation. Committee selection starts with suggestions from the sponsoring organization, members of the Academy, outside professional colleagues, and Academy staff. After review of the suggestions, the President of the Academy selects committee candidates. The Academy’s procedures require that each committee candidate fill out a form on his or her potential conflicts of interest. The form consists of five questions asking for the member’s relevant organizational affiliations, financial interests, research support, government service, and public statements and positions concerning the committee’s topic. We reviewed a sample (about 10 percent) of the 331 current committees to determine whether the forms had been filed and found that the Academy’s procedures were generally being followed. Under Academy procedures, 5 of the 30 committees selected were not required to file the conflict-of-interest forms because they were not subject to section 15 for various reasons. Of the remaining 25 committees, we found that almost all members (316 out of 341 or 93 percent) had forms on file.

At the first meeting of every committee, the Academy’s procedures require a confidential discussion among committee members and project staff of potential conflicts of interest. If a conflict of interest is identified, the committee member may be asked to resign from the committee. If the Academy determines that the conflict is unavoidable, the Academy will make the conflict public and will retain the committee member. After this meeting, the executive director of the relevant study unit makes a tentative determination of whether the committee as constituted is composed of individuals with the requisite expertise to address the task and whether the points of view of individual members are adequately balanced such that the committee as a whole can address its charge objectively. Final approval of the committee membership, however, rests with the President of the Academy.

Committee Work

Committees meet in data-gathering sessions that are generally open to the public and in deliberative sessions that are closed to the public. The Academy defines a data-gathering meeting as “any meeting of a committee at which anyone other than committee members or officials, agents, or employees of the institution is present, whether in person or by telephone or audio or video teleconference.” Committees also meet in closed sessions to discuss financial and personnel matters, to discuss conclusions, and to draft the committee report. The Academy’s responsible staff officer facilitates the meetings.

In order to identify the number of open versus closed meetings, we reviewed the meetings held from December 1997 through June 1998 for the 331 committees. Since we found that most meetings were a combination of open and closed sessions, we identified the number of open and closed hours during these meetings. Of the 331 committees, 129 either had no meetings or were not subject to section 15 for various reasons. The remaining 202 committees held a total of 353 meetings. For 300 (or 85 percent) of those meetings, at least some portion of the meeting was closed. For 139 of the 300 meetings where complete information about open and closed sessions was available, we found that slightly less than half (45 percent) of the time was spent in closed sessions. For 251 projects, we determined the reasons for the closed sessions: 61 meetings included discussions of potential bias of committee members, 36 meetings included discussions of the committee’s composition and balance, and 201 meetings involved drafting the committee report. We also found that seven data-gathering meetings were closed under Freedom of Information Act exemptions.

Every report is the collective product of the committee. According to the Academy’s documents, a committee member may draft a chapter or portion of a report, but the author of record is the entire committee. The Academy’s responsible staff officer can help with many aspects of developing the report, including researching, integrating portions of the report written by committee members, and ensuring consistent style and format, but the conclusions and recommendations are attributed to the committee as a whole. Throughout its work, the committee is subject to the oversight of the Academy’s supervisory boards and commissions.

Report Review

The next step in the process is an independent review of the draft by individuals whose review comments are provided anonymously to the study committee. This process allows the Academy to exercise internal oversight and provides an opportunity for the study committee to obtain reactions from a diverse group of people with broad technical and policy expertise in the areas addressed by the report. The anonymity of the reviewers is intended to encourage individual reviewers to express their views freely and to permit the study committee to evaluate each comment on its merits without regard for the reviewer’s position or status.

The Academy Report Review Committee, composed of members of the Academy, oversees the report review process and appoints either a monitor and/or coordinator depending on the type of study. Liaisons are appointed from the Academy’s membership to the major study unit for the purpose of suggesting qualified reviewers. The monitor and/or coordinator either participates in the selection of reviewers or checks the list of reviewers for their relevant expertise or particular perspective. Typically six to eight reviewers are appointed, although more are acceptable for a major policy report.

According to the Academy’s report review guidelines, the review of a manuscript takes about 10 weeks, on average, from when a report is sent to the reviewers until final approval; however, the time ranges from a few days to many months. The reviewers look at whether or not the report addressed the committee’s charge; findings are supported by the evidence given; exposition of the report is effective; and tone of the report is impartial.

All study committee members are given copies of the reviewers’ comments (with the names of the reviewers removed from the comments) in time to prepare or approve a response to the comments. After the comments have been submitted, the monitor and/or coordinator may prepare a brief summary of the key review issues for the study committee. The study committee may provide a written explanation of how each comment was handled, or it may address the key review issues. The monitor and/or coordinator judges the adequacy of the committee’s responses and may require a resubmission to the reviewers.

The Academy’s procedures state that no report is to be released to the project sponsor or the public, and no findings or recommendations are to be disclosed until this review process has been satisfactorily completed. All committee members are contacted to ensure that they approve the report before it is published or released. The Report Review Committee chair provides the final approval of the reports.

Report Release and Dissemination

The Academy is responsible for the report’s dissemination plan. The report sponsor may also be involved in developing the plan. Targeted groups are selected to ensure that the report reaches all appropriate audiences. The report may also be made available via the National Academy Press web site. Briefings are often arranged for interested groups, and reports may become topics of future Academy workshops or symposia.

While the Academy Has Increased Public Access to Current Project Data, the Data Are Not Always Timely or Complete

The Academy developed a web site for current project information to increase public access as a result of section 15, added by the Federal Advisory Committee Act Amendments. However, we found that this information is not always posted in a timely manner and is sometimes incomplete.

Among other things, section 15 generally requires the Academy to make names and brief biographies of committee members public, post notice of open meetings, make available written materials presented to the committee, post summaries of meetings that are not data-gathering meetings, make copies of the final committee report available to the public, and make available the names of the principal non-Academy reviewers of the draft report.

The committee members’ names and biographies, notice of open meetings, and summary minutes of closed meetings are available on the web site of current projects. Copies of reports, which include the names of the external reviewers of the reports, are available on the National Academy Press web site. According to Academy officials, written materials presented to the committees by individuals who are not agents, officials, or employees of the Academy are available for inspection at the Academy’s public reading rooms in Washington, D.C.

We reviewed a sample of the 331 current projects to determine whether the database included the names of the committee members. Five of the 30 projects that we reviewed were not required by the act or by the Academy to post committee membership for various reasons. We found that 24 of the 25 projects had the names of the members available on the web site. Five projects had only the names of the members and no biographical statements. However, these five committees were not required to post biographies because the committees were created prior to the act.

The Academy’s guidelines state that the summary minutes for closed meetings should be posted to the web site, preferably within 10 business days of the meeting. In order to determine whether this requirement was met by the Academy, we reviewed data on the closed meetings for the 202 committees that held meetings from December 17, 1997, through June 17, 1998. As previously stated, these committees held a total of 353 meetings, with 300 of those meetings having some portion closed. We found that 270 (or 90 percent) had the minutes of the closed sessions on the web site.The minutes of these closed sessions had an average posting time of 13.5 calendar days, within the Academy’s guidelines of 10 business days. However, the amount of time to post the minutes ranged from 0 to 124 calendar days, with 26 percent of the minutes posted 15 or more days after the meeting.

At the time of our audit, spot checks of information posted on the web site were conducted at least once a week for missing or improper information. However, we found that for a total of 63 out of 331 current committees (about 19 percent) there were chronological or typographical errors or missing data in the information provided on one or more of the meetings. For example, the listings of the meetings for three projects were out of order. One meeting had two different dates listed on the project web site. For 34 projects, the agenda or summary minutes were not posted. The Academy has already taken action to correct this information or has adequately explained these specific problems. In addition, since we conducted our audit, the Academy created a records officer position responsible for checking the timeliness and accuracy of data on a daily basis.

Through the web site, the Academy also elicits public comments about committee composition. The public is allowed 20 calendar days to comment about the proposed committee members and/or suggest new members. Since the web site’s inception in December 1997 through June 1998, the Academy received a total of 120 comments. Only 13 of those comments concerned committee composition—all concerning four committees: those on smokeless and black powder, illegal drug policy, repetitive motion and muscular disorders, and cancer research among minorities. Of these comments, six included suggestions for additional committee members, three provided general or positive comments about committee membership, three included negative comments regarding specific committee members (one of the three members later was removed from consideration), and two comments discussed the length of the public comment period.

Observations

Prior to the passage of the Federal Advisory Committee Act Amendments, the Academy had efforts under way to increase public access to and participation in the Academy’s committee work. After the amendments were passed, the Academy’s web site of current projects increased public access to project information. However, the Academy had to quickly create and operationalize its web site of current projects in December 1997 and additional enhancements are under consideration pursuant to suggestions received from the public. Thus, it will be some time before an assessment can be made of the extent to which the general public uses the web site. Regarding the untimely posting of data and incomplete data, the Academy’s new procedures should address our concerns. However, the availability of timely information on current projects depends on the effective implementation of the new procedures.

Agency Comments and Our Evaluation

We provided a draft of this report to the National Academy of Sciences and GSA for their review and comment. In general, the Academy said that the report was accurate and balanced. Regarding our finding that the Academy’s data available on the web site are not always timely or complete, the Academy believed that it was important to note that in no case was there a violation of the requirements of section 15. We agree. Since section 15 does not provide a time frame for posting summaries of closed meetings, we noted instances in which data were untimely by the Academy’s own guidelines and instances in which the information provided had some errors. The full text of the Academy’s comments appears in appendix I. GSA had no comments on the report.

Scope and Methodology

To determine why the Academy sought relief from the act, we interviewed Academy officials and reviewed their statements to the Congress. We also talked with several committee members to obtain their views on the act—the Academy selected the committee members, with input from us. Each Academy study unit and the Presidents of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine selected members to respond to our questions. The Academy narrowed this sample, and each candidate was asked whether he or she would participate in the survey. The sample included past and current committee members and chairs of committees from across the country and from private industry, academia, and not-for-profit institutions.

To identify the Academy’s procedures for providing advice to the federal government, we interviewed Academy officials. We also reviewed the Academy’s internal documents outlining the procedures, the treasurer’s reports, and annual reports.

To determine whether the Academy had implemented section 15, we interviewed Academy officials and reviewed official documents. We also reviewed the Academy’s web site information, including committee meeting agendas for both open and closed portions of meetings and the content of the closed meetings as described in summary minutes, for Academy projects that were active as of June 17, 1998.

To make this determination, we calculated the hours of open and closed meetings, calculated the time in which summary minutes were posted for closed meetings, and categorized the reasons for closed meetings. Each step was verified for accuracy and completeness. Only meetings that occurred in the 6-month period from December 17, 1997, to June 17, 1998, were analyzed. Of the 331 current Academy projects, 69 had no meetings within the stated 6-month time frame, and 24 had no meetings whatsoever. Thirty-six projects were standing committees that were not subject to section 15 and were therefore excluded from our analyses. None of the current project information from the web site was independently verified against the Academy’s original records.

For the analysis of open versus closed hours, we considered only the 139 meetings with both open and closed hours. For the closed meetings, we looked only at those meetings with summary minutes or with posted agendas. Of the 300 possible meetings with some closed sessions, 294 were analyzed to determine the reasons for the closed sessions.

To measure the Academy’s compliance with the section 15 requirement to make committee members’ names and biographies available for public comment, we reviewed a random sample of 30 current projects’ potential bias and conflict-of-interest forms to determine whether they were present in the Academy’s files and signed by the committee members. We compared the Academy’s files to the committee’s printed lists from the Academy’s current projects web site. Projects that did not have meetings within the December 17, 1997, to June 17, 1998, time frame were not sampled.

We conducted our work from May through November 1998 in accordance with generally accepted government auditing standards.

As arranged with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report for 10 days. At that time, we will send copies of this report to the President of the National Academy of Sciences and the Administrator of the General Services Administration. We will also make copies available to others on request.

Please call me at (202) 512-3841 if you or your staff have any questions concerning this report. Major contributors to this report were Diane B. Raynes, Gregory M. Hanna, Lynn M. Musser, and Robin M. Nazzaro.

Comments From the National Academy of Sciences

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.