Federal Research: Policies Guiding the Dissemination of Scientific Research from Selected Agencies Should Be Clarified and Better Communicated
Released: 2007-06-18T00:00:00

Letter

Background

In recent years, there have been a number of allegations concerning interference with federal researchers in their efforts to disseminate their research results to a wider audience, external to their agency. Many of these allegations have concerned interference with researchers’ attempts to speak with the media or present research findings at conferences. These claims have frequently surrounded research on politically sensitive topics, such as climate change and the role it may play in connection to other weather events, such as hurricanes. For example, some federal researchers have alleged that they were not allowed to use the phrase “global warming,” and others have alleged that NOAA has limited scientific debate by not reporting all of the research regarding a possible connection between global warming and increased hurricane intensity.

These concerns have resulted in a number of inquiries into the dissemination practices and policies at several federal agencies. For example, in May 2006, in response to a congressional request, the National Science Foundation’s National Science Board examined the existing policies of selected federal science agencies, including NASA and NOAA, to determine if they contained steps to ensure the credibility of research results and insulate these results from suppression or distortion. The board found that there was no consistent federal policy regarding the dissemination of research results by federal employees and recommended that all agencies that conduct research establish policies and procedures to encourage an open exchange of data and results. The board also recommended that the administration develop and issue an overarching set of principles for the communication of scientific information by government scientists, policymakers, and managers that could be used as the framework under which each agency would develop its specific policies and procedures.

In addition to the National Science Board review, other groups have sought information on the dissemination activities of federal agencies. For example, in 2005, Environmental Science and Technology, an online and print journal published by the American Chemical Society, requested and received copies of NOAA Office of Public Affairs documents through the Freedom of Information Act pertaining to climate change, including exchanges between NOAA scientists and public affairs staff. Furthermore, in September 2006, 14 Senators requested that the Inspectors General at Commerce and NASA investigate reports of political interference with the work of scientists at NASA and NOAA. In February 2007, the Union of Concerned Scientists and the Government Accountability Project, two nongovernmental advocacy organizations, jointly issued a report examining the extent to which politics play a role in scientific research. More recently, over the first few months of 2007, Congress has held several hearings exploring allegations of political interference with the work of government climate change scientists.

A number of federal agencies conduct scientific research that is used to inform federal policy decisions. This report focuses on NASA and 2 of the 13 agencies within Commerce—NIST and NOAA. NASA’s organic statute—the National Aeronautics and Space Act of 1958, as amended— calls for NASA to “provide for the widest practicable and appropriate dissemination” of the scientific and technical information resulting from NASA’s research efforts. This work is structured around four mission areas—aeronautics research, exploration systems, science, and space operations—and is performed at 10 research and flight centers across the United States. Each of these centers performs a variety of research, engineering, construction, and support functions related to the mission areas, and each has a separate management structure, including a center director, to guide its activities. Furthermore, each center has its own public affairs office that is responsible for managing the center’s media contacts and communications. In addition, NASA headquarters has its own public affairs office that oversees agencywide programs and activities to coordinate and maintain open and credible communication channels to the news media and the public.

Each of Commerce’s 13 agencies is headed by its own director, undersecretary, or assistant secretary. Even though each of these agencies has its own public affairs office, Commerce’s Office of Public Affairs is responsible for overseeing public affairs interactions for the department, such as issuing press releases and answering media inquiries, as well as for keeping abreast of the activities at all 13 agencies, many of which disseminate research results, as is the case with NIST and NOAA. NIST’s mission is “to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life.” To a large extent, this mission is fulfilled by the research undertaken by scientists, engineers, and technical experts at the agency’s Maryland and Colorado facilities. NIST is specifically required to disseminate research conducted under several of its programs. Most research at NIST is conducted at one of its eight laboratories, covering research areas such as manufacturing engineering, physics, and information technology. It is NIST’s policy to publish the results of these technical programs openly, widely, and promptly, and this is accomplished through scientific reports issued by researchers as well as press releases and Web-based newsletters issued by NIST’s Public and Business Affairs Office. NOAA is the largest of the Commerce agencies, and its work is structured around four program goals—ecosystems, climate, weather and water, and commerce and transportation—and seven operating organizations, or program offices. Similar to NASA and NIST, NOAA headquarters has a public affairs office—the Office of Communications—which is led by a director and tasked with facilitating media and public communications with the agency. Each of NOAA’s program offices also has a public affairs officer who reports to the director of the Office of Communications and serves as the primary public affairs contact for program office researchers.

Research dissemination at each of these agencies is guided by departmental or agency policies. In addition, the Office of Management and Budget (OMB) has also issued guidelines and bulletins that have an impact on the dissemination of research results. For example, in October 2001, OMB issued its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which required agencies to issue their own guidelines to ensure the quality of information being disseminated. NASA, NIST, and NOAA have all issued their own information quality guidelines. In addition, OMB issued the Final Information Quality Bulletin for Peer Review in December 2004 that established governmentwide guidance establishing that important scientific information shall be peer-reviewed by qualified specialists before it is disseminated by the federal government. Among other things, the bulletin provides guidance to federal agencies on what information is subject to peer review and requires agencies to begin a systematic process of peer-review planning for influential scientific information (including highly influential scientific assessments) that the agency plans to disseminate in the foreseeable future.

In addition to the research activities of specific agencies, OSTP advises the President and others in the Executive Office of the President (EOP) on the effects of science and technology on matters being considered in the policymaking process. OSTP serves as a source of scientific and technological information and advice for the President with respect to major policies of the federal government and acts as the scientific and technical arm of the EOP policymaking process. As part of this work, OSTP and OMB collaborate to help agencies develop research and development budgets to address national priorities. OSTP meets regularly with agency staff as well as representatives of industry, academia, and the general public to seek or share information about science and technology policy and leads interagency efforts to develop and implement science and technology policies.

NASA, NIST, and NOAA Policies Generally Facilitate the Dissemination of Research, but Some Do Not

Researchers at NASA, NIST, and NOAA must comply with both office specific and agencywide or departmental-level policies, most of which facilitate dissemination of research. NASA’s researchers are subject to agency-level policies and center-specific policies for the dissemination of research through publications, presentations, press releases, and media interviews, and these policies generally facilitate dissemination. NIST and NOAA have agency-level policies that generally facilitate the dissemination of research results through publications and presentations. Researchers at NIST and NOAA must follow both agency-level policies and guidance and Commerce’s department-level policies if they want to disseminate their research through media interviews and press releases. However, NOAA’s agency-level and Commerce’s department-level policies may hinder dissemination through these routes.

NASA Policies Provide Clear Guidance to Facilitate Research Dissemination

At NASA, researchers must comply with both agencywide policies and center-specific policies when disseminating research results, regardless of the form this dissemination takes. For example, for publications and presentations, all researchers are directed to comply with the agency’s policy for approval, publication, and dissemination of scientific and technical information. Scientific and technical information is defined as the results—including facts, analyses, and conclusions—of basic and applied scientific, technical, and related engineering research and development. This information can be disseminated through a variety of channels, including NASA publications, outside journals, presentations at meetings or workshops, and Web sites. The policy directs that all scientific and technical information released outside the agency through these means must be reviewed to determine whether public access to it should be prohibited or restricted, for national security or intellectual property reasons. In addition, NASA’s policies direct that all research results that are to be disseminated undergo professional and technical reviews to ensure that the information is being clearly communicated, is technically accurate, and meets data quality standards. These technical review and management approval procedures vary depending upon the information content, publication route, and intended audience—domestic or international. To indicate that a document has completed the necessary reviews, researchers generally must complete two forms—one identifying what restrictions, if any, need to be placed on the document prior to dissemination and the other indicating that the document has undergone technical peer review. NASA managers with whom we spoke at the centers said that they found the agency’s policies to be clear, easy to follow, and similar to procedures researchers follow in the academic community, although some said that the agency’s reviews to identify security restrictions can be cumbersome at times.

Regarding dissemination of research through media interviews and press releases, researchers at NASA are subject to the agency’s policy on the release of information to news and information media, hereafter referred to as the “media policy.” This policy, which was revised in March 2006, governs the release of information to the media, especially information with the potential to generate significant media or public interest, including press releases, media advisories, news features, and Web postings. According to NASA, the media policy in place prior to March 2006 had not been substantively modified since 1987 and required clarification to be both useful and practical to implement. Furthermore, the NASA public affairs policy review team, composed of representatives from throughout NASA’s scientific and public affairs offices, found that the previous policies were convoluted, bureaucratic, and resulted in a breakdown between researchers and public affairs staff.

We found that NASA’s revised 2006 media policy is generally clear and should help facilitate dissemination of research results by devolving decision making and providing a process to resolve disputes. The policy, its accompanying “Frequently Asked Questions” guide, and statements by the NASA Administrator assert NASA’s commitment to a culture of openness with the media and the public and affirm that the agency values the free exchange of ideas, data, and information as part of scientific and technical inquiry. The policy defines the roles, responsibilities, and methods of coordination for managers, researchers, and public affairs staff and lays out guidelines for working with the public affairs office, clearly stating what public affairs officials can and cannot do. Specifically, regarding media interviews, the updated policy clearly affirms that NASA employees may speak to the media and public about their work without prior approval; however, they must notify their immediate supervisor and their center public affairs office immediately thereafter. While researchers are encouraged to have a public affairs official present during interviews, it is not required and the policy clearly defines the role of public affairs staff as one of logistical support and clarification, not interference. Finally, the policy states that NASA employees may communicate conclusions on the basis of their research to the media. However, NASA employees who present personal views outside of their official area of expertise or responsibility must make clear that they are presenting their individual views and not those of the agency.

In addition to clarifying the policy for conducting media interviews, NASA’s revised media policy also describes the process for dissemination via press releases or other media items, such as news features, media advisories, or news-related Web postings. Specifically, the policy, augmented by accompanying operating procedures, clearly outlines the coordination, review, and approval process that is required prior to issuing a press release. While some public affairs officials and managers told us that some of the reviews for press releases can require more time than available to meet the deadlines of the news cycle, they also felt that the revised policy had improved the approval process. In addition, the policy specifically delegates authority to NASA centers, independent of headquarters, to issue public information that is of local interest, among other things. Items eligible for issuance by the centers include releases on specific research topics that have a targeted audience, such as the development of a new “superplastic” that would be of interest to people who use polymers, or an announcement of an upcoming lecture series to be held at the center. Delegating authority to the centers to issue their own press releases allows NASA to better publicize the work it does to targeted audiences of interest. Finally, the revised media policy describes the process for researchers to use when they disagree with the agency’s decision regarding whether to issue a press release or another type of public information. The policy allows such appeals to be elevated to the Office of the Administrator for resolution, if needed.

NIST and NOAA Policies Provide Clear Guidance to Facilitate the Dissemination of Research through Publications and Presentations

NIST’s publication and presentation policy, called the Technical Communications Program, generally facilitates the dissemination of scientific results. Similar to NASA’s publications policy, NIST’s policy clearly describes the extent and type of reviews for technical and scientific accuracy that are required before publications and presentations may be released outside the agency. For example, abstracts and outlines for talks that are not of a sensitive nature and will be limited in distribution can be approved by the researcher’s division chief and do not need to go through a more extensive review. However, manuscripts for journals and books that are intended for public distribution must be reviewed and approved by the laboratory to which the researcher belongs as well as the NIST Editorial Review Board. This board, composed of NIST technical authors and editors from all organizational units, conducts a critical evaluation of the technical content and methodology, among other things. If the review board does not support publication, it generally recommends changes, which if made by the author, would result in approval. If the author chooses not to accept these changes, the policy delineates the steps that the author should take to resolve any disagreement. NIST managers said that the guidance and procedures for getting potential publications through the editorial review process and disseminated were adequate, clear, and readily accessible to researchers on the internal NIST Web site.

Similar to NIST, NOAA’s publication and presentation policy facilitates research dissemination and clearly describes the process for disseminating scientific and technical publications. Specifically, NOAA’s policy encourages each program office to review, edit, and disseminate its own publications, and the policy clearly delegates responsibility to each program office to ensure the scientific and technical quality of its publications and presentations. Furthermore, NOAA has developed a standard form to guide the review process and to document final approval for release. In addition, several NOAA program offices have supplemented these agencywide policies with office-specific ones. For example, the National Weather Service’s policy clearly delineates the review and approval process for their publications. Similarly, the National Environmental Satellite, Data, and Information Service created an office- specific form that documents the multiple reviews a proposed manuscript must undergo. This form, signed by the branch chief, division chief, and office director or deputy director, certifies that a research paper is worthy of scientific review or useful for education or outreach purposes. Several NOAA managers told us that NOAA’s publication review process was clear and worked well.

Researchers at NIST and NOAA Have to Comply with Commerce and Agency-Specific Policies for the Dissemination of Research through Media Interviews and Press Releases

Researchers at NIST and NOAA are subject to Commerce’s department- level policies when disseminating research results through media interviews and press releases; however, these policies are outdated and unrealistic and can impede the dissemination of research results. At NIST, researchers have been provided with agency-specific guidance to supplement Commerce’s department-level media interview and press release policies. At NOAA, researchers must also adhere to agency-level media and press release policies, which are generally unclear and can lead to inconsistent interpretations that can further hinder the dissemination of research through these dissemination routes.

Researchers at NIST and NOAA Are Subject to Outdated Commerce Policies for Media Interviews and Press Releases

Researchers at NIST and NOAA must follow Commerce’s department-level policies for dissemination of research results through media interviews and press releases; however, these policies, which have not been revised for over 20 years, are outdated and unrealistic. Commerce policies direct all of its constituent agencies to submit all proposed media interviews and press releases to its Office of Public Affairs for review and approval, and the department does not delegate authority to approve any of these activities to the agency-level public affairs office. Because Commerce’s public affairs office is responsible for overseeing the media activities for 13 agencies, according to Commerce officials, it is not realistic to expect this office to review every media interview and press release request that each of these agencies generates. For example, from a large agency like NOAA, the volume of media requests can range from dozens on a normal business day to hundreds following a major event, such as a tsunami. In addition, Commerce’s policy states that disputes concerning a potential media interview or press releases that are not resolved by the Director of Public Affairs shall be referred to the Secretary of Commerce.

At a February 2007 Senate hearing, a Commerce official referred to the department’s communication policies as “contradictory” and “woefully outdated.” These policies, which have not been revised since 1983, were implemented prior to the expansion and widespread adoption of newer media outlets, such as the Internet and 24-hour cable news stations, and therefore do not take these routes of dissemination into consideration. While officials from Commerce’s Office of Public Affairs said that they would like to be informed about all press releases and interviews, they realize that this request is not realistic given the volume of releases from some agencies. Consequently, Commerce public affairs officials acknowledged that even though it is required, they do not generally review low-level or routine releases.

To streamline the approval process and respond to allegations by some researchers about the department’s interference with media dissemination of research results, Commerce began an effort in November 2005 to revise its media and communications policies. According to a Commerce official, this effort included three rounds of input. During the first round, Commerce solicited informal feedback from managers and some scientists at selected agencies, including at NIST and NOAA. As part of the subsequent two rounds, Commerce solicited input from managers and public affairs officials at all Commerce agencies. In addition, these officials were encouraged to share the draft with researchers and other public affairs officials within their agency to obtain their feedback and input. We reviewed a draft of the revised policy in March 2007. Because the final policy was released after our audit work was completed, we did not conduct a detailed evaluation of the new policy. However, we did note that the revised policy included provisions to address some of the concerns we identified with the prior policy. For example, the revised policy affirms the department’s support for open and free communication of scientific and technical ideas, findings, and conclusions based on researchers’ official work and asserts that researchers are free to participate in interviews, without prior approval, on matters directly related to their research. In addition, the revised policy takes other positive steps, such as the inclusion of an appeals process and the clarification of the role of the public affairs office. However, it also contains some unclear elements that may cause confusion for the agencies and researchers who are trying to adhere to the policy. The policy was officially released on March 29, 2007, and, according to a Commerce official, it will be accompanied by a training program to educate public affairs officials on how to implement it.

Researchers at NIST Have Been Provided with Supplemental Agency Guidance

To supplement Commerce policies for dissemination of research results through media interviews and press releases, NIST has issued Web-based guidance that provides an overview of the Commerce policy and serves as a reference tool for NIST researchers who interact with the media. The guidance on responding to media inquiries explicitly states that interview requests from the media are to be cleared in advance through Commerce’s Office of Public Affairs. In addition, this guidance provides information on Commerce’s media policy in a question-and-answer format, including instructions that researchers should discuss only the research facts and should not express personal opinions when communicating with the media.

NIST managers commented that while NIST issues between 30 and 50 press releases a year, they have found that other dissemination methods, such as e-mailing a biweekly publication called Tech Beat, have been a more effective way to reach the media. In fact, the managers estimated that 60 percent of the news articles about NIST research are generated from Tech Beat articles. Like press releases, Tech Beat articles go through the Commerce review and approval process, and they are subject to NIST technical and scientific review. NIST officials with whom we spoke said that while they forward most proposed interview requests to Commerce for review and approval, they generally do not seek approval for interviews that result from a press release or from a Tech Beat article previously approved by Commerce.

Researchers at NOAA Are Subject to Unclear Agency- Level Media Interview and Press Release Dissemination Policies

In addition to adhering to Commerce’s policies for media interviews and press releases, researchers at NOAA must also follow NOAA-specific policies and procedures, which are at times unclear, leading to inconsistent interpretation. For example, regarding media interviews, one part of NOAA’s policy directs its employees to inform NOAA headquarters or program office public affairs officials prior to participating in interviews that are of national news interest, concern regulatory or controversial issues, or pertain to research having potential policy interest; however, another part of this policy states that responsibility of the NOAA headquarters’ public affairs office is to approve and coordinate media communications, including interviews and press releases. As a result of this unclear language, public affairs officials have interpreted these requirements differently. Some believe that researchers are required to obtain prior approval for all interviews, while others believe researchers must notify the public affairs office after an interview has occurred. This has resulted in an uneven application of the policy among researchers. Furthermore, the policy does not define what qualifies as “of national news interest” or “controversial,” thereby leaving the interpretation to each individual.

In addition, some of NOAA’s procedures required for media interview approvals can be burdensome and could delay dissemination of research. To seek approval for a media interview, NOAA program office public affairs staff are expected to complete a form that contains information about the topic, the potential questions, the media outlet, and the reporter, among other things. The completed form is forwarded to the headquarters’ public affairs office for review and approval. However, while interview approvals typically occur within a few hours, in some cases they have arrived after the reporters’ deadlines, resulting in missed interviews, according to NOAA public affairs officials. Furthermore, some of these officials expressed concern that reporters may shy away from contacting NOAA researchers because of the drawn out approval process, ultimately causing NOAA to miss opportunities to inform the public about its work.

Regarding press releases, NOAA public affairs officials have also interpreted approval requirements differently from one another. For example, some officials believe that the policy requires all press releases to be approved by the NOAA headquarters’ public affairs office, while others believe headquarters approval is not required for releases of regional interest or involving routine activities. NOAA’s press release policy, unlike NASA’s policy, does not distinguish between press releases on local topics and press releases on national topics and does not delegate authority to program office public affairs officials to issue such press releases. However, according to public affairs officials with whom we spoke, routine releases may not warrant the same scrutiny as ones likely to be of national interest or controversial. For example, a NOAA public affairs official commented that routine releases about StormReady—a community safety and emergency preparedness program—should be treated differently from press releases that have a broader scope, such as annual releases of climate data.

According to NOAA public affairs officials, the approval process for press releases is lengthy and burdensome, and approvals that took 1 to 2 days a few years ago can now take as long as 2 weeks to obtain. This is because the process involves 13 separate steps for approval that begin after the press release has been drafted, which itself can be a lengthy process. (See fig. 1.) In November 2006, another requirement was added to the approval process—that is, a public affairs briefing memorandum must precede certain requests to issue a press release, such as those involving a controversial or “hot” topic. According to a senior NOAA public affairs official, the intent of requiring the memorandum was to inform NOAA management about upcoming announcements and help ensure that the program office public affairs staff had gathered enough information about the proposed release to judge its merit.

The myriad steps in the approval process can hinder the timely issuance of press releases. As was the case with delays in approving interviews, officials told us press releases that come out too late are in danger of not making it into the news and cited several instances in which requests to issue press releases never received an approval or a denial from headquarters public affairs and, consequently, were never released. Moreover, no explanation for the lack of response was offered to the public affairs office or the researcher. Furthermore, officials at a NOAA laboratory have become discouraged from seeking press releases for research conducted at the laboratory because of delays and the lack of responsiveness to such requests, further limiting NOAA’s ability to publicize its research.

Despite Agencies’ Efforts to Communicate Research Dissemination and Dispute Resolution Policies, Many Researchers Are Not Confident about How to Comply with Them

According to our survey, researchers at NASA, NIST, and NOAA usually learned about their agency’s dissemination policies through informal communication methods, rather than through formal training provided by their agency. In addition, more researchers at the three agencies are confident that they understand their agency’s policies on the dissemination of research through publications and presentations well enough to comply with them, than are confident that they understand their agency’s media interview and press release policies. Furthermore, many researchers at NASA, NIST, and NOAA are unsure whether their agency’s policies allow them to discuss research results with potential policy implications or to express personal views related to federal policy decisions. Finally, we estimate that only one-quarter of all researchers across the three agencies are aware of their agency’s processes to appeal decisions made regarding requested dissemination.

Most Researchers Learned about Agency Dissemination Policies through Informal Methods

According to our survey and agency officials with whom we spoke, researchers at NASA, NIST, and NOAA were most likely to learn about their agency’s dissemination policies through informal communication methods. Most researchers at each agency learned about these policies through a variety of informal means, such as e-mails from their agency or operating unit managers, or through on-the-job training with supervisors. Researchers were less likely to learn about agency dissemination policies through formal training sessions, a method that agency officials told us was typically offered on a limited basis focused toward specific groups, such as those likely to have media contact. Specifically: At NASA, we estimate that three-quarters of researchers learned of their agency’s dissemination policies through e-mails or other correspondence from center or office managers, 70 percent through on-the-job training, and about one-half through notifications on internal agency Web sites. Managers and public affairs officials said that limited training on dissemination of research through publications, presentations, and media interviews is available. At most centers, managers said that training on the publication and presentation review process is done on the job by supervisors or managers, although some centers have conducted limited formal training on how to comply with export control regulations, which are a required segment of NASA’s scientific and technical review process.

Several NASA center public affairs officials said they have also conducted limited formal training on media interviews, but this training was usually targeted toward researchers who are expected to have significant exposure to reporters. Headquarters and several center officials told us that little or no formal training had been conducted on NASA’s recently revised media policy.

At NIST, officials told us that they generally rely on the agency’s internal Web site to inform employees about NIST’s publication and presentation policies; however, they mentioned that more specific, individual training is offered on an as-needed basis. Media and communications training targeted toward researchers with high media exposure is offered periodically, covering the approval processes for media interviews and press releases. However, according to our survey estimates, nearly 80 percent of researchers received information about their agency’s policies through on the job training, and nearly two-thirds learned about them through e-mail communication from the agency. Additionally, about one- half of researchers at NIST learned of the dissemination policies through notifications on internal agency Web sites or at staff or town-hall meetings; and fewer (about 24 percent) attended formal training sessions to learn about the policies.

At NOAA, researchers most often learned of the agency’s dissemination policies through informal methods, such as e-mails or other correspondence from center or office managers (75 percent), on-the-job training (59 percent), and notifications on internal agency Web sites (37 percent). These sentiments were echoed by NOAA managers who told us that they typically inform researchers about the agency’s policies on dissemination of publications and presentations through on-the-job training, on an as-needed basis. They added that researchers are usually exposed to standard scientific peer review processes during graduate school, and that these processes are similar to the agency’s review processes. Regarding media interviews, at least one public affairs branch office offered a formal training course that included an overview of NOAA requirements in this area and offered advice on effectively communicating with the media. According to program office officials, this training was well received by those who participated.

Figure 2 shows the methods by which researchers at NASA, NIST, and NOAA generally learned about their agency’s research dissemination policies.

Researchers Have More Concerns about Adhering to Their Agency’s Media Interview and Press Release Policies than to Their Publication and Presentation Policies

Most researchers at NASA, NIST, and NOAA feel confident that they know how to comply with their agency’s policies for publications and presentations, but many do not feel as confident that they know how to comply with their agency’s policies on media interviews and press releases. (See fig. 3.) According to our survey, we estimate that more than 80 percent of researchers at each agency are confident that they understand their agency’s policies on publications and presentations well enough to comply with them. At NIST, about three-quarters of researchers are also confident that they know how to follow their agency’s policies on media interviews and press releases. In contrast, nearly one-half of researchers at NASA and over one-third of researchers at NOAA are not confident that they understand their agency’s policies on media interviews well enough to follow them, and 40 percent of NASA researchers and 35 percent of NOAA researchers are not confident they understand their agency’s policies on press releases.

Researchers Are Often Unclear about Whether They Can Discuss Research Results with Potential Policy Implications or Express Their Personal Views

Many researchers at NASA, NIST, and NOAA are unclear about whether their agency allows them to discuss research results with policy implications. This is particularly striking at NASA and NOAA, given that the leaders of these two agencies have each assured their researchers that they may discuss the policy implications of their research. When research results are consistent with their agency’s policy or position statements, we estimate that two-thirds of researchers across NASA, NIST, and NOAA believe that they are allowed to discuss these potential policy implications. However, when research results differ from their agency’s policy or positions statements, 23 percent of researchers across all three agencies believe they may not discuss those results, and 40 percent are not sure. In addition, even though NASA’s recently revised media policy clearly states that researchers are allowed to discuss conclusions drawn from their work with the media as long as they disclose them as personal views and not views of the agency, many researchers are unsure whether they are allowed to discuss personal views related to potential policy implications of their research results. On the basis of our survey, we estimate that only one-half of researchers at NASA believe that they may make such statements, and another 40 percent are not sure whether they can.

Unlike NASA researchers, researchers at NIST are not allowed to discuss their personal views on the policy implications of their research, even if they disclose that the views are their own and not those of their agency. This is because NIST and Commerce policies state that when speaking publicly on topics related to research conducted at the agency, it is not realistic for a researcher to express personal views without the perception that the employee is speaking on behalf of the agency. Nevertheless, we estimate that 24 percent of researchers at NIST believe that they are free to discuss potential policy implications of their research provided they identify such views as their personal opinions and not those of the agency, and 49 percent of researchers at NIST are not sure whether they can discuss their personal views.

NOAA researchers, like NIST researchers, are also subject to Commerce’s policy that restricts their ability to discuss their personal views. However, in a February 2006 e-mail to NOAA employees, the NOAA Administrator said that communicating personal views was allowed provided that the researcher stated the views were the researcher’s opinion. Despite the clarifying instruction from the administrator, we estimate that 42 percent of NOAA researchers are uncertain whether they may discuss personal views with the media, and another 40 percent believe they may do so if they offer a disclaimer.

Researchers Are Generally Unaware of Their Agencies’ Process for Appealing Dissemination Decisions

According to our survey, only 33 percent of NASA researchers, 26 percent of NIST researchers, and 17 percent of NOAA researchers are aware of their agencies’ processes to address disputes over dissemination requests. Even at NASA, where officials told us that the agency rolled out its revised media policy, which includes a dispute resolution process, with great fanfare, we estimate that only one-third of researchers are aware of the appeals process for dissemination of research. According to NASA headquarters and center public affairs officials with whom we spoke, no one, to date, has attempted to use the dispute resolution process for media-related concerns. For other types of dissemination, such as through publications, center managers generally told us that formal processes for appeal did not exist; however, most managers said that a researcher would likely raise concerns through the chain of command or rely on other agency processes, such as those for personnel disputes. Most NASA center managers told us that disputes over disseminating scientific and technical information are rare, and some managers said that they had never needed a formal process to resolve disputes.

NIST managers told us that the NIST technical review policy delineates a process for appeals of dissemination decisions for publications and presentations. However, according to our survey, we estimate that only 26 percent of researchers at NIST are aware that such an appeals process existed. NIST officials said that no disputes over the dissemination of information via publications had occurred in the past 5 years. Regarding media contacts, Commerce officials said existing Commerce policies include a process to handle disputes. However, these officials acknowledged that the process may not have been clear to staff, and they plan to clarify this process in future policies.

At NOAA, we estimate that 17 percent of researchers are aware of the agency’s appeals process. NOAA managers told us that the agency’s policies do not include an appeals process. The managers said that if a dispute arose, they expect the researcher to appeal the decision through the chain of command or use the agency’s alternative dispute resolution process, which involves mediation of the dispute by a neutral third party, often from another NOAA program office.

Six Percent of Researchers at NASA, NIST, and NOAA Had Dissemination Requests Denied, and Many Believe That Their Agencies Are Less Supportive of Media Interviews than Other Dissemination Methods

On the basis of our survey, we estimate that across NASA, NIST, and NOAA, 6 percent of researchers were denied the opportunity to disseminate their research results in the past 5 years, sometimes, they believe, without explanation. In general, however, most researchers at these agencies believe that their agency either insists on or encourages dissemination of research results. At the same time, significantly more researchers at NASA, NIST, and NOAA believe that their agency supports dissemination through publications and presentations rather than interviews with the media. Finally, most researchers at NIST and NOAA believe that their agencies consistently applied the dissemination policies across all routes of dissemination. In contrast, many researchers at NASA believe that policies for press releases and media interviews are not applied as consistently as its policies for publications.

Some Researchers at Each Agency Who Were Denied Approval to Disseminate Their Results Believe They Were Not Provided with an Adequate Explanation

Six percent of the approximately 5,000 researchers, or about 200, across the three agencies have had their requests for dissemination denied for reasons other than those stemming from standard technical review. According to our survey, we estimate that 7 percent of NASA’s 1,794 researchers, 4 percent of NIST’s 1,337 researchers, and 6 percent of NOAA’s 1,815 researchers have in the past 5 years been denied the opportunity to disseminate their research. The denials involved a wide range of research topics and occurred across all routes of dissemination, within each NOAA branch office, and within nearly all of the NASA centers and NIST laboratories included in our survey.

Researchers at each agency were given a variety of reasons for the denials, but some indicated that they were given no reason. For NASA researchers, representing a variety of areas, including aeronautics; biological or life sciences; and climate, environment, or atmosphere research, among the most common reasons reported for denials was that the information was restricted or that the topic discussed or results presented were sensitive. The subject of some restricted research involved government classified information, or results that were covered by export control regulations. For example, one researcher reported his request for dissemination of research was denied because of restrictions on dissemination of high frequency communications technology. As a result of agency denials, most of these NASA researchers gave up trying to disseminate these particular results, although some subsequently disseminated the results of their work using a different dissemination route.

Researchers at NIST who had their requests denied also represented a range of research areas, including information technology, electrical engineering, and physical or chemical sciences. Many of these researchers responded that they either were not given a reason for the denial or did not know the reason. In these situations, most researchers either found another route to disseminate these particular research results or gave up trying. Other researchers who had requests denied at NIST reported that the reasons given for the denial included that the results or topic was sensitive or that there was a question about the scientific merit of the results. In some of these cases, the researchers were able to disseminate their results once they added a disclaimer that the opinions expressed in the research results did not reflect the views of the agency.

At NOAA, researchers who had requests denied represented a diverse cadre of research areas, including climate, environment, or atmosphere; oceans and coasts; and fisheries and ecosystems. Among the most common reasons that researchers reported for the denial of their requests to disseminate research were that the topic or results were sensitive and that resources, such as money for travel, were limited. For example, one researcher reported that his request was denied because there was concern that the results might be misinterpreted in light of recent hurricane events. In addition, like NIST researchers, many NOAA researchers reported that they were not given a reason for the denial or did not know the reason. Those who indicated they were denied for budgetary reasons had requested funding to travel to a conference, and reported that after the denial they disseminated their work through another route. Those who said their requests were denied because their results were considered sensitive took various steps to obtain subsequent approval to disseminate their results, including resubmitting the same or a similar document or seeking an alternative dissemination route. Of the researchers who either did not know the reason for denial or were not given one, most gave up trying to disseminate these particular results.

Researchers Believe That Their Agencies Provide Greater Support for Dissemination of Research through Publications and Presentations than through Press Releases and Media Interviews

While researchers at NASA, NIST, and NOAA believe that, overall, their agencies support dissemination of research results, many believe that their agency is more supportive of some dissemination routes than others. We estimate that at least 85 percent of researchers at each agency believe that, overall, their agency either encourages researchers to disseminate their research results or insists that they do so. However, we also found that researchers believe that their agency is more supportive of sharing results within the scientific community through publications, such as peer- reviewed journals, and presentations, than sharing results with the general public through media interviews or press releases. Specifically, we estimate that 91 percent of researchers at NASA and NOAA and 97 percent of researchers at NIST believe that their agency supports dissemination of research results through publications. However, significantly fewer researchers at each agency believe their agency supports dissemination through press releases, ranging from 73 percent of researchers at NOAA to 87 percent at NIST. (See fig. 4.) Similarly, regarding dissemination through media interviews, significantly fewer researchers, ranging from 54 percent at NOAA to 68 percent at NIST, believe that their agency is supportive of this dissemination route.

Researchers echoed these sentiments in their comments on ways the dissemination of research results works well and how it could be improved. Many researchers responded that agency support for dissemination was good, particularly for scientific publications and presentations, but could be improved for press releases and media interviews. For example, one NIST researcher commented that the agency has a strong cultural belief in the technical integrity of peer-reviewed journals compared with general news media outlets. A NOAA researcher said that although his division supported and encouraged original, cutting- edge research and dissemination through scientific literature, he did not believe that NOAA or Commerce effectively disseminate research results to the public or the Congress. Moreover, a representative of a NOAA laboratory told us that although the laboratory issues over 100 peer- reviewed articles every year, only a few are publicized through press releases. In addition, representatives of one NASA center believe that the role of public affairs has shifted from help to hindrance, and that some requests for press releases announcing contentious research results had not been approved by headquarters, sometimes without explanation. In addition, several agency managers told us that support for dissemination depends on the tone set by senior managers. More specifically, one manager said that without commitment and support from agency leaders, even good policies will not ensure that important research gets disseminated.

Most Researchers Generally Believe That Dissemination Policies Are Consistently Applied

According to our survey, over 72 percent of researchers at NIST and NOAA believe that their agency consistently applies its dissemination policies across all dissemination routes. In contrast, more researchers at NASA believe that the agency consistently applies its policies for publications than believe the agency consistently applies its policies for press releases and media interviews. We estimate that only 67 percent and 57 percent of researchers believe NASA consistently applies its press release and media interview dissemination policies, respectively, as opposed to 84 percent and 74 percent, respectively, of researchers who believe the agency consistently applies its policy for publications and presentations. (See fig. 5.)

In contrast, most researchers at NIST and NOAA believe their agencies consistently apply all of their dissemination policies. According to our survey, over 85 percent of the researchers within NIST and approximately 75 percent of the researchers at NOAA believe their agencies consistently apply their dissemination policies for publications, presentations, press releases, and media interviews.

OSTP Provides Indirect Oversight of Agencies’ Dissemination Policies

According to OSTP, the office does not conduct any scientific research on its own nor does it formulate or directly oversee the development of dissemination policies or decisions at individual agencies. However, OSTP told us that it has affirmed to agency leaders the value of science as the basis for effective federal action and recognizes the importance of timely, complete, and accurate communication of scientific information. According to OSTP, the director of the office has on several occasions asked the leaders and chief scientists of federal agencies to develop, revise, or reemphasize their dissemination policies and to ensure that both employees and managers understand their rights and obligations under the policies. For example, in an April 2006 letter that transmitted copies of NASA’s policy to agency chief scientists, the director praised NASA’s media policy as a model and cited the Frequently Asked Questions supplement as a best practice. In addition, according to OSTP, the director instructed agencies that federal scientists, like federal employees generally, are obliged to distinguish their personal views from the official positions of their agencies but are in no way to be restricted in their ability to openly communicate their scientific findings.

With regard to disseminating the results of interagency research, OSTP sometimes plays a role in notifying agencies when research reports of interest are published by another agency, and it will disseminate research results to members of various National Science and Technology Council committees, subcommittees, and working groups, according to OSTP officials. For example, to ensure accuracy and readability, OSTP reviews interagency reports on the scale, quality, and effectiveness of the federal science and technology effort as well as all reports from interagency working groups and task forces. In addition, OSTP may, at its own initiative or in response to an agency’s request, review presentations developed by individual agencies that are based on interagency reports. According to OSTP, the office is part of the interagency review process when federal scientists testify before Congress, and, accordingly, OSTP officials review all of the statements prepared by federal scientists who are testifying in their official capacities. Regarding presentations and articles prepared by OSTP employees, OSTP officials told us that the office has a process by which these presentations and articles are reviewed and for making decisions about whether its employees can participate in policy-related media interviews.

In discussing OSTP’s role with NASA, NIST, and NOAA public affairs officials, we found that some of these agencies may inform OSTP of certain dissemination events, but their policies do not call for routine OSTP review or approval. For example, Commerce and NOAA public affairs officials told us that OSTP often receives informational copies of press releases that are particularly sensitive or involve regulatory issues.

Conclusions

More and more of the major policy debates of the day hinge on the results of scientific research. Therefore, timely and thorough dissemination of research results within the research community and to the public at large is crucial. To help ensure that this kind of dissemination is taking place, NASA, NIST, and NOAA have each taken some steps to develop policies to guide the dissemination of research through publications and presentations for the scientific community. However, the lack of clarity and consistency in the application of agency-level policies for requests to disseminate research to the public through press releases and media interviews continues to generate significant concern among some agency researchers, such as those at NASA. For researchers at NIST and NOAA, this situation is further exacerbated because they must comply with Commerce’s outdated and unrealistic department-level policies, which can further impede the dissemination of research results through media interviews and press releases. Moreover, when disputes arise about an agency’s decision regarding a dissemination request, few researchers are aware of the dispute resolution processes that exist within their agency. To some extent, this situation may have been partly caused by a lack of formal training provided by the agencies on the dissemination and dispute resolution policies to their research staff. Instead most researchers at these agencies learn about their agency’s policies through informal means and, therefore, are not particularly confident that they know how to comply with all of them.

Recommendations for Executive Action

To ensure that the policies to guide researchers and public affairs officials in their efforts to disseminate research to the scientific community and the public are clear, transparent, consistently applied, and completely understood, we recommend that the Secretary of Commerce and the NOAA Administrator each take the following action: Clarify their policies for disseminating research results via press releases and media interviews.

To ensure that (1) researchers have a clear process to follow for appealing decisions regarding dissemination of research results and (2) all researchers are aware of and understand how to comply with their agencies’ policies for disseminating research, we recommend that the Secretary of Commerce, the NASA Administrator, the NIST Director, and the NOAA Administrator each take the following two actions: Review their dissemination policies and ensure that they clearly identify a process by which researchers can appeal dissemination decisions.

Provide formal training to inform, reinforce, and update managers, researchers, and public affairs staff on these policies.

Agency Comments and Our Evaluation

We provided Commerce, NASA, NIST, NOAA, and OSTP with a copy of this report for review and comment. Commerce, on behalf of NIST, NOAA, and itself, generally concurred with our findings and recommendations. In its letter, Commerce noted that the draft report highlighted some ongoing challenges for the department, NIST, and NOAA that they will address. Furthermore, in an addendum from NOAA, the agency stated that it is updating its public communications policies to ensure consistent understanding by researchers and public affairs officials. In addition, NIST suggested some technical changes that we have incorporated in this report as appropriate. NASA also concurred with the recommendations we made to the agency. In its letter, NASA stated that it plans to direct the Chief of Strategic Communications and the Assistant Administrator for Public Affairs to, among other things, review its current dissemination policies to ensure they clearly identify a process for appealing dissemination decisions and to provide biannual training to reinforce dissemination guidelines. Finally, OSTP generally agreed with the report’s recommendations made to Commerce, NASA, NIST, and NOAA. The office commented that the report is fair and accurately describes OSTP and its role in this area. The comment letters from Commerce, NASA, and OSTP appear in appendixes III through V.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to interested congressional committees and Members of Congress, the Secretary of Commerce, the NASA Administrator, the NIST Director, and the NOAA Administrator. We also will make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have questions about this report, please contact me at (202) 512-3841 or mittala@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

Our objectives for this review were to (1) identify and evaluate the policies that guide the dissemination of federal research results at the National Aeronautics and Space Administration (NASA), the National Institute of Standards and Technology (NIST), and the National Oceanic and Atmospheric Administration (NOAA); (2) determine how effectively the dissemination and dispute resolution policies of these agencies have been communicated to researchers; (3) determine the extent to which researchers at these agencies have experienced restrictions on the dissemination of their research results, and (4) provide an overview of the role of the Office of Science and Technology Policy (OSTP) in helping agencies develop and implement policies related to the dissemination of research results.

To identify and evaluate the policies that guide the dissemination of federally funded research results at NASA, NIST, and NOAA, we obtained, reviewed, and analyzed the dissemination policies for these agencies, in addition to relevant policies from the Department of Commerce. In addition, we spoke with managers and researchers at NASA, NIST, and NOAA as well as public affairs officials at each of these agencies and Commerce. Specifically, at NASA, we spoke with managers at each of nine research, space, and space flight centers, public affairs officials from NASA headquarters and the four largest centers in terms of researchers who would be in a position to disseminate research, and the agency’s Chief of Staff. At NIST, we spoke with agency officials, including the Chief of Staff, the program office director, and the director of the public and business affairs office. At NOAA, we spoke with managers in each of the NOAA program offices, as well as each of the laboratories in the Oceanic and Atmospheric Research program office and each of the National Marine Fisheries Service fisheries science centers. We also spoke with public affairs officials in each of the NOAA program offices. We followed a standard set of questions for each of these interviews, confirmed that we had copies of the relevant dissemination policies, and discussed at length the ways in which these policies were put into practice.

To determine how effectively the dissemination and dispute resolution policies have been communicated to researchers and to determine the extent to which researchers at these agencies had experienced restrictions on the dissemination of their research, we conducted a Web-based survey of a stratified random sample of 1,811 researchers across scientific and engineering disciplines at NASA, NIST, and NOAA. This sample included 578 NASA researchers, 699 NIST researchers, and 534 NOAA researchers. We selected these researchers from among the 1,794 researchers at NASA, 1,337 researchers at NIST, and 1,815 researchers at NOAA. Overall, we received a 66 percent response rate to our survey, for a total of 1,177 respondents. We stratified by agency subcomponent—such as the NASA center, NIST laboratory, and NOAA program office—and produced estimates at the agency level. Each subcomponent has a known nonzero probability of being selected, and each subcomponent is weighted in the analysis to account statistically for all subcomponents in the population, including those that were not selected. We sought the researchers’ views on their agency’s research dissemination policies, the level of agency support for dissemination, and their experiences with dissemination. In addition, we asked the researchers to provide examples of ways in which dissemination of results works well and ways in which it could be improved. We defined researchers to be included in our population as federally employed scientists, engineers, or other researchers who are in a position to disseminate their research results to a wider audience. To identify these names, we asked center or program managers at NASA, NIST, and NOAA to provide a list of researchers in their organization who would meet our defined criterion. We cross-referenced this list with other published lists of researchers in each facility, where available. We further refined the list by sending out notification e-mails that allowed recipients who did not meet our criteria to notify us prior to issuing the survey.

Information about accessing the survey was provided via e-mail for all survey participants. The survey was activated, and researchers were informed of its availability on November 16, 2006; it was available through December 31, 2006. To ensure security and data integrity, we provided all participants with a user name and a personal password that allowed them to access and complete the survey. No one else could access that survey or edit its data. To reduce survey nonresponse, we sent out e-mail reminder messages and a contractor was used to make follow-up telephone calls to all nonrespondants to encourage them to complete the survey. From the initial notification, we identified 21 individuals who were outside the target population. For example, some individuals had retired from the agency. In all, we received a 66 percent response rate. Response rates for our survey population by agency were as follows: NASA, 63.6 percent; NIST, 61.5 percent; and NOAA, 73.7 percent.

All sample surveys are subject to sampling error—that is, the extent to which the survey results differ from what would have been obtained if the whole population had been observed. Measures of sampling error are defined by two elements, the width of the confidence intervals around the estimate (sometimes called the precision of the estimate) and the confidence level at which the intervals are computed. Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Because each sample could have provided different estimates, we expressed our confidence in the precision of our particular sample’s results as a 95 percent confidence interval. This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. As a result, we are 95 percent confident that each of the confidence intervals based on the survey includes the true values in the sample population. Unless otherwise noted, all percentage estimates have a 95 percent confidence interval within plus or minus 8 percentage points. The 95 percent confidence intervals for numeric estimates are presented along with those estimates in the body of the report. In addition, any comparison between point estimates is statistically significant at the 0.05 level, unless otherwise noted. In some instances, we used general modifiers (i.e., few, some, many, and most) to characterize percentage point estimates or to indicate the number of center or office managers who made a particular statement. We used the following method to assign these modifiers to our statements: “few” represents less than 10 percent of respondents or managers representing centers or offices, “some” or “many” represents 10 to 50 percent of respondents or managers representing centers or offices, and “most” represents over 50 percent of respondents or managers representing centers or offices. These ranges do not represent a standard; rather, we assigned these numeric ranges on the basis of natural breaks in our data. In all cases where we use these modifiers to characterize point estimates from our survey, their use implies statistical significance. For example, where we use the term “most,” the lower bound of the confidence interval of the point estimate is greater than 50 percent.

Regarding the survey data on instances in which researchers were denied the opportunity to disseminate, the percentage of researchers reporting such instances can be generalized to the population. However, because the number of denials is small, the details concerning the reason for denial, the researcher’s primary field of research, and the actions taken in response to the denial cannot be generalized. Because the issue of a researcher being denied is a salient piece of our analysis, we included this nongeneralizable information to provide context to this important issue. To analyze select, open-ended questions on our survey, including those given as the reason researchers said they were denied, we conducted a content analysis to develop our agreement statistics. Reviewers (two per question) collaboratively developed content categories based on survey responses, and then independently assessed and coded each survey response into those categories. Intercoder reliability (agreement) statistics were electronically generated in the coding process, and agreement statistics for all categories were 90 percent or above. Coding disagreements were resolved through reviewer discussion or a third-party arbiter.

In addition to the reported sampling errors, as previously indicated, the practical difficulties of conducting any survey may introduce errors, commonly referred to as “nonsampling errors.” For example, differences in how a particular question is interpreted, the information sources available to respondents, or the types of sample members who do not respond can introduce unwanted variability into the survey results. Our estimation method assumes that nonrespondents are missing at random. If characteristics of respondents are different than nonrespondents on key items, it could introduce a bias not accounted for in our analysis. We took extensive steps in questionnaire development, data collection, and the editing and analysis of the survey data to minimize nonsampling errors. For example, the survey was developed by a GAO survey specialist in conjunction with subject matter experts, and then reviewed by a second, independent survey specialist. In addition, we pretested the survey by telephone with researchers from all three agencies. During these pretests, we asked each researcher to complete the survey as they would when they received it. We then interviewed the respondents to ensure that (1) the questions were clear and unambiguous, (2) the terms used were precise, (3) the survey did not place an undue burden on the researchers completing it, and (4) the survey was independent and unbiased. On the basis of the feedback from the pretests, we revised the questions, as appropriate. The SAS and SUDAAN programs that produced our survey estimates, including estimates of categories derived from content analysis, were reviewed by a second, independent programmer to ensure accuracy in the logic and syntax of the program.

To determine OSTP’s role in helping agencies develop and implement policies for the dissemination of research results, we submitted written questions. They subsequently replied to these questions in writing.

We conducted our work from June 2006 through March 2007 in accordance with generally accepted government auditing standards.

Appendix II: Selected Survey Results for NASA, NIST, and NOAA

The following tables contain summary results of selected questions from our survey of researchers at NASA, NIST, and NOAA. For each question reported below, the estimated percentage is presented along with its 95 percent confidence interval. These tables do not include summary-level data for the demographic questions and do not include the results from any open-ended questions.

Q8. Overall, how familiar or unfamiliar are you with the policies that your agency currently has in place regarding the dissemination of research results outside of the agency?

Q9. Has your agency used any of the following means to help you understand how to comply with its policies regarding the dissemination of research results outside of the agency? 14.5-19.4 Q9. Has your agency used any of the following means to help you understand how to comply with its policies regarding the dissemination of research results outside of the agency? b. On-the-job training (including mentoring)

3.8-10.2 Q9. Has your agency used any of the following means to help you understand how to comply with its policies regarding the dissemination of research results outside of the agency?

Q9. Has your agency used any of the following means to help you understand how to comply with its policies regarding the dissemination of research results outside of the agency? d. E-mails or other correspondence from Administrator-level offices sent to all staff 10.5-19.6 Q9. Has your agency used any of the following means to help you understand how to comply with its policies regarding the dissemination of research results outside of the agency? e. E-mails or other correspondence from Center or Office management sent to all Center or Office staff Q9. Has your agency used any of the following means to help you understand how to comply with its policies regarding the dissemination of research results outside of the agency? f. Notifications on internal agency websites 23.2-34.3 Q9. Has your agency used any of the following means to help you understand how to comply with its policies regarding the dissemination of research results outside of the agency?

Q10. Overall, how effective or ineffective have your agency's efforts been to help you understand how to comply with its policies regarding dissemination of research results though each of the following routes? a. Publications (such as peer reviewed publications or non-peer reviewed publications) 4.4-8.9 Q10. Overall, how effective or ineffective have your agency's efforts been to help you understand how to comply with its policies regarding dissemination of research results though each of the following routes? b. Presentations (such as keynote addresses, conferences, workshops, symposia, professional society meetings, or Congressional hearings) 12.6-22.0 Q10. Overall, how effective or ineffective have your agency’s efforts been to help you understand how to comply with its policies regarding dissemination of research results though each of the following routes? c. Agency releases (such as press releases, web posting on an agency website, or agency reports or fact sheets)

Q10. Overall, how effective or ineffective have your agency’s efforts been to help you understand how to comply with its policies regarding dissemination of research results though each of the following routes? 21.6-33.2 Q10. Overall, how effective or ineffective have your agency’s efforts been to help you understand how to comply with its policies regarding dissemination of research results though each of the following routes? 32.3-84.7 Q11. Regardless of how effective or ineffective your agency’s efforts have been, how confident, if at all, are you that you understand your agency’s policies regarding the dissemination of research results through each of the following routes well enough to follow them?

Q11. Regardless of how effective or ineffective your agency’s efforts have been, how confident, if at all, are you that you understand your agency’s policies regarding the dissemination of research results through each of the following routes well enough to follow them? 8.6-15.6 Q11. Regardless of how effective or ineffective your agency’s efforts have been, how confident, if at all, are you that you understand your agency’s policies regarding the dissemination of research results through each of the following routes well enough to follow them? 29.3-41.7 Q11. Regardless of how effective or ineffective your agency's efforts have been, how confident, if at all, are you that you understand your agency’s policies regarding the dissemination of research results through each of the following routes well enough to follow them?

Q11. Regardless of how effective or ineffective your agency’s efforts have been, how confident, if at all, are you that you understand your agency’s policies regarding the dissemination of research results through each of the following routes well enough to follow them? 33.7-89.7 Q12. Overall, in your opinion, how consistently or inconsistently does your NASA Center, NIST Laboratory, or NOAA Branch/Line Office apply its policies regarding the dissemination of research results through each of the following routes? 0.7-2.8 Q12. Overall, in your opinion, how consistently or inconsistently does your NASA Center, NIST Laboratory, or NOAA Branch/Line Office apply its policies regarding the dissemination of research results through each of the following routes? 7.6-13.4 Q12. Overall, in your opinion, how consistently or inconsistently does your NASA Center, NIST Laboratory, or NOAA Branch/Line Office apply its policies regarding the dissemination of research results through each of the following routes? 1.5-5.0 Q12. Overall, in your opinion, how consistently or inconsistently does your NASA Center, NIST Laboratory, or NOAA Branch/Line Office apply its policies regarding the dissemination of research results through each of the following routes? 4.3-10.1 Q12. Overall, in your opinion, how consistently or inconsistently does your NASA Center, NIST Laboratory, or NOAA Branch/Line Office apply its policies regarding the dissemination of research results through each of the following routes? 0.0 Q13. Overall, in your opinion, how consistently or inconsistently does as a whole apply its policies regarding the dissemination of research results through each of the following routes? 0.5-2.6 Q13. Overall, in your opinion, how consistently or inconsistently does as a whole apply its policies regarding the dissemination of research results through each of the following routes? 7.6-13.6 Q13. Overall, in your opinion, how consistently or inconsistently does as a whole apply its policies regarding the dissemination of research results through each of the following routes? 2.7-7.6 Q13. Overall, in your opinion, how consistently or inconsistently does as a whole apply its policies regarding the dissemination of research results through each of the following routes? 5.1-11.3 Q13. Overall, in your opinion, how consistently or inconsistently does as a whole apply its policies regarding the dissemination of research results through each of the following routes? 35.4-89.0 Q14. Overall, do you believe your agency’s dissemination policies are effective or ineffective in ensuring access to the results of research conducted at your agency?

Q15. How supportive, if at all, is your agency toward disseminating research results through each of the following routes? a. Through publications such as peer reviewed journals, or non-peer reviewed journals 6.2-12.4 Q15. How supportive, if at all, is your agency toward disseminating research results through each of the following routes? b. Through presentations, such as at conferences or at congressional hearings 7.0-14.9 Q15. How supportive, if at all, is your agency toward disseminating research results through each of the following routes? c. Through agency releases, such as press releases, web postings, and agency reports and fact sheets Q15. How supportive, if at all, is your agency toward disseminating research results through each of the following routes? 54.8-68.3 Q16. Overall, which one of the following statements best characterizes the extent to which your agency supports the dissemination of research results? 0.2-2.7 Q17. Does your area of research have the potential to impact federal policy decisions? 10.6-18.7 Q18. When disseminating research results that have potential federal policy implications, does each of the following statements correctly or not correctly represent your agency's position on discussing these research results? a. I can discuss research results with potential policy implications when the results are consistent with the agency's policy or position statements. 12.4-25.5 Q18. When disseminating research results that have potential federal policy implications, does each of the following statements correctly or not correctly represent your agency's position on discussing these research results? b. I can discuss research results with potential policy implications when the results differ from the agency's policy or position statements. 32.8-48.9 Q18. When disseminating research results that have potential federal policy implications, does each of the following statements correctly or not correctly represent your agency's position on discussing these research results? c. I can discuss research results with potential policy implications on issues for which my agency does not have a policy or position statement. 29.5-45.5 Q18. When disseminating research results that have potential federal policy implications, does each of the following statements correctly or not correctly represent your agency's position on discussing these research results? d. I can discuss potential policy implications of research results provided that I state the policy implications as my personal views and not those of the agency. 34.1-50.3 Q18. When disseminating research results that have potential federal policy implications, does each of the following statements correctly or not correctly represent your agency's position on discussing these research results? e. I am not allowed to discuss the potential policy implications of research results. 30.7-47.4 Q19. In the past 5 years, have you sought to disseminate the results of your own scientific research outside of the agency?

Q20. Which of the following reasons describe why in the past 5 years you have not sought to disseminate the results of your scientific research outside the agency? a. I don’t conduct research that requires dissemination 63.9-87.0 Q20. Which of the following reasons describe why in the past 5 years you have not sought to disseminate the results of your scientific research outside the agency Q20. Which of the following reasons describe why in the past 5 years you have not sought to disseminate the results of your scientific research outside the agency? c. Someone else disseminates research results on my behalf 77.0-95.3 Q20. Which of the following reasons describe why in the past 5 years you have not sought to disseminate the results of your scientific research outside the agency? d. I sought to disseminate results in the past and was not allowed to Q20. Which of the following reasons describe why in the past 5 years you have not sought to disseminate the results of your scientific research outside the agency? e. My research is ongoing and is not ready to be released 62.9-87.4 Q20. Which of the following reasons describe why in the past 5 years you have not sought to disseminate the results of your scientific research outside the agency? f. My research is not eligible for public dissemination Q20. Which of the following reasons describe why in the past 5 years you have not sought to disseminate the results of your scientific research outside the agency? g. Agency policies did not allow me to disseminate 78.9-98.2 Q20. Which of the following reasons describe why in the past 5 years you have not sought to disseminate the results of your scientific research outside the agency?

Q21. Over the past 5 years, through which methods did you seek to disseminate your research results? a. Publications (such as peer reviewed publications or non-peer reviewed publications) 0.0 Q21. Over the past 5 years, through which methods did you seek to disseminate your research results? b. Presentations (such as keynote addresses, conferences, workshops, symposia, professional society meetings, or Congressional hearings)

Q21. Over the past 5 years, through which methods did you seek to disseminate your research results? c. Agency releases (such as press releases, web posting on an agency website, or agency reports or fact sheets) 0.6-4.5 Q21. Over the past 5 years, through which methods did you seek to disseminate your research results?

Q21. Over the past 5 years, through which methods did you seek to disseminate your research results? 10.9-34.6 Q22. Other than for standard technical review reasons, over the past 5 years, have you ever encountered a situation when your agency did not allow you to disseminate your research results?

Q23. Through which method(s) were you seeking to disseminate your research results when you were denied agency approval? a. Publication (such as peer reviewed publications or non-peer reviewed publications) 36.2-86.9 Q23. Through which method(s) were you seeking to disseminate your research results when you were denied agency approval? b. Presentations (such as keynote addresses, conferences, workshops, symposia, professional society meetings, or Congressional hearings)

Q23. Through which method(s) were you seeking to disseminate your research results when you were denied agency approval? c. Agency releases (including press releases, web posting on an agency website, or agency reports or fact sheets) 67.5-92.4 Q23. Through which method(s) were you seeking to disseminate your research results when you were denied agency approval?

Q23. Through which method(s) were you seeking to disseminate your research results when you were denied agency approval? 81.2-97.0 Q25. Did you take any of the following actions when you did not receive approval to disseminate your research results? a. Appealed the decision using established procedures Q25. Did you take any of the following actions when you did not receive approval to disseminate your research results? 13.2-66.2 Q25. Did you take any of the following actions when you did not receive approval to disseminate your research results? c. Disseminated the results through a different route Q25. Did you take any of the following actions when you did not receive approval to disseminate your research results? d. Added a disclaimer that the opinions expressed in the research results do not reflect the views of the agency 74.8-99.1 Q25. Did you take any of the following actions when you did not receive approval to disseminate your research results? e. Resubmitted the same or similar document 0.0 Q25. Did you take any of the following actions when you did not receive approval to disseminate your research results?

Q25. Did you take any of the following actions when you did not receive approval to disseminate your research results? g. Received approval following other types of revisions - Please specify below. 0.8-16.8 Q25. Did you take any of the following actions when you did not receive approval to disseminate your research results?

Appendix III: Comments from the Department of Commerce

Appendix IV: Comments from the National Aeronautics and Space Administration

Appendix V: Comments from the Office of Science and Technology Policy

Appendix VI: GAO Contact and Staff Acknowledgments

Staff Acknowledgments

In addition to the contact person named above, Cheryl Williams (Assistant Director), Allen Chan, Nancy Crothers, Elizabeth Erdmann, Stuart Kaufman, Matthew LaTour, Grant Mallie, Lisa Mirel, and Rebecca Shea made key contributions to this report.