Scientific research has traditionally been considered to be valuable to society. The knowledge gained from federally supported research leads to the development of new products and processes. Research results also provide information needed to make policy decisions. The Congress is concerned that quality research is used to provide a rational basis for federal rule- making, as with regulations established to protect human health and safety and the environment. Peer review has been used to judge the quality of science for over 3 centuries. Historically, peer review has been used extensively in the selection of proposed research projects and to a lesser extent to evaluate R&D programs and their likely effects. The Congress and the President have taken steps to help improve the management of and accountability for federal R&D spending. In 1976, the Congress established the Office of Science and Technology Policy, which serves as a source of scientific, engineering, and technological analysis and judgment for the President and assists him in providing leadership and coordination for federal R&D programs. In November 1993, by executive order, the President also established the Cabinet-level National Science and Technology Council to coordinate federal R&D and to establish clear national goals for science and technology investments. Through this Council, the Office of Science and Technology Policy helps the President coordinate science, space, and technology policy and programs across the federal government and leads an interagency effort to develop and implement science and technology policies and budgets across federal agencies. In 1993, the Congress passed the Government Performance and Results Act to encourage efficiency, effectiveness, and accountability in federal programs. The act requires federal agencies to produce strategic performance plans with annual targets and annual performance reports that explain whether those targets have been met. A recent report by the Committee on Science, Engineering, and Public Policy concluded that the most effective means of evaluating federally funded research programs is expert review. The best known form of expert review is peer review; two other forms are relevance review that examines whether the research program focuses on an agency’s mission and international benchmarking that determines whether the research is at the forefront of scientific and technological knowledge. There is no written definition of peer review that applies across the federal government. Officials at the Office of Science and Technology Policy described peer review as a merit-based process or independent merit assessment, generally used in decisions about which research projects to fund. Individual agencies define peer review somewhat differently; however, all of the agencies’ definitions or descriptions of peer review contained the fundamental concept of a review of technical or scientific merit by individuals with sufficient technical competence and no unresolved conflict of interest. Peers generally are considered to be scientists or engineers who have qualifications and expertise equivalent to those of the researcher whose work they review. In addition, peers must be capable of making an independent judgment of the merits and relevance of the research. Officials at the Office of Science and Technology Policy said that peers do not have to be external to the funding agency, as long as there is no unresolved conflict of interest. Officials at the Office of Science and Technology Policy said that there is no governmentwide legislation or policy that requires agencies to conduct peer reviews or dictates how the reviews should be conducted. However, the office advocates the use of peer review and provides guidance to agencies on the use of peer review to assess the quality of research. For example, the office works with the Office of Management and Budget to promote peer review. Beginning with the fiscal year 1996 budget cycle, the two offices have jointly provided annual direction to agencies encouraging them to emphasize the funding of peer-reviewed research over nonpeer-reviewed research. As a result, Office of Science and Technology Policy officials said that agencies have shifted funding toward more peer-reviewed research. The guidance also encourages agencies to solicit proposals from many researchers on how to solve research problems. Officials at the Office of Science and Technology Policy said that agencies’ peer review practices should be flexible and tailored to agency missions and type of research, and that specific uniform practices should not be dictated for every agency or all federally funded research. A variety of peer review methods is viewed by the agencies as both appropriate and essential, reflecting the varying nature of the research and its purposes, the differences in research timelines, the broad spectrum of R&D performers, and the varying funding mechanisms, such as grants, contracts, and cooperative agreements. For example, agency officials noted that specific peer reviews can have quite different purposes such as the review of research results prior to publication versus the review of agency or program performance. While peer review has come to be viewed by some observers as the best assurance that quality criteria will prevail over social, economic, and political considerations, others view peer review as an element of elitism in science that tends to discount such concerns as economic considerations. Officials at the National Science Foundation said that under certain conditions, over emphasis on peer review for funding decisions may discourage funding of innovative research because peers generally tend to view research somewhat conservatively. Officials at the Office of Science and Technology said that this phenomenon has been noted by other agencies, and the Office is initiating an interagency effort to examine how peer review practices (and other federal R&D policies) might better facilitate innovation. “Science agencies must devise assessment strategies that are appropriate to the nature of scientific processes and to the enabling role of fundamental science in support of over-arching national goals... should be designed to...respond to surprises, pursue detours, and revise program agendas in response to new scientific information and technical opportunities essential to the future well-being of all our people.” All the agencies that we contacted identified policies, orders, or other internal guidance regarding the conduct of peer review. Some of these policies are legislatively mandated. For example, the Agricultural Research, Extension, and Education Reform Act of 1998 requires the U.S. Department of Agriculture to peer review Department-funded research.The law also requires grantees to arrange for a peer review of special grants mandated by the Congress. Overall, we found that eight agencies—the Agricultural Research Service, the Cooperative State Research, Education, and Extension Service, the National Institute of Standards and Technology, the National Institutes of Health, the Department of Energy, the Environmental Protection Agency, the National Aeronautics and Space Administration, and the Federal Aviation Administration—have laws or regulations that require peer review of competitively selected grant proposals. The peer review practices differ among and within federal research agencies in two ways. First, agencies use peer review in varying degrees to assess the merit of research at different stages in the research process, including selecting research projects for funding; monitoring in-progress research; and evaluating research products prior to publication. Second, the implementation of the peer review process varies. The following highlights the extent to which agencies use peer review and some of the various ways they implement peer review. Appendixes I through XII provide descriptions of the agencies’ peer review practices, to the extent that peer review is used, for each of the 12 agencies included in our review. All the agencies conduct peer reviews to help determine which competitive research proposals to fund. All 12 agencies also use peer review to help determine funding for at least a portion of their other research, including peer reviews of the agencies’ intramural or internal research proposals or plans. The agencies use a combination of external and internal reviewers with subject matter expertise. However, Federal Aviation Administration officials said that their peer reviewers are primarily agency employees who are not involved in the project but have the required subject matter expertise. The agencies conduct the peer reviews by mail, panels or committees, or a combination of methods. They also differ in the number of reviewers used in the process. For example, the Cooperative State Research, Education, and Extension Service uses panels of outside experts to review competitive research proposals. One Cooperative State Research, Education, and Extension Service program, the National Research Initiative, uses panels that meet throughout the year. Each panel member reviews about 30 research proposals and provides written comments on about 20 of the proposals. Mail reviews and ad hoc reviewers are also used when additional expertise is needed. In contrast, the National Aeronautics and Space Administration generally conducts reviews by mail to obtain specialized expertise on technical issues and approaches, followed by reviews by panels of 7 to 10 experts with broader perspectives to reconcile differences among the mail reviewers’ comments. The agencies use various criteria to assess proposed research, including technical or scientific merit, relevance to agency mission and priorities, and the qualifications of the researcher. Agency officials responsible for selecting research for funding generally consider the peer reviewers’ advice or recommendations along with other financial and management factors to make their funding decisions. In some circumstances, agencies use these same types of peer review processes to assess the merit of research that is not funded through competitive selection, generally this research is internal to the agencies. For example, the Forest Service conducts peer reviews of its scientists’ research study plans prior to approval by project leaders. The U.S. Geological Survey conducts peer reviews of all intramural project proposals to ensure technical quality prior to final approval and implementation. At the Agricultural Research Service, internal peers, who are not involved in the funding decision, review detailed project plans. While all 12 agencies provided examples of peer reviews of research that are in-progress, there is much variation in the frequency and purpose of those reviews. The purposes of these reviews include assessments of research projects to determine if funding should be renewed or to assess the progress of on-going research at the program level, or at research stations or laboratories. For example, in-progress Cooperative State Research, Education, and Extension Survey projects that are assessed for renewal within 2 to 3 years compete with new proposals in the same merit review process. If the review panel considers the research progress to be unsatisfactory, the research project will not receive additional funding. The National Aeronautics and Space Administration requires that for any project that continues for more than 3 years, the researcher must submit a new proposal, which is subject to external peer review. The National Institutes of Health uses boards of outside experts to peer review its on-going intramural research, and the Agricultural Research Service convenes panels in a workshop format to review intramural research projects.