Agricultural Research: Two USDA Agencies Can Enhance Safeguards against Project Duplication and Strengthen Collaborative Planning
Released: 2013-05-13T00:00:00

Letter

Background

The United States’ publicly funded agricultural research system is a critical component in finding answers to urgent global problems, including establishing sustainable food production, providing clean and abundant water, and strengthening food safety (see fig. 1). This public system is based on a federal-state partnership in which USDA and the states, through their land-grant colleges and universities, play multiple roles in agricultural research, extension, and education. According to a 2011 USDA report, studies have shown that private research productivity depends on government and university investments in science and technology. The federal, state, and private sectors provided 27 percent, 12 percent, and 61 percent, respectively, of total U.S. agricultural research funding in 2007, according to the most recent USDA estimate.

The federal portion of this public research system, with roots dating back to the 19th century, is overseen by USDA’s REE mission area. The REE mission area comprises the following four agencies:

ARS, USDA’s key intramural research agency;

NIFA, USDA’s key extramural research agency; the Economic Research Service, USDA’s key intramural economics and social science research agency; and the National Agricultural Statistics Service, focused on agricultural statistics.

The Food, Conservation, and Energy Act of 2008 (2008 Farm Bill) reorganized this mission area to include an umbrella coordinating unit within the Office of the Under Secretary for REE. The 2008 Farm Bill also designated the Under Secretary for the REE mission area as USDA’s Chief Scientist, and USDA also refers to this umbrella unit as the Office of the Chief Scientist. Accordingly, the Chief Scientist is to ensure, among other responsibilities, that USDA’s research, education, and extension activities are coordinated and integrated across agencies and disciplines. In addition, six division chiefs—commonly referred to by USDA as senior advisors—are to assist the Chief Scientist in identifying and addressing emerging needs and priorities in research, education, and extension.

Mandated by the 2008 Farm Bill, the Under Secretary of REE (i.e., USDA’s Chief Scientist) developed a Roadmap for USDA Science in 2010 to set USDA’s research, education, and extension agenda. This document identified five USDA priorities: bioenergy, climate change, food safety, international food security, and child nutrition. The Roadmap stated that solutions to many of modern society’s most intractable problems demand change that USDA would bring about by, for example, (1) better coordinating its science planning among and between REE science agencies and with other federal science agencies; (2) listening to the needs of stakeholders; and (3) institutionalizing outcome-driven scientific program planning and implementation. The 2008 Farm Bill specified that the Roadmap was not to be subject to review by any federal employee other than the Secretary of Agriculture or his designee.

In early 2012, the Chief Scientist followed up on issuance of the Roadmap with a document titled Research, Education, and Economics Action Plan (REE Action Plan). The REE Action Plan identifies seven goals that support top priorities for USDA science: (1) local and global food supply and security, (2) climate and energy needs, (3) sustainable use of natural resources, (4) nutrition and childhood obesity, (5) food safety, (6) education and science literacy, and (7) rural prosperity and rural-urban interdependence. Each of these seven goals has strategies and planned actions designating the specific USDA agencies that are to be responsible for implementing the actions (see fig. 2 for an excerpt). In addition, the first three goals are divided into multiple subgoals, each of which also has planned actions, or actionable items, designating the specific USDA agencies that are to be responsible.

Within REE, research conducted or funded by ARS and NIFA, respectively, may be basic or applied. According to USDA’s website, basic research discovers the underlying processes and systems that make a plant, animal, ecosystem, food system, community, or marketplace work. For example, basic research might seek to discover the genetic map of a plant or animal or to show how economic and human resources affect economic growth in rural areas. In contrast, applied research expands on basic research findings to uncover practical ways in which new knowledge can be advanced to benefit individuals and society. For example, in an applied research project, researchers might use a genetic map constructed through basic research to develop methods to combat plant and animal diseases. NIFA-funded research may also be integrated, in that it involves education and extension activities, as well as research.

Both ARS and NIFA solicit stakeholder input. Agency stakeholders consist of individuals and institutions that conduct or use agricultural research, education, and extension services provided by or for the agencies. These stakeholders include representatives of commodity groups, industry, interagency federal working groups, scientific societies, and university partners. Stakeholders also include members of the REE advisory board, which comprises 25 members, each representing a specific stakeholder category, such as a national nutritional science society or national farm organizations. ARS itself is considered a NIFA stakeholder because ARS scientists are eligible to compete for certain NIFA grants. In turn, NIFA is a stakeholder of ARS.

USDA differs from other federal science agencies in that the majority of its annual research appropriation supports research done in-house, or intramurally; in contrast, most federal science agencies primarily fund research done elsewhere, or extramurally. As USDA’s principal intramural research agency, ARS had over 2,200 research scientists in fiscal year 2012, who carried out about 800 research projects at more than 90 facilities nationwide and in a few foreign countries. To manage its 18 national research programs, ARS has developed a 5-year research cycle, consisting of four activities: stakeholder input, planning, implementation, and assessment. About 25 ARS national program leaders define and articulate the scope and objectives of each of these national programs with input from their various stakeholders; ARS scientists and managers; and others, including Congress. A key means of acquiring stakeholder input is through a meeting with stakeholders conducted at the start of each 5-year cycle. During this meeting, ARS gathers stakeholder input and defines which research problems it will address in the next 5 years to help meet high-priority needs within the agricultural community. Each national program uses this stakeholder input to help develop a research plan.

NIFA is USDA’s primary extramural research agency, and, unlike staff of ARS, who conduct their own research, NIFA staff do not conduct research. Rather, NIFA’s 350 staff members, including about 70 national program leaders, help fund research at the state and local level, primarily through grants, and provide program leadership to state and local officials in research, education, and extension. Recipients of NIFA funds include 59 state agricultural experiment stations (i.e., the research arm of land- grant colleges and universities), 63 schools of forestry, 18 historically black land-grant colleges, 27 colleges of veterinary medicine, and other institutions and individual scientists. Figure 3 identifies the locations of ARS facilities, along with the land-grant colleges and universities established under the Morrill Act of 1862, those established under the Second Morrill Act of 1890 (historically black land-grant colleges), and those established under the Equity In Educational Land-Grant Status Act of 1994 (tribal colleges).

Congressional legislation establishes the funding levels, eligibility requirements, and scope for each of NIFA’s research, education, and extension programs, and NIFA’s national program leaders manage and oversee these programs. NIFA generally funds grants for 1 to 4 years. In addition to direction from Congress, stakeholder input informs the development of NIFA requests for applications to its competitive grants programs. Each request generates from 2 to 600 applications, each of which must undergo external peer review by a panel of 3 to 25 individuals with expertise in the relevant field. The panels evaluate each application for scientific merit, how well the application addresses the research priorities, and the work’s potential to make a positive scientific impact. The panels then rank the applications and recommend the top projects for eventual funding. NIFA awards the following types of grants for distributing its research funds:

Formula grants (commonly referred to as formula funds). Through several program authorities, NIFA provides funds for research and extension to land-grant colleges and universities, schools of forestry, and schools of veterinary medicine. The amount of funding provided to each institution is determined by formulas, often statutorily defined, which may include variables such as rural population, farm population, and poverty levels. According to NIFA’s website, local or regional university leaders decide which specific projects are to be supported by an institution’s formula grant allotment. As a result, according to a 2011 Congressional Research Service report, specific research priorities for formula grants tend to be more geographically focused than for NIFA’s other grants.

Competitive grants. NIFA awards nearly 40 types of competitive grants for research, extension, and higher education, as well as for projects that integrate all three activities. Competitive grants are designed to enable USDA to (1) attract a wide pool of applicants to work on agricultural issues of national interest and (2) select the best- quality applications submitted by highly qualified individuals, institutions, or organizations. Eligibility, administrative rules, and procedures vary for each specific competitive grant program according to authorizing statutes, and NIFA awards funds following external peer review of the applications. NIFA’s flagship competitive grant program is the Agriculture and Food Research Initiative, which funds research, education, and extension in the food and agricultural sciences and, according to NIFA’s website, awards grants to address agricultural problems of national, regional, or multistate importance.

Noncompetitive grants. Before fiscal year 2011, according to the 2011 Congressional Research Service report, appropriated NIFA funds were directed by Congress to support a designated institution or set of institutions for particular research, education, or extension topics of local, regional, or national priority. These projects were supported through “special research grants” or “direct federal administration research or education grants.” Recent appropriations have not included such spending items, allowing NIFA to award more of its funds through competitive grants, according to USDA officials.

Even with growing demand for agricultural research over the past decade, total appropriations for ARS and NIFA decreased when adjusted for inflation. Specifically, from fiscal year 2002 through fiscal year 2011, funding for ARS decreased by 23 percent, from $1.2 billion to $934 million, and funding for NIFA increased by more than 28 percent, from $1.1 billion to $1.3 billion. When adjusted for inflation, however, ARS and NIFA funding decreased by nearly 42 percent and 3 percent, respectively.

To document USDA’s publicly funded activities, including research conducted or funded by the department, and to provide ready access to information about its research projects and related work, USDA in 1966 developed CRIS. According to officials from NIFA, which is responsible for managing this system, CRIS contains descriptive project-by-project information for all completed and ongoing projects carried out or funded by ARS and NIFA, respectively, and by some other USDA agencies and state institutions. USDA scientists, stakeholders, and the public can search CRIS for ARS and NIFA projects, as well as for some information on related publications and researchers. USDA has expanded CRIS over time to also cover a number of education, extension, and integrated activities. According to NIFA’s website, a part of CRIS’s mission is to help users do the following: keep abreast of the latest developments in agricultural and forestry research and education, plan future activities, avoid costly duplication of publicly funded work, and establish valuable contacts within the research and education community.

Concerning collaboration among federal agencies, in March 2000, we reported that agencies face a range of barriers when they attempt to collaborate with other agencies. Faced with these barriers, federal agencies carry out programs in a fragmented, uncoordinated way, resulting in a patchwork of programs that can waste scarce funds, confuse and frustrate program customers, and limit the overall effectiveness of the federal effort. Subsequently, in an October 2005 report, we discussed key practices that can help enhance and sustain collaboration among federal agencies. Among the practices we identified were (1) defining and articulating a common outcome; (2) defining and agreeing on roles and responsibilities; (3) establishing mutually reinforcing or joint strategies; (4) identifying and addressing needs by leveraging resources; and (5) establishing compatible policies, procedures, and other means to operate across agency boundaries. In our 2005 report, we stated that, while we generally believe that the application of as many of these practices as possible increases the likelihood of effective collaboration, we also recognize that agencies work together in a wide range of situations and circumstances.

ARS and NIFA Use Agency and Scientific Safeguards to Prevent Duplication of Research Projects in Similar Topic Areas, but Some Shortcomings Exist

ARS and NIFA generally focus on many of the same broad topics, and the agencies rely on agency-developed safeguards, as well as on the scientific community’s professional norms, to prevent duplication of research projects within and between the agencies. Indeed, agency officials and stakeholders could provide no recent examples of duplication within or between the two agencies, and our review of 20 randomly selected agency research projects did not identify duplicative projects. Nevertheless, our review identified some shortcomings in certain safeguards the agencies rely on—shortcomings that somewhat limit the utility of these safeguards in preventing duplicative research.

ARS and NIFA Generally Focus on the Same Broad Topic Areas

ARS and NIFA generally focus their work on the same broad topic areas. Our analysis of each agency’s research spending for fiscal years 2002 through 2011 shows that during this period, ARS performed and NIFA funded scientific activities in largely the same six broad topic areas: (1) plant systems; (2) human nutrition; (3) animal systems; (4) natural resources; (5) food and nonfood products; and (6) agricultural, natural resource, and biological engineering. For example, in fiscal year 2011— the most recent year for which data from the agencies were available— both agencies performed or funded work on plant systems (ranging from plant genetics to plant pests) more than on any other topic area, representing about 41 percent of ARS’s total $1.1 billion research spending and about 23 percent of NIFA’s total $1 billion fiscal year 2011 research spending (see table 1). ARS and NIFA both focused the next- largest shares of their spending in fiscal year 2011 on human nutrition, animal systems, and natural resources. NIFA also funded activities in two topic areas on which ARS did limited or no research from fiscal year 2002 through 2011: (1) economics, markets, and policy and (2) families, youth, and communities. Similarly, within some topic areas, ARS conducted research on certain issues that NIFA did not fund. For example, ARS includes six human nutrition research centers that conduct studies on a wide range of topics, such as the nutrient content of commonly eaten foods and food choices in schools.

ARS and NIFA also share responsibility for many actions needed to address REE goals as identified in its 2012 REE Action Plan. Our analysis of the 2012 REE Action Plan shows that ARS and NIFA share responsibility for two-thirds of the 139 planned actions delineated as involving either agency or both. Specifically, the action plan identifies ARS and NIFA as jointly responsible for 92 (66 percent) of the 139 planned actions. For the remaining 47 planned actions (34 percent), ARS or NIFA is solely or partially responsible. These planned actions align with strategies REE identified as needed to address its seven goals. For example, to help address its nutrition and childhood obesity goal, the action plan discussed a strategy to develop and extend approaches for preventing obesity and related diseases, and it designated both ARS and NIFA as responsible for determining the causes and consequences of obesity and related conditions.

The Agencies Rely on Various Safeguards to Prevent Duplication of Research Projects

ARS and NIFA use various safeguards to prevent duplication of research projects within and between the agencies. Specifically, each agency has built its own safeguards against project duplication into processes for developing and reviewing research proposals. The agencies also rely on the scientific community’s professional norms, which discourage duplicating the work of other scientists, even in similar topic areas. Indeed, agency officials and stakeholders could provide no recent examples of duplication within or between the two agencies, and our review of 20 randomly selected agency research projects did not identify duplicative projects.

ARS has developed several internal agency safeguards to help prevent duplication, which operate at different stages in the agency’s 5-year research cycle. The agency implemented this research cycle to help ensure the relevance, quality, and impact of its research. As part of the cycle, leaders from ARS’s 18 national programs assign research objectives to scientists in the field, and to meet those objectives, these scientists, in turn, work with national program and local area management to develop project plans for the forthcoming 5-year period. Project plans are reviewed by independent experts in the relevant field of science, ARS managers and officials, including those in ARS’s Office of Science Quality Review. The key safeguards ARS developed to help prevent duplicative research projects include the following:

National program leaders set research project objectives. ARS’s national program leaders set the objectives for all proposed research under the agency’s 18 national programs and review each proposal to ensure that research plans align with assigned objectives. Specifically, national program leaders work with local managers to translate the research priorities of each national program into research objectives for the next 5 years. According to ARS officials, this step serves to prevent research duplication because national program leaders who write and assign the objectives can see if, for two different research projects, they are writing objectives that are too similar, and these national program leaders can rework objectives to be distinct.

Scientists develop project plans using specific guidance. Teams of ARS scientists then use the objectives approved by the national program leaders to develop project plans—research proposals for either a single assigned research objective or a group of assigned objectives. These project plans are to discuss the nature of the problem to be addressed and detail experimental approaches for examining the problem. According to USDA officials, each national program has multiple project plans, each addressing different objectives within the national program. ARS guidance states that each project plan must (1) discuss gaps in knowledge that proposed research is intended to address and (2) describe how proposed research relates to, and complements, other related research within and outside of USDA. Together, these required discussions of gaps in knowledge and relation to other research help the agency identify and prevent approval of potentially duplicative research projects within and outside of ARS.

National program leaders review project plans. Throughout development of each project plan and after a plan is complete, ARS national program leaders and local managers review the proposed research to ensure that the final objectives are consistent with the ones assigned. This process can identify when two research proposals are too similar or when a proposal has the potential to duplicate other proposed or existing work. For example, one scientist told us that his national program leader noticed that his proposed applied microbiology research was very similar to work already being done at another ARS laboratory. As a result, this scientist ceded his research to the other laboratory, thus averting duplication within ARS.

External peer review by independent experts. In response to the Agricultural Research, Extension, and Education Reform Act of 1998, which requires the establishment of procedures for external peer review every 5 years of all research conducted by ARS, ARS scientists now submit completed project plans for review by an external panel of scientific experts before ARS approves funding for any research. According to ARS officials, successful completion of external peer review as a prerequisite for ARS’s funding of its own intramural research was a significant change for the agency. ARS’s Office of Science Quality Review coordinates this external peer review, in which external peer review panels—comprising non-ARS scientists selected for their expertise in a field pertinent to the research proposals under review and coming from different places, such as universities, industry, or other federal agencies including NIFA—independently assess the quality of ARS’s proposed research, including specific assessment of each proposal’s significance and relevance. According to agency officials, these external peer review panels each typically review from 4 to 12 ARS project plans and evaluate individual project plans for adequacy of their experimental approaches, probability of their success in achieving stated objectives, and significance of the proposed research. On the basis of recommendations from the peer review panels, officials from ARS’s Office of Science Quality Review certify project plans that successfully pass peer review or decline to certify plans that fail to adequately address peer reviewers’ concerns. ARS’s best practices on peer review state that the agency’s project plans must provide assurance that proposed work addresses a unique gap in knowledge and is not duplicative of ongoing or completed ARS work. Moreover, ARS guidance states that ARS scientists are to demonstrate to peer reviewers that they are aware of others performing similar research and to show how ARS’s proposed research is unique. Successful completion of ARS’s external peer review means that a proposed project has scientific merit and addresses relevant gaps in knowledge—including, in essence, that the proposed research is not duplicative.

CRIS search for duplication. ARS guidance directs the agency’s scientists to search CRIS for potentially duplicative projects when preparing their project plans. As part of “due diligence,” ARS scientists must show that research proposed in their project plans is relevant and unique, and ARS guidance directs the scientists to survey relevant literature, including searching CRIS. According to ARS officials, the purpose of this CRIS search is to see if ARS or NIFA has already funded the research the scientists are proposing. ARS officials responsible for managing external peer review commented that this CRIS search is typically the best method for identifying potentially duplicative government research. In addition, the officials told us that reviewers expect to see that such a CRIS search has been done and will say if they feel that the search has not been done adequately. ARS officials told us that this CRIS search generally consists of using keyword lists associated with each project in CRIS and names of scientists leading the work to identify projects that might be too similar or have overlapping objectives. For example, an ARS team proposing to work on avian influenza found through its CRIS search that NIFA had funded a grant on various avian influenza strains; after reading the objectives of that grant, however, the team determined that, although similar, these objectives were not the same and that the ARS team’s research would be substantively different and not duplicative.

Like ARS, NIFA has also built several agency safeguards into its reviews of proposed work, which help prevent the funding of potentially duplicative research projects. These safeguards operate in NIFA’s competitive grants programs and, to some extent, in the provision of formula funds to the states. The key safeguards NIFA relies on to help prevent duplicative projects include the following:

External peer review of competitive grant applications. NIFA uses panels of external scientists—typically comprising 3 to 25 experts in the field pertinent to the applications being reviewed—to assess the strengths and weaknesses of all competitive grant applications and to confirm that proposed work is original. These panels generally review each application on the basis of scientific merit and how well the application addresses, among other things, NIFA’s research priorities. For example, criteria for the agency’s Agriculture and Food Research Initiative include, among other factors, originality of proposed research and relevance to NIFA priorities, including the importance of the proposed topic to agriculture. As part of their assessments, peer review panels check that the proposed work is not already funded by other grant-making agencies. According to USDA officials, panelists rely on their own familiarity with work in their fields to determine whether a proposed project duplicates existing work in those fields.

Application forms for competitive grants. According to agency officials, they use the application process to help identify and discourage potential duplication. NIFA requires competitive grant applicants to submit two forms, each of which serves as a partial check against potentially duplicative projects. The first form, called the Current and Pending Support form, requires a project’s lead researcher to document all sources of funding requested or received for his or her active and pending projects. The second form requires applicants to verify whether their application has also been submitted to other agencies for funding and states that submitting the same application in response to more than one NIFA request for application is not permissible. According to NIFA officials, information in these forms can alert the peer review panel or national program leader reviewing applications to instances where the research proposed in an application under review has already been funded, or is currently being considered for funding, by NIFA as part of a separate application and thus serves as a safeguard against duplication.

CRIS duplication check for competitive grants under the Agriculture and Food Research Initiative. NIFA’s national program leaders direct staff to conduct a CRIS duplication check, using relevant keywords and the names of lead scientists, before awarding funds for any application submitted to this key competitive grant program. NIFA guidance on reviewing Agriculture and Food Research Initiative applications states that national program leaders must check CRIS to determine if the proposed work has already been funded by NIFA or ARS and to ensure that it is not unnecessarily repeating work not yet published. National program leaders must also check if the proposed work is receiving funds from two different sources for essentially the same work. NIFA guidance states that the Agriculture and Food Research Initiative will not fund projects that overlap with work already being funded by USDA or work that is unnecessarily duplicative.

National program leaders are to document this CRIS search and attest to the proposed work’s originality and uniqueness.

Review of formula funds for innovation. Even though formula-funded projects do not undergo external peer review, and NIFA’s national program leaders are not required to check CRIS for duplication when reviewing formula-funded projects, agency guidance emphasizes that work proposed for formula funds be relevant and innovative. NIFA guidance states that, as part of their general quality review, national program leaders should consider whether the proposed research addresses an important problem. In addition, NIFA guidance directs national program leaders to assess whether the proposed work is innovative, by asking whether it has the potential to add to the body of scientific knowledge by, among other things, addressing new issues or taking an innovative approach. This assessment in effect evaluates whether the proposed project is potentially duplicative of other research and thus serves to safeguard against duplication.

In addition to the agency safeguards they have developed, ARS and NIFA rely on several of the scientific community’s professional norms to help safeguard against duplicating research projects. According to agency officials, scientists we spoke with, and the literature we examined about science, the agencies rely on three such professional norms:

Scientists have a strong incentive to produce original research and to make new discoveries. According to scientists and stakeholders, scientists are rewarded for producing new or original research, not for re-creating existing work; grants are not awarded for duplicative research. Scientists we spoke with often told us that a scientist would be penalized professionally for conducting research that unnecessarily duplicated another’s work. According to the 1991 National Academies report, a premium is placed on making breakthrough discoveries, and penalties may accrue for doing unnecessarily duplicative work, including difficulty in obtaining federal or private funding for future research.

Scientists must stay abreast of new research in their field to stay competitive. According to numerous scientists and USDA officials we spoke with, scientists must stay abreast of new research in their fields to incorporate novel ideas into their research, compete for research funding, and advance in their careers. Scientists keep current by reading professional journals; attending scientific conferences and related gatherings, where they and their peers present new findings; and communicating with colleagues worldwide on a continual basis. Moreover, scientists must successfully publish the results of their work in scientific journals to be competitive for new funding and promotions at the university level. This practice is also true for scientists at ARS, according to several ARS officials we spoke with. In fact, several scientists we spoke with echoed the expression “publish or perish,” underscoring the need for scientists to publish new findings in journals to remain competitive. According to USDA officials, the requirement to publish in peer-reviewed journals is an element in every ARS scientist’s annual performance plan, and a chronic failure to publish is grounds for termination of employment. Staying in touch with other scientists through online networks, conferences, and scientific publications reduces the likelihood of duplication.

The peer review process used by scientific journals limits the publication of unnecessarily duplicative research. As described in the literature we examined and illustrated by scientific journals’ instructions to authors and reviewers, peer review is the means journals use to vet the accuracy and quality of articles before they are published. Scientists submit articles detailing their research findings to a publication of their choice, which then sends the articles to several scientists working in the research area discussed by the article. These reviewers provide feedback on submitted articles and recommend to the publication’s editor whether they think the research merits publication. As explained in the literature we examined, only articles that meet certain scientific standards—including building on and not duplicating other work in the research area, as well as being sufficiently original, significant, and valid to merit the attention of other scientists—are accepted for publication. In effect, according to the 1991 National Academies’ report, the peer review process serves to limit unnecessary duplication of research.

USDA officials we spoke with stated that duplication of research projects had occurred in previous years, especially before ARS implemented its external peer review process, and NIFA shifted to more competitive grants. Moreover, in a March 2012 hearing, the Secretary of Agriculture testified before Congress that some duplication in research projects may exist today. During our review, however, USDA officials and stakeholders could provide no recent examples of duplication within or between ARS and NIFA, and many stakeholders told us that duplicative research projects were not a problem for ARS and NIFA today. In addition, we did not identify duplicative projects within or between the two agencies during our review of selected research projects. Some stakeholders and USDA officials called our attention to instances where they believed ARS or NIFA supported research that appeared duplicative but that, upon further investigation, was found to be complementary. For example, ARS researchers who used the agency’s base funding to study the potential of ultraviolet processing to improve the healthfulness of foods received a NIFA grant under the Agriculture and Food Research Initiative to investigate whether ultraviolet light improved the nutritional properties of a wide range of specialty crops, such as berries and apples. We found that this work built on previous ARS research in mushrooms in a way that extended ARS’s research, rather than duplicating it. Other instances called to our attention were, in fact, cases where industry intentionally funded research duplicating what ARS or NIFA was already funding to see which team would find the better solution. Further, we found no instances of actual duplication within or between ARS and NIFA during our review of 10 ARS and 10 NIFA research projects (see app. I for more information).

Some Shortcomings May Limit the Utility of Certain Agency Safeguards

Our review also identified some shortcomings with certain agency safeguards, which may somewhat limit their usefulness and increase the potential risk of duplication between or within the two agencies. Three of these shortcomings limit CRIS’s utility to guard against duplication of research projects, specifically, (1) ARS data in CRIS are not current; (2) NIFA directs staff to conduct a CRIS duplication check for what amounts to two-thirds, but not all, of its competitive grants; and (3) CRIS searches are not user friendly.

According to USDA officials, the effectiveness of CRIS in preventing duplicative research projects depends on the system’s information on new, ongoing, and completed research. Our analysis of agency safeguards, however, showed that project information in CRIS may not be current because ARS has not always sent its project information in a timely manner to be entered into CRIS by NIFA, which maintains the system. ARS sends information in batches once or twice per fiscal year on its newly started or revised projects for uploading into CRIS. In contrast, NIFA officials stated that they upload information on new and revised NIFA projects continually throughout the year. According to ARS officials, the agency sent its project information for uploading only once in fiscal year 2011 (June), twice in fiscal year 2010 (February and July), and twice in fiscal year 2012 (February and June). According to ARS officials, any given batch contains data on all research projects started or revised since ARS sent its last batch of project data. For example, the July 2010 batch contained information on all ARS projects started or revised since March 1, 2010 (see fig. 4). All together, we found that information in CRIS about ARS projects started from February 2010 to December 2012 was typically at least 6 months out-of-date; at least one new project we identified was nearly 21 months old when ARS sent information about it to NIFA for uploading into CRIS.

ARS project information that is typically at least 6 months out-of-date when uploaded limits CRIS’s utility as a safeguard to prevent duplication of research projects and increases the risk of inadvertent project duplication at the two agencies. For example, a university scientist or NIFA national program leader searching CRIS for possible duplication could inadvertently propose or approve a project that unnecessarily duplicates an ARS project that might have started nearly a year before but had not been posted in CRIS. Agency information technology staff and managers agreed that sending ARS project data more often—such as quarterly—would be feasible, cost no more than 1 additional staff day of work, and increase CRIS’s utility as a safeguard to prevent project duplication by keeping ARS’s project information up-to-date. NIFA and ARS officials also acknowledged that requiring submissions of project information at least quarterly would help prevent other workload priorities from delaying ARS submissions. After discussing this topic with us, agency officials said that ARS now expects its staff to send ARS project information on a quarterly basis to NIFA for uploading. This expectation, however, is not documented in guidance. Under federal standards of internal control, agencies are to clearly document internal control in writing, in the form of management directives, administrative policies, or operating manuals.

NIFA directs its staff to conduct a CRIS duplication search for certain projects accounting for about two-thirds of the funding it awarded for competitive grants over the last decade. In response to a 2008 USDA Inspector General’s recommendation specifically directed at grant applications for the predecessor to the Agriculture and Food Research Initiative, NIFA developed written guidance directing that such a duplication check specifically be done for Agriculture and Food Research Initiative grants. That recommendation advised the agency to develop written guidance—rather than rely on informal, unwritten guidance—for these types of grants. As a result, NIFA guidance states that its national program leaders must search CRIS for possible duplication when reviewing project applications recommended for funding under its Agriculture and Food Research Initiative; these projects, however, accounted for about two-thirds, but not all, of the agency’s competitive grant funding. The duplication search directive does not apply to projects funded by NIFA’s other competitive grants or to formula-funded grants. The amount of funding for competitive grants not subject to this directive has increased each year from about 31 percent of the funds the agency awarded for competitive grants in fiscal year 2002 to more than one-third, or about 37 percent, of the funds the agency awarded for competitive grants in fiscal year 2011. Thus, an increasing amount of the agency’s competitive grant funding has been awarded to projects without the benefit of this safeguard. In contrast, ARS’s CRIS search directive applies to all project plans proposed by ARS.

Although NIFA responded to the 2008 USDA Inspector General’s recommendation by issuing written guidance directing staff to conduct CRIS duplication checks for the Agriculture and Food Research Initiative, it has not issued similar written guidance for the rest of its competitive grant programs. According to NIFA officials, many but not all national program leaders informally conduct CRIS duplication checks when reviewing competitive grant applications that are not part of the initiative, even though no written guidance explicitly directs them to do so. The agency recently convened a task force to study, among other issues, whether this directive should be extended to other competitive grants. At the time of our report, the task force was still studying expanding the CRIS duplication search to all competitive grants and had not yet made a decision. Under federal standards for internal control, agencies are to clearly document internal control in writing, and this documentation should be readily available for examination. Without written guidance directing staff to conduct CRIS duplication checks for all of NIFA’s competitive grant programs, the agency does not have an internal control in place to ensure consistency among national program leaders in checking for potential duplication when reviewing applications. As a result, NIFA cannot be assured that grant funds are consistently used for supporting unique work.

In addition, NIFA does not conduct CRIS duplication checks on formula- funded grants, meaning that research and related work proposed in the states’ plans of work are also not subject to this safeguard. According to NIFA officials, the agency’s formula funds are often used for researcher salaries and preliminary research, and CRIS duplication checks are not typically applicable. For example, the officials said, enough detail is seldom contained in these high-level plans of work to check CRIS for potential duplication on research objectives. Moreover, many projects proposed for formula funds, such as component parts of multistate projects, are all required to have the same title; consequently, a search in CRIS on any one of these would flag the others as being potentially duplicative, even though they are complementary and intentionally titled the same.

USDA officials, including those responsible for system management, acknowledge that CRIS is not user friendly, especially for project searches. Specifically, they reported that CRIS is not straightforward to search; links to authors, publications, and patents are limited or nonexistent; and researcher names are listed inconsistently. For example, as explained by agency officials, one must go through several nonintuitive steps to search all text in the system for specific keywords or subjects, such as “bark beetles” or “bioenergy.” In addition, because CRIS does not have a standard way of listing researcher names, someone using CRIS must enter a single researcher’s name into the system in multiple ways— such as Doe, J., and Doe, J. A.—to be certain of finding all projects by that one researcher. Consequently, CRIS searches are not always efficient, which, according to some scientists, has discouraged or kept researchers from using the system for duplication searches. NIFA has been trying to upgrade CRIS to make it less cumbersome but, according to USDA officials, it has not secured the resources to do so.

To provide additional capabilities for accessing and disseminating federal agricultural research, USDA’s Chief Scientist is implementing a new web platform called VIVO, which, according to USDA officials, has powerful search functionality for locating scientists and their research. According to USDA officials, a key purpose of VIVO—which is expected to become operational in 2013—is to serve as a one-stop source for USDA science expertise by integrating project information from all five USDA science agencies, thereby allowing USDA scientists to find possible research partners and allow those outside USDA to easily find USDA expertise in specific scientific topics (e.g., honeybees) or geographic areas (e.g., the Pacific Northwest). Unlike CRIS, VIVO is expected to contain not only information on ARS and NIFA projects but also active links to scientists and associated publications, organizations, patents, projects, and locations and to scientists’ contact information. VIVO also lists scientists—authors and coauthors—in a standardized format. Further, USDA officials stated that VIVO will reflect any new improvements made in the quality of project information fed into CRIS from other USDA project information systems because these systems of record are the source of VIVO’s data. For example, VIVO is expected to reflect improvements under way at NIFA to standardize researchers’ names in CRIS. In addition, USDA officials stated that they expect a related initiative known as STAR METRICS, a collaboration between federal science agencies and research institutions to coordinate federal investments in science, to build on VIVO and use the data to, among other things, illuminate research gaps and potentially duplicative research projects at ARS and NIFA, as well as at other federal science agencies. USDA officials agreed that VIVO has the potential to be a more robust and user-friendly system than CRIS in helping to identify USDA scientists and their work and, thus, in identifying potentially duplicative research projects.

Under policy issued by the Office of Management and Budget in 2000, agencies are to ensure that the information systems they select (1) maximize the usefulness of information and minimize any burden on the public and (2) do not unnecessarily duplicate existing information technology capabilities within the same agency, from other agencies, or from the private sector. USDA officials have identified potential benefits of VIVO for both elements of this 2000 policy because of VIVO’s more robust search capabilities. They said, however, that the department has no plans to assess whether VIVO significantly improves search capabilities compared with CRIS or is more effective or efficient than CRIS in identifying potentially duplicative research projects within and between ARS and NIFA. As a result, USDA has not assessed its information systems in a way that would allow it to (1) maximize the usefulness of information and minimize the burden on the public and (2) avoid unnecessarily duplicating existing information technology capabilities, as described in the 2000 Office of Management and Budget policy. Unless USDA officials assess VIVO’s effectiveness and efficiency, and unless USDA officials establish written procedures for staff to use the best method when searching for duplicative research projects, ARS and NIFA scientists and managers could remain dependent on a less robust safeguard.

High-Level Collaborative Planning Has Increased, but Agency-Level Collaborative Planning between ARS and NIFA Has Not Been Systematic

Over the past few years, USDA’s Chief Scientist has facilitated increased collaborative planning within the REE mission area, particularly between ARS and NIFA, but agency-level collaborative planning between these two agencies has not been systematic. Specifically, the Chief Scientist and her staff have led several REE-level planning efforts that, over the past several years, have brought together staff from the two agencies and generated key products. Nevertheless, six USDA officials with planning responsibilities said that collaborative planning between ARS and NIFA national program leaders has not been systematic for topic areas the agencies have in common, and 20 officials and stakeholders said that ARS and NIFA should systematically hold joint planning and stakeholder input meetings to make the best use of limited agricultural research resources.

High-Level Collaborative Planning Has Generated Key Products

Over the past few years, the Chief Scientist, her senior advisors, and other staff have led efforts that have facilitated REE-level collaborative planning among ARS, NIFA, and the other REE mission area agencies, and these efforts have resulted in several key products. First, the Chief Scientist led a collaborative planning effort that involved the four REE agencies to develop the REE Action Plan. The Chief Scientist and her staff established seven teams, focused on the action plan’s seven goals, which held planning meetings from November 2010 to February 2012 and involved staff from multiple agencies in developing the subgoals, strategies, planned actions, and agency responsibilities corresponding to the action plan’s goals. For each goal and subgoal, the teams also identified specific challenges facing agriculture, natural resources, and conservation to be addressed by the planned actions. Several USDA officials told us that because the meetings led to more collaboration across REE agencies, these planning meetings were a major benefit of REE Action Plan development. For example, according to one USDA official, discussions among USDA officials during meetings about a subgoal related to energy and biofuels research highlighted the fact that research areas overlapped between USDA and the Department of Energy, creating the potential for duplicative work. Such discussions suggested a solution—a decision to focus USDA research efforts on developing biofuel feedstocks (i.e., materials such as perennial grasses from which biofuels are generated) and to move away from technology for converting materials into biofuels, a research area for which the Department of Energy has predominant responsibility. Further, according to this official, the discussions between ARS and NIFA officials during the development of the REE Action Plan clarified each REE agency’s role in each of the goals, strategies, and planned actions. Overall, this planning effort made use of practices consistent with two we identified in our October 2005 report as helpful to enhance and sustain agency collaboration, in particular, establishing mutually reinforcing or joint strategies and agreeing on roles and responsibilities.

Second, the Chief Scientist’s senior advisors led a collaborative effort from early 2011 to mid-2012 that resulted in six specific “white papers,” issued by the Office of the Chief Scientist, on the topics of global food security, bioenergy, nutrition and childhood obesity, climate change, food safety, and sustainable agricultural systems. These white papers were developed through a series of meetings and discussions with multiagency teams from ARS, NIFA, and other USDA agencies, and each white paper was independently peer reviewed. The senior advisors worked with these teams to describe in more detail challenges and strategies identified in the REE Action Plan, including (1) roles to be played by the agencies in addressing each challenge, (2) key strategies to focus on in fulfilling those roles, (3) USDA’s current actions, (4) USDA’s future plans, and (5) the research outcomes that USDA anticipates. Like the collaborative planning that led to the REE Action Plan, the white paper planning effort also made use of collaboration practices consistent with several we previously identified, particularly, defining and articulating a common outcome, establishing mutually reinforcing or joint strategies, and agreeing on roles and responsibilities.

Third, the Chief Scientist’s senior advisors stated that in 2010 and 2011 they coordinated input from ARS, NIFA, and other agencies to help develop a number of products, including Feed the Future: Global Food Security Research Strategy. Issued in May 2011, this global research portfolio serves as an integral part of the President’s broader Feed the Future initiative. Specifically, the senior advisors said that they established multiagency working groups of USDA officials, who worked with their counterparts in the Agency for International Development to identify research priorities—aimed at sustainably advancing productivity, transforming production systems, and enhancing nutrition and food safety around the world—to help implement this strategy.

The Chief Scientist’s senior advisors also facilitated collaborative planning efforts not resulting in products. For example, in September 2012, one senior advisor began an effort to better coordinate ARS’s intramural and NIFA’s extramural research on nutrition and obesity. In this instance, the Chief Scientist provided guidance for the initial joint meeting, and the senior advisor scheduled follow-up meetings to address unresolved issues.

Several USDA Officials Said That Collaborative Planning between ARS and NIFA Has Not Been Systematic

Program leaders in ARS and NIFA meet throughout the year for various purposes, but, according to six USDA officials who have planning responsibilities, the program leaders have not systematically held joint planning meetings to help determine research priorities for topic areas the agencies have in common or their roles and responsibilities for addressing these priorities. According to our review of meeting agendas and other agency documents, ARS and NIFA program leaders meet with stakeholders—such as agricultural producers and industry representatives—at scientific society, commodity group, stakeholder input, and other meetings throughout the year. The six USDA officials acknowledged, however, that the two agencies do not systematically plan or hold joint stakeholder input meetings, nor do they hold joint planning meetings. Rather, each agency generally holds separate stakeholder input meetings, even when agency research topics and stakeholders are the same or similar. For example, according to our review of meeting documentation, ARS held most of its recent stakeholder input meetings for its national programs independently. Even though ARS and NIFA generally hold such meetings independently, ARS’s and NIFA’s websites both identify collaborative ARS-NIFA stakeholder input meetings for the animal sciences programs. According to these websites and agency documents, animal sciences is the one broad topic area in which the agencies have for several years systematically held joint ARS-NIFA national stakeholder input meetings. The animal sciences team also holds joint ARS-NIFA planning meetings afterward to address stakeholder input and help each agency determine its research priorities as well as roles and responsibilities. In contrast, according to the six officials, national program leaders have not systematically held joint planning meetings to (1) help the agencies determine their research priorities for topic areas they have in common or (2) help each agency determine its roles and responsibilities for addressing these priorities.

According to USDA officials and stakeholders we interviewed, collaborative agency planning is important because the collective knowledge of both ARS and NIFA is needed to ensure that the agencies’ research areas and projects address top priorities and that research is complementary and not duplicative. In addition, these officials and stakeholders said that collaborative planning helps identify the appropriate entity—whether ARS itself, universities, or other organizations—to address a research problem. Each entity, they said, may have unique expertise, facilities, and equipment, and no one agency or official has all the relevant knowledge.

Over the last decade, USDA and others have emphasized the need for enhanced collaborative planning and joint stakeholder input, but USDA has not implemented key recommendations. USDA’s 2010 Roadmap states that solutions to many of the nation’s most serious problems demand change that USDA will bring about, in part, by better coordinating its science planning among and between REE science agencies and with other federal science agencies and by listening to the needs of stakeholders. Also, a 2010 annual report in the plant sciences area cited the lack of strategic planning for research across federal agencies and it stated that regular team meetings held quarterly would help improve collaboration within NIFA. In addition, reports by USDA and others made specific recommendations to enhance collaborative planning and joint stakeholder input. USDA, however, has not implemented these recommendations. For example, USDA has not implemented the recommendation from an ARS-NIFA-sponsored study that the agencies develop a unified approach to the stakeholder community to address and solve critical problems. Neither has USDA implemented the recommendation in a 2003 National Research Council report that REE hold a national summit every 2 to 3 years to engage the agencies and a broad spectrum of national, regional, and local stakeholders.

The Chief Scientist told us that she envisions greater collaborative planning between the agencies in the future through, in particular, more joint meetings for obtaining stakeholder input and discussing agency-level research plans to support the REE Action Plan. Several USDA officials and stakeholders told us that joint ARS-NIFA stakeholder input should be sought at least every 5 years, to coincide with ARS’s 5-year planning cycle, and suggested that the two agencies hold joint planning meetings annually. In addition, two industry stakeholders said that joint meetings would benefit them by increasing their contact with NIFA national program leaders, whom they may not know as well as they know ARS national program leaders.

Promising examples of systematic collaborative stakeholder input and planning, initiated independently by officials and individual teams within ARS and NIFA, already exist in one broad topic area—animal sciences— and in more limited areas that may fall within a national program or broad topic area. Specifically, for more than a decade, an animal sciences team of ARS and NIFA national program leaders has jointly sponsored large stakeholder meetings led by both agencies every 5 years, with smaller, annual stakeholder meetings on more specific topics (e.g., swine). The team started holding these meetings after a stakeholder requested such collaboration to save time spent attending two sets of similar meetings. The national program leaders meet after the stakeholder meetings to discuss ongoing research projects and associated resources, gaps in addressing stakeholders’ needs and priorities, and their respective agency’s potential roles and responsibilities; the program leaders also meet monthly by telephone or in person. Collectively, these meetings facilitate development of specific ARS and NIFA plans or requests for application. Several USDA officials and stakeholders we interviewed praised the animal sciences team’s collaborative effort as a best practice, and the team was nominated for a USDA-wide award for sustained collaboration that “has facilitated strong engagement with stakeholders and leveraged intramural and extramural programs to benefit U.S. animal agriculture.” Two industry stakeholders we spoke with who participate in these joint meetings said that the meetings assure them that the two agencies are coordinating their research plans.

According to USDA officials, many benefits have come from the animal science team’s collaboration. For example, according to USDA officials, this team’s collaborative planning led to the development of a network of researchers working to enable development of veterinary vaccines for cattle, swine, poultry, and fish. This network now includes scientists from ARS, several universities, and industry and coordinates with similar endeavors internationally. According to a USDA official and the network’s website, products developed from these collaborative efforts benefit a large group of researchers, including veterinary clinicians, immunologists, pathologists, and microbiologists. Additionally, joint stakeholder and planning meetings hosted every 5 years by the aquaculture subgroup of the animal sciences team have led to NIFA funding for a 2008 Regional Collaborative Research and Extension Catfish Forum, linking ARS, NIFA multistate committees, and land-grant college and university programs. According to NIFA and ARS aquaculture program leaders, the collaboration helped bring about the development of an innovative catfish production system that is now used commercially.

USDA officials also provided examples of other ARS-NIFA collaborative stakeholder input meetings and planning efforts that are more limited in scope or fall within a national program or topic area. For example, the National Plant Germplasm Coordinating Committee was established in 2006. Largely funded by ARS, with support from NIFA and universities, the National Plant Germplasm System protects plant hereditary material, or germplasm, including seeds, to preserve the genetic diversity of our nation’s crops. The coordinating committee, comprising ARS and NIFA officials and academic scientists, meets annually to review and set national scientific priorities for the system. The committee also initiates a strategic planning effort to better define and communicate a vision, mission, and short- and long-term goals for the system. In addition, in the area of food safety, ARS and NIFA national program leaders hold a joint annual meeting with a group of key stakeholders—three federal agencies involved in regulating food safety and preventing foodborne illness—to identify the research needs of these stakeholders. ARS and NIFA then use this information to help coordinate each agency’s research plans.

Even though the benefits of collaboration are widely recognized, part of the reason that ARS-NIFA collaborative planning may not be systematic, agency officials told us, is that REE does not require its agencies to systematically hold joint planning meetings or jointly meet with stakeholders. Moreover, some USDA officials we interviewed stated that it is a challenge for national program leaders—at both ARS and NIFA, where many national program leaders have heavier workloads because of recent retirements—to make systematic collaboration a high priority. One NIFA official stated that without a requirement for systematic collaborative planning, such planning is unlikely to increase. By enhancing collaborative planning for topic areas that ARS and NIFA have in common, as the animal sciences area has done, the agencies can take full advantage of their collective knowledge and expertise in setting their research priorities and determining their roles and responsibilities for carrying out these priorities.

Conclusions

Agricultural science, like science in general, advances through research that may at first seem duplicative but may (1) actually replicate, and thereby validate or disprove existing research, or (2) complement and extend existing research. Recognizing the importance of both replication and complementarity in science—and in light of some duplication at USDA’s largest research agencies in the past and the potential for some duplication in the future—ARS and NIFA have each developed important agency safeguards for reviewing projects, which work in tandem with the scientific community’s professional norms, to prevent duplicative research projects within and between the agencies. We commend the agencies for developing multitiered safeguards. Modifications to these safeguards, however, could better position both agencies to further reduce the risk of duplicative research projects in the future. For example, information on ARS projects is not current in CRIS because ARS has no written guidance for how often to send its project information to NIFA for uploading into the system; as a result, ARS has not always sent its project information in a timely manner. Thus, ARS project information that is typically at least 6 months out of date limits CRIS’s utility as a safeguard against duplicative research. In addition, since NIFA does not direct that all competitive grant applications recommended for funding be checked against CRIS for potential duplication, about one-third of the funds NIFA awards in competitive grants go toward projects that are, in effect, not subject to a key agency safeguard to prevent potential duplication and are thus at increased risk of inadvertent duplication. NIFA recently convened a task force to study, among other issues, whether this directive should be extended to other competitive grants. At the time of our report, the task force was still studying expanding the CRIS duplication search to all competitive grants and had not made a decision. Moreover, without an assessment of whether the search functionality of the forthcoming VIVO platform is more effective or efficient than CRIS at finding duplicative research projects, and until assessment results are used to drive procedures for identifying potentially duplicative projects, scientists and national program leaders cannot be assured that they are using the best method or system for detecting duplication. Finally, by successfully engaging ARS, NIFA, and other REE agencies, USDA’s Chief Scientist has enhanced high-level collaborative planning. In addition, some collaborative planning initiated jointly by ARS and NIFA has occurred across national program areas, such as in the animal sciences area. By enhancing collaborative planning—including planning that involves key stakeholders—for topic areas the agencies have in common, ARS and NIFA national program leaders can consistently take full advantage of their collective knowledge and expertise to help set their research priorities and help the agencies determine their roles and responsibilities for carrying out these priorities.

Recommendations for Executive Action

To help further reduce the risk of duplicative research within and between ARS and NIFA and to improve the agencies’ collaborative planning, we recommend that the Secretary of Agriculture direct the Under Secretary of Research, Education, and Economics to take the following four actions:

Require that ARS issue written guidance specifying that it should update its research project data in CRIS at least quarterly and ensure that this requirement is also applied to any new systems that receive these project data.

Require that NIFA issue written guidance that its national program leaders ensure that staff check all of NIFA’s competitive grant awards against CRIS for potential duplication.

Require, as USDA identifies promising systems for searching and disseminating research project information, the appropriate entity to (1) determine whether VIVO or other systems are more effective or efficient than CRIS in identifying potentially duplicative research and (2) revise guidance to reflect the best system and methods for identifying potentially duplicative research projects.

Enhance ARS’s and NIFA’s collaborative approach across all topic areas they have in common—including holding jointly sponsored meetings with stakeholders—to help set their research priorities and help them determine their roles and responsibilities for carrying out these priorities.

Agency Comments and Our Evaluation

We provided a copy of the draft report to the Department of Agriculture for its review and comment. In written comments from the department, which are reproduced in appendix II, the Chief Scientist stated that the department generally agreed with our findings. Nevertheless, USDA responded to the statement in our report that “collaborative planning between ARS and NIFA has not been systematic.” In our report, we define the term collaborative planning as the act of bringing research agencies and stakeholders together to discuss priorities, roles, and responsibilities, and we use the term systematic to mean marked by thoroughness and regularity. In its comments, USDA outlined six steps within REE that together it states constitute a system for collaborative planning, noting that the report accurately describes these steps. As we describe in the report, some of these steps have contributed to collaborative planning conducted jointly by ARS and NIFA across national program areas, such as in the animal sciences area. We concluded, however, that by enhancing collaborative planning—including planning that involves key stakeholders—for all topic areas the agencies have in common, ARS and NIFA national program leaders can consistently take full advantage of their collective knowledge and expertise to help set their research priorities and help the agencies determine their roles and responsibilities for carrying out these priorities. Such an approach would be more systematic in carrying out collaborative planning.

In addition, while neither explicitly agreeing nor disagreeing with our recommendations, the Chief Scientist stated that the department will give strong consideration to our recommendations as it seeks to improve how it identifies fruitful approaches to address the highest priority food and agricultural research needs. In these written comments, the Chief Scientist did not comment specifically on one of our recommendations: that USDA require that NIFA issue written guidance that its national program leaders ensure that staff check all of NIFA’s competitive grant awards against CRIS for potential duplication. Nonetheless, the written comments cite the benefits of implementing the three other recommendations. The following list summarizes the department’s comments on our recommendations:

For our recommendation that USDA require that ARS issue written guidance specifying that it should update its research project data in CRIS at least quarterly and ensure that this requirement is also applied to any new systems that receive these project data: In the written comments, the Chief Scientist stated that such written guidance to direct more frequent updates of ARS data “to common data systems could benefit the use of these systems as decision- support tools.” The Chief Scientist also stated that such a “benefit extends to several efforts under way in REE to develop and improve common data systems,” including VIVO and others mentioned in the report.

For our recommendation that USDA require, as it identifies promising systems for searching and disseminating research project information, the appropriate entity to (1) determine whether VIVO or other systems are more effective or efficient than CRIS in identifying potentially duplicative research and (2) revise guidance to reflect the best system and methods for identifying potentially duplicative research projects: In the written comments, the Chief Scientist stated that, as the initiatives “mature and provide validated improvements over existing systems, guidance for use of the best available tools for safeguarding against duplication” could “enhance this benefit.”

For our recommendation that USDA enhance ARS’s and NIFA’s collaborative approach across all topic areas they have in common— including holding jointly sponsored meetings with stakeholders—to help set their research priorities and help them determine their roles and responsibilities for carrying out these priorities: While USDA did not expressly agree or disagree with this recommendation, it stated in its comments that it expects ARS and NIFA to “continue to increase the number of jointly sponsored planning activities” and that ARS- NIFA joint stakeholder meetings in Plant Sciences and Aquaculture are already planned for 2013.

We continue to believe that all four recommendations would enhance safeguards against project duplication and strengthen collaborative planning.

USDA’s Chief Scientist also provided technical comments, which we incorporated into the report as appropriate.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of Agriculture, the appropriate congressional committees, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or garciadiazd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to examine (1) the topics the Agricultural Research Agency (ARS) and the National Institute of Food and Agriculture (NIFA) focus on and the safeguards they use to prevent duplication of research projects, along with any shortcomings in those safeguards, and (2) collaborative planning ARS and NIFA engaged in and how, if at all, such planning could be enhanced.

For both objectives, we collected and reviewed relevant law and agency guidance, as well as reports from the National Academies; the Inspector General of the U.S. Department of Agriculture (USDA); the National Agricultural Research, Extension, Education, and Economics Advisory Board (REE advisory board); and others. We also interviewed and gathered relevant documents from headquarters and field officials of USDA, including the Office of the Under Secretary for Research, Education, and Economics (REE); Office of the Chief Scientist; ARS; NIFA; and the Economic Research Service. We interviewed officials on the telephone and during our two site visits—one to ARS and NIFA headquarters offices in the Washington, D.C., area and the other to the University of California, Davis, and ARS’s field office in Albany, California. These officials included program leaders and other managers and scientists. In addition, we interviewed 16 stakeholders from the REE advisory board; industry organizations; and academic organizations, including land-grant colleges and universities. Stakeholders representing industry organizations came from the Florida Citrus Processors Association, American Dairy Science Association, National Pork Board, Iowa Soybean Association, and the National Grape and Wine Initiative. Stakeholders representing academic organizations were current or former officials of the Association of Public and Land-grant Universities; the University of Arkansas; the University of California, Davis; the University of Georgia Agricultural Experiment Stations; and Ohio State University. We selected industry and academic stakeholders on the basis of input from officials with USDA and the National Academy of Sciences’ Board on Agriculture and Natural Resources.

To examine the topics ARS and NIFA focus on, we collected and analyzed agency funding, spending, and planning data related to the topics on which ARS and NIFA focus their work. We collected and analyzed the President’s budgets to identify total ARS and NIFA funding for fiscal years 2002 through 2011 and NIFA budget data on the breakdown of its funding by mechanism (formula funding, competitive grants, and noncompetitive grants) for fiscal years 2001 through 2011. We assessed the reliability of these data and found them to be sufficiently reliable for our purposes. We also collected and analyzed spending by ARS and NIFA on research from fiscal year 2002 through fiscal year 2011. To assess the reliability of spending data provided from NIFA’s Current Research Information System (CRIS)—USDA’s primary database of completed and ongoing projects, which includes both financial and project information—we interviewed knowledgeable officials about the data, possible limitations of the data, and internal controls for the system containing the data. We found these data to be sufficiently reliable for our purposes. Nevertheless, our ability to quantify and compare the agencies’ spending on research topics was limited because, unlike ARS, NIFA could not isolate its research spending. Consequently, NIFA could only provide research spending data that also included an unknown amount of spending on education and extension activities. Moreover, we could compare ARS and NIFA research topics only at a high level and could not comparatively assess in detail the agencies’ spending on subtopic areas, such as which agency spent more money or a greater share of its money on childhood obesity or colony collapse disorder in honeybees. We also collected and analyzed estimates provided by NIFA on its research spending for fiscal years 2002 through 2011. These estimates are the same as those provided annually to the National Science Foundation. We assessed the reliability of these estimates by interviewing knowledgeable officials and comparing these estimates against data on spending and funding, and we determined them to be sufficiently reliable for our purposes.

To examine the safeguards used by the agencies to identify and prevent duplication within and between ARS and NIFA, we interviewed current and former scientists and agency officials, reviewed relevant scientific literature discussing professional norms and practices, and analyzed documentation of safeguards used at each agency to identify and prevent duplicative research. To identify shortcomings in those safeguards, we interviewed agency officials and analyzed relevant guidance. Also, we used agency guidance on searching CRIS to review for possible duplication a total of 10 ARS and 10 NIFA projects started in 2011, randomly sampled from the animal sciences and human nutrition areas. This review did not assess the extent of duplication across all NIFA and ARS projects. To ensure the reliability of the review, we also worked with knowledgeable USDA officials to develop the formulas used to query CRIS and obtain a master list of ARS and NIFA projects started in 2011 from which to derive a sample. As part of this review, we followed ARS and NIFA guidance to search CRIS by keywords and scientists’ names for potentially duplicative research projects within and between the two agencies. One analyst reviewed all 20 selected projects to determine whether they were potentially duplicative or, rather, complemented or replicated existing work. A second analyst then reviewed the first analyst’s initial determinations. We resolved any disagreement between the analysts through discussion and afterward obtained additional input from relevant ARS and NIFA scientists and officials for any project that either analyst determined to be potentially duplicative. Our review of selected projects flagged one ARS project and one NIFA project as potentially duplicative; after discussion with agency officials, however, we concluded that neither project was in fact duplicative. In ARS’s case, the same project was recorded twice because of two external funding sources, and in NIFA’s case, an existing formula-funded project supported the collection of preliminary data for a related competitive- grant-funded project.

To examine collaborative planning engaged in by ARS and NIFA and how, if at all, such planning could be enhanced, we analyzed the Chief Scientist’s actions aimed at increasing collaboration and the products resulting from these actions. We also interviewed USDA officials and stakeholders regarding the effects of these actions. To examine the collaborative planning ARS and NIFA engaged in, we reviewed agency plans and agendas and other documents related to stakeholders’ input into research priorities. In addition, we examined our own and USDA’s documents, issued from 1996 to 2010, discussing the need for improved collaboration and planning. For example, we reviewed two of our reports on barriers agencies face when attempting to collaborate on selected key practices that can help enhance and sustain collaboration among federal agencies. To identify practices that can enhance and sustain agency collaboration, we also reviewed a 2005 USDA-sponsored study on improving ARS-NIFA collaboration, two reports we issued on federal agency collaboration, other reports of ours on agricultural research planning, and National Research Council studies. Finally, we analyzed our interviews with USDA officials and stakeholders to report their views on whether collaborative planning—including stakeholder input meetings and priority-setting meetings—was sufficient and whether they had any recommendations that could enhance such collaborative planning. We also examined whether the agencies’ collaborative efforts reflected key practices we identified in October 2005 that can help enhance and sustain collaboration among federal agencies.

We conducted this performance audit from January 2012 through April 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the U.S. Department of Agriculture

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Daniel Garcia-Diaz, (202) 512-3841 or garciadiazd@gao.gov.

Staff Acknowledgments

In addition to the individual named above, Karen A. Jones (Assistant Director), Kevin Bray, Tony Calero, Ellen W. Chu, Paula Moore, Beverly Peterson, Dan Royer, Michael Silver, and Tama Weinberg made significant contributions to this report.