Agricultural Research: Two USDA Agencies Can Enhance Safeguards against Project Duplication and Strengthen Collaborative Planning
Published: 2013-04-12T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The USDA's principal research agencies, ARS and NIFA, play a key role in supporting agricultural science, and questions have been raised about the extent to which the two agencies may be performing duplicative research and whether the agencies collaborate in planning research. Research duplication is the inadvertent repetition of research that does not confirm or verify conclusions from prior studies. Collaborative planning involves bringing together research agencies and stakeholders to discuss priorities and roles and responsibilities. In this context, GAO was asked to assess how these agencies ensure the efficient use of their resources for research. This report examines (1) the topics ARS and NIFA focus on and the safeguards the agencies use to prevent duplication of research projects, along with any shortcomings in those safeguards, and (2) collaborative planning ARS and NIFA engaged in and how, if at all, such planning could be enhanced. GAO reviewed USDA safeguards against duplication within and between ARS and NIFA; reviewed 20 randomly selected projects; analyzed information on collaborative planning; and interviewed officials from USDA, universities, and industry.
section_title: What GAO Found
paragraphs: The Department of Agriculture's (USDA) Agricultural Research Service (ARS) and the National Institute of Food and Agriculture (NIFA) generally focus on many of the same broad topics and rely on agency safeguards, as well as on the scientific community's professional norms, to prevent inadvertent duplication of research projects within and between the agencies. Shortcomings with certain agency safeguards, however, may increase the potential risk of project duplication within or between the two agencies. ARS and NIFA built in their own safeguards to help prevent project duplication, such as (1) panels of independent external scientists who review proposed projects and (2) agency requirements for staff to ensure that proposed work is relevant, including checking the Current Research Information System (CRIS)--USDA's primary system containing project-level information on its ongoing and completed research projects--for potentially duplicative research projects in both agencies. The agencies also rely on professional norms to safeguard against duplication, such as the peer review process used by scientific journals to limit the publication of unnecessarily duplicative research. Indeed, agency officials and stakeholders could not provide recent examples of duplication within or between the two agencies, and GAO's review of 20 randomly selected research projects did not identify duplicative projects. Nevertheless, GAO identified a few shortcomings that somewhat limit the utility of certain agency safeguards. First, information in CRIS about ARS projects was typically at least 6 months out-of-date when uploaded, which undermines CRIS's utility as a safeguard. ARS officials said that the agency now expects staff to provide ARS project information on a quarterly basis, but ARS has not issued guidance about this expectation. Second, NIFA directs staff to conduct a CRIS duplication check for projects that accounted for about two-thirds of the funding it awarded for competitive grants; as a result, about one-third of its competitive grants are not subject to this safeguard against duplication. NIFA recently convened a task force to study, among other issues, whether the directive to check CRIS should be extended to all competitive grants.
USDA's Chief Scientist facilitated high-level collaborative planning, particularly between ARS and NIFA, in recent years, but 20 USDA officials and stakeholders said that agency-level collaborative planning between ARS's and NIFA's national program leaders working in common topic areas could be more systematic to make the best use of limited agricultural research resources. Specifically, the Chief Scientist and her staff led several high-level planning efforts that brought together staff from the two agencies and generated key products, such as a plan that identified USDA's seven goals for implementing its science priorities and the agencies responsible for implementing these goals. Nevertheless, national program leaders at the two agencies generally do not, and are not required to, systematically hold joint meetings for seeking stakeholder input and for setting research priorities. Some systematic collaborative planning, however, has been jointly initiated by ARS and NIFA national program leaders, such as in the animal sciences area. By enhancing collaborative planning across national program areas, as the animal sciences area has, ARS and NIFA can take fuller advantage of their collective knowledge and expertise to help set their research priorities.
section_title: What GAO Recommends
paragraphs: GAO recommends, among other things, that ARS issue guidance that project information be provided to CRIS on a quarterly basis and that ARS and NIFA enhance collaborative planning. USDA generally agreed with GAO's findings, and cited benefits for three of the four recommendations.

section_title: Letter
paragraphs: For over a century, U.S. agricultural research has been a major catalyst in  creating a vigorous agricultural economy and a plentiful, low-cost supply  of food and fiber. As the 21st century unfolds, agriculture and natural  resources stand at the crossroads of some of the world’s most critical  problems, such as establishing sustainable food production, providing  clean and abundant water, and ensuring food safety. According to a U.S.  Department of Agriculture (USDA) report, by 2050, global agricultural  demand is projected to increase by 70 to 100 percent because of  population growth, energy demands, and higher incomes in developing  countries. Further, meeting this demand from existing agricultural  resources will require substantial increases in global and U.S. agricultural  productivity—productivity based on long-term investments in public  agricultural research. Advances resulting from basic and applied  agricultural sciences, such as disease-resistant crop varieties and  efficient irrigation practices, are fundamental to boosting agricultural  yields, increasing farm-sector profitability, and improving nutrition and  human health. In addition, USDA economists estimated in 2007 that each  dollar spent on public agricultural research returned an estimated $10 in  benefits to the economy, and university researchers have estimated even  greater returns.
USDA’s principal internal (intramural) and external (extramural) research  agencies—the Agricultural Research Service (ARS) and the National  Institute of Food and Agriculture (NIFA), respectively—have long played a  critical role in supporting agricultural science, including in areas such as  plant and animal sciences and human nutrition. These two agencies are  housed within USDA’s Research, Education, and Economics (REE)  mission area, and the Under Secretary for REE serves as USDA’s Chief  Scientist. ARS is USDA’s primary intramural research agency, typically  working on more than 800 research projects within 18 national program  areas in more than 90 ARS locations. Unlike ARS, NIFA does not do its  own research but supports research through grants (to individuals,  institutions, and organizations), education to improve scientific and  agricultural literacy, and its extension network to solve public agricultural  problems. By itself or with other partner organizations, NIFA offers  competitive and noncompetitive grants to land-grant colleges and  universities in all 50 states and to other universities as well.
The nation’s increasingly tight budget environment underscores the need  for federal research agencies to set priorities carefully and make effective  use of limited research funding. At both ARS and NIFA, national program  leaders are responsible for setting the agencies’ research priorities;  obtaining input from stakeholders in industry, academia, and elsewhere;  and identifying gaps in agricultural research. As agency budgets continue  to tighten, pressure increases on program leaders to ensure that dollars  go to the highest-priority activities and that research projects are  complementary, rather than unnecessarily duplicative.
Previously, we have reported that duplication occurs when two or more  agencies or programs are engaged in the same activities or provide the  same services to the same beneficiaries; we recognized, however, that  duplication may be appropriate in some instances, depending on the  nature or magnitude of the federal effort. With respect to scientific  research, the National Academies distinguish among duplication,  replication, and complementarity. Specifically, according to the  academies, duplication represents the inadvertent repetition of research  that does not serve to confirm or otherwise verify conclusions from  previous studies. In contrast, replication represents the deliberate  repetition of research efforts, intended to confirm or extend previously or  simultaneously obtained, but still uncertain, findings. Complementarity  represents overall strategies to confirm, overturn, or extend particular  research findings. According to the academies, even though some  duplicative research may be acceptable—for example, during periods of  great crisis, such as the federal response to AIDS (acquired  immunodeficiency syndrome), in which usually stringent research  practices were eased somewhat in the pursuit of as many promising  leads as possible—eliminating unnecessarily duplicative research is a  means of saving public dollars and thus minimizing waste. In this report,  we use the term duplication to mean research that is scientifically  unnecessary to replicate or complement prior research results, or  research inadvertently conducted or funded by ARS or NIFA that is very  similar to other research.
In this context, you raised questions about duplication in agricultural  research and asked how ARS and NIFA ensure the efficient use of their  resources for research. This report examines (1) the topics ARS and  NIFA focus on and the safeguards they use to prevent duplication of  research projects, along with any shortcomings in those safeguards, and  (2) collaborative planning ARS and NIFA engaged in and how, if at all,  such planning could be enhanced.
For both objectives, we collected and reviewed relevant law and agency  guidance, as well as reports from USDA’s Inspector General; the National  Agricultural Research, Extension, Education, and Economics Advisory  Board (REE advisory board); and others. We also interviewed and  gathered relevant documents from headquarters and field officials in  USDA, including the Office of the Under Secretary for REE, the Office of  the Chief Scientist, ARS, NIFA, and the Economic Research Service. In  addition, we interviewed 16 stakeholders from the REE advisory board;  industry organizations; and academic organizations, including land-grant  colleges and universities. We selected industry and academic  stakeholders primarily on the basis of input from officials with USDA and  the National Academy of Sciences’ Board on Agriculture and Natural  Resources. In addition, to examine the topics ARS and NIFA focus on  and the safeguards used by the agencies to prevent duplication within  and between ARS and NIFA, we collected and analyzed agency data on  funding, spending, and planning for fiscal years 2002 through 2011  related to the topics on which ARS and NIFA focus their work, and we  analyzed the scientific literature discussing professional norms, as well as  documentation of safeguards used at each agency to identify and prevent  duplicative research. We assessed the reliability of spending, funding,  and planning data provided from NIFA’s Current Research Information  System (CRIS)—USDA’s primary database of completed and ongoing  projects, which includes both financial and project information—by  interviewing knowledgeable officials about the data, possible limitations of  the data, and internal controls for the system containing the data. We  found these data to be sufficiently reliable for our purposes, but our ability  to quantify and compare the agencies’ spending on specific topics was  limited because, unlike ARS, NIFA could not isolate its research spending  and thus could only provide data that also included some education and  some extension spending. In addition, to identify shortcomings, if any, in  the agencies’ safeguards against duplication of research projects within  and between ARS and NIFA, we interviewed agency officials and  reviewed relevant guidance. We also followed agency guidance and  reviewed for possible duplication 20 randomly selected ARS and NIFA  projects started in 2011 in the animal sciences and human nutrition areas.  To ensure the reliability of this review, we worked with knowledgeable  USDA officials to develop the formulas from which we derived the random  sample for review and followed ARS and NIFA guidance on searching  CRIS for potentially duplicative research projects. Two analysts reviewed  all selected projects for possible duplication and followed up with the  agencies on instances of apparent duplication. To examine the  collaborative planning ARS and NIFA engaged in, we analyzed agency  plans and agendas and other documents related to stakeholders’ input  into research priorities. We also examined whether the agencies’  collaborative efforts reflected key practices we identified in October 2005  that can help enhance and sustain collaboration among federal  agencies.
We conducted this performance audit from January 2012 through April  2013 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The United States’ publicly funded agricultural research system is a  critical component in finding answers to urgent global problems, including  establishing sustainable food production, providing clean and abundant  water, and strengthening food safety (see fig. 1). This public system is  based on a federal-state partnership in which USDA and the states,  through their land-grant colleges and universities, play multiple roles in  agricultural research, extension, and education. According to a 2011  USDA report, studies have shown that private research productivity  depends on government and university investments in science and  technology. The federal, state, and private sectors provided 27 percent,  12 percent, and 61 percent, respectively, of total U.S. agricultural  research funding in 2007, according to the most recent USDA estimate.
The federal portion of this public research system, with roots dating back  to the 19th century, is overseen by USDA’s REE mission area. The REE  mission area comprises the following four agencies:
ARS, USDA’s key intramural research agency;
NIFA, USDA’s key extramural research agency;  the Economic Research Service, USDA’s key intramural economics  and social science research agency; and  the National Agricultural Statistics Service, focused on agricultural  statistics.
The Food, Conservation, and Energy Act of 2008 (2008 Farm Bill)  reorganized this mission area to include an umbrella coordinating unit  within the Office of the Under Secretary for REE. The 2008 Farm Bill also  designated the Under Secretary for the REE mission area as USDA’s  Chief Scientist, and USDA also refers to this umbrella unit as the Office of  the Chief Scientist. Accordingly, the Chief Scientist is to ensure, among  other responsibilities, that USDA’s research, education, and extension  activities are coordinated and integrated across agencies and disciplines.  In addition, six division chiefs—commonly referred to by USDA as senior  advisors—are to assist the Chief Scientist in identifying and addressing  emerging needs and priorities in research, education, and extension.
Mandated by the 2008 Farm Bill, the Under Secretary of REE (i.e.,  USDA’s Chief Scientist) developed a Roadmap for USDA Science in  2010 to set USDA’s research, education, and extension agenda. This  document identified five USDA priorities: bioenergy, climate change, food  safety, international food security, and child nutrition. The Roadmap  stated that solutions to many of modern society’s most intractable  problems demand change that USDA would bring about by, for example,  (1) better coordinating its science planning among and between REE  science agencies and with other federal science agencies; (2) listening to  the needs of stakeholders; and (3) institutionalizing outcome-driven  scientific program planning and implementation. The 2008 Farm Bill  specified that the Roadmap was not to be subject to review by any federal  employee other than the Secretary of Agriculture or his designee.
In early 2012, the Chief Scientist followed up on issuance of the  Roadmap with a document titled Research, Education, and Economics  Action Plan (REE Action Plan). The REE Action Plan identifies seven  goals that support top priorities for USDA science: (1) local and global  food supply and security, (2) climate and energy needs, (3) sustainable  use of natural resources, (4) nutrition and childhood obesity, (5) food  safety, (6) education and science literacy, and (7) rural prosperity and  rural-urban interdependence. Each of these seven goals has strategies  and planned actions designating the specific USDA agencies that are to  be responsible for implementing the actions (see fig. 2 for an excerpt). In  addition, the first three goals are divided into multiple subgoals, each of  which also has planned actions, or actionable items, designating the  specific USDA agencies that are to be responsible.
Within REE, research conducted or funded by ARS and NIFA,  respectively, may be basic or applied. According to USDA’s website,  basic research discovers the underlying processes and systems that  make a plant, animal, ecosystem, food system, community, or  marketplace work. For example, basic research might seek to discover  the genetic map of a plant or animal or to show how economic and  human resources affect economic growth in rural areas. In contrast,  applied research expands on basic research findings to uncover practical  ways in which new knowledge can be advanced to benefit individuals and  society. For example, in an applied research project, researchers might  use a genetic map constructed through basic research to develop  methods to combat plant and animal diseases. NIFA-funded research  may also be integrated, in that it involves education and extension  activities, as well as research.
Both ARS and NIFA solicit stakeholder input. Agency stakeholders  consist of individuals and institutions that conduct or use agricultural  research, education, and extension services provided by or for the  agencies. These stakeholders include representatives of commodity  groups, industry, interagency federal working groups, scientific societies,  and university partners. Stakeholders also include members of the REE  advisory board, which comprises 25 members, each representing a  specific stakeholder category, such as a national nutritional science  society or national farm organizations. ARS itself is considered a NIFA  stakeholder because ARS scientists are eligible to compete for certain  NIFA grants. In turn, NIFA is a stakeholder of ARS.
USDA differs from other federal science agencies in that the majority of  its annual research appropriation supports research done in-house, or  intramurally; in contrast, most federal science agencies primarily fund  research done elsewhere, or extramurally. As USDA’s principal intramural  research agency, ARS had over 2,200 research scientists in fiscal year  2012, who carried out about 800 research projects at more than  90 facilities nationwide and in a few foreign countries. To manage its  18 national research programs, ARS has developed a 5-year research  cycle, consisting of four activities: stakeholder input, planning,  implementation, and assessment. About 25 ARS national program  leaders define and articulate the scope and objectives of each of these  national programs with input from their various stakeholders; ARS  scientists and managers; and others, including Congress. A key means  of acquiring stakeholder input is through a meeting with stakeholders  conducted at the start of each 5-year cycle. During this meeting, ARS  gathers stakeholder input and defines which research problems it will  address in the next 5 years to help meet high-priority needs within the  agricultural community. Each national program uses this stakeholder  input to help develop a research plan.
NIFA is USDA’s primary extramural research agency, and, unlike staff of  ARS, who conduct their own research, NIFA staff do not conduct  research. Rather, NIFA’s 350 staff members, including about 70 national  program leaders, help fund research at the state and local level, primarily  through grants, and provide program leadership to state and local officials  in research, education, and extension. Recipients of NIFA funds include  59 state agricultural experiment stations (i.e., the research arm of land- grant colleges and universities), 63 schools of forestry, 18 historically  black land-grant colleges, 27 colleges of veterinary medicine, and other  institutions and individual scientists. Figure 3 identifies the locations of  ARS facilities, along with the land-grant colleges and universities  established under the Morrill Act of 1862, those established under the  Second Morrill Act of 1890 (historically black land-grant colleges), and  those established under the Equity In Educational Land-Grant Status Act  of 1994 (tribal colleges).
Congressional legislation establishes the funding levels, eligibility  requirements, and scope for each of NIFA’s research, education, and  extension programs, and NIFA’s national program leaders manage and  oversee these programs. NIFA generally funds grants for 1 to 4 years. In  addition to direction from Congress, stakeholder input informs the  development of NIFA requests for applications to its competitive grants  programs. Each request generates from 2 to 600 applications, each of  which must undergo external peer review by a panel of 3 to 25 individuals  with expertise in the relevant field. The panels evaluate each application  for scientific merit, how well the application addresses the research  priorities, and the work’s potential to make a positive scientific impact.  The panels then rank the applications and recommend the top projects for  eventual funding. NIFA awards the following types of grants for  distributing its research funds:
Formula grants (commonly referred to as formula funds). Through  several program authorities, NIFA provides funds for research and  extension to land-grant colleges and universities, schools of forestry,  and schools of veterinary medicine. The amount of funding provided  to each institution is determined by formulas, often statutorily defined,  which may include variables such as rural population, farm population,  and poverty levels. According to NIFA’s website, local or regional  university leaders decide which specific projects are to be supported  by an institution’s formula grant allotment. As a result, according to a  2011 Congressional Research Service report, specific research  priorities for formula grants tend to be more geographically focused  than for NIFA’s other grants.
Competitive grants. NIFA awards nearly 40 types of competitive  grants for research, extension, and higher education, as well as for  projects that integrate all three activities. Competitive grants are  designed to enable USDA to (1) attract a wide pool of applicants to  work on agricultural issues of national interest and (2) select the best- quality applications submitted by highly qualified individuals,  institutions, or organizations. Eligibility, administrative rules, and  procedures vary for each specific competitive grant program  according to authorizing statutes, and NIFA awards funds following  external peer review of the applications. NIFA’s flagship competitive  grant program is the Agriculture and Food Research Initiative, which  funds research, education, and extension in the food and agricultural  sciences and, according to NIFA’s website, awards grants to address  agricultural problems of national, regional, or multistate importance.
Noncompetitive grants. Before fiscal year 2011, according to the 2011  Congressional Research Service report, appropriated NIFA funds  were directed by Congress to support a designated institution or set of  institutions for particular research, education, or extension topics of  local, regional, or national priority. These projects were supported  through “special research grants” or “direct federal administration  research or education grants.” Recent appropriations have not  included such spending items, allowing NIFA to award more of its  funds through competitive grants, according to USDA officials.
Even with growing demand for agricultural research over the past decade,  total appropriations for ARS and NIFA decreased when adjusted for  inflation. Specifically, from fiscal year 2002 through fiscal year 2011,  funding for ARS decreased by 23 percent, from $1.2 billion to  $934 million, and funding for NIFA increased by more than 28 percent,  from $1.1 billion to $1.3 billion. When adjusted for inflation, however, ARS  and NIFA funding decreased by nearly 42 percent and 3 percent,  respectively.
To document USDA’s publicly funded activities, including research  conducted or funded by the department, and to provide ready access to  information about its research projects and related work, USDA in 1966  developed CRIS. According to officials from NIFA, which is responsible  for managing this system, CRIS contains descriptive project-by-project  information for all completed and ongoing projects carried out or funded  by ARS and NIFA, respectively, and by some other USDA agencies and  state institutions. USDA scientists, stakeholders, and the public can  search CRIS for ARS and NIFA projects, as well as for some information  on related publications and researchers. USDA has expanded CRIS over  time to also cover a number of education, extension, and integrated  activities. According to NIFA’s website, a part of CRIS’s mission is to help  users do the following:  keep abreast of the latest developments in agricultural and forestry  research and education, plan future activities, avoid costly duplication of publicly funded work, and establish valuable contacts within the research and education  community.
Concerning collaboration among federal agencies, in March 2000, we  reported that agencies face a range of barriers when they attempt to  collaborate with other agencies. Faced with these barriers, federal  agencies carry out programs in a fragmented, uncoordinated way,  resulting in a patchwork of programs that can waste scarce funds,  confuse and frustrate program customers, and limit the overall  effectiveness of the federal effort. Subsequently, in an October 2005  report, we discussed key practices that can help enhance and sustain  collaboration among federal agencies. Among the practices we  identified were (1) defining and articulating a common outcome;  (2) defining and agreeing on roles and responsibilities; (3) establishing  mutually reinforcing or joint strategies; (4) identifying and addressing  needs by leveraging resources; and (5) establishing compatible policies,  procedures, and other means to operate across agency boundaries. In  our 2005 report, we stated that, while we generally believe that the  application of as many of these practices as possible increases the  likelihood of effective collaboration, we also recognize that agencies work  together in a wide range of situations and circumstances.
section_title: ARS and NIFA Use Agency and Scientific Safeguards to Prevent Duplication of Research Projects in Similar Topic Areas, but Some Shortcomings Exist
paragraphs: ARS and NIFA generally focus on many of the same broad topics, and  the agencies rely on agency-developed safeguards, as well as on the  scientific community’s professional norms, to prevent duplication of  research projects within and between the agencies. Indeed, agency  officials and stakeholders could provide no recent examples of duplication  within or between the two agencies, and our review of 20 randomly  selected agency research projects did not identify duplicative projects.  Nevertheless, our review identified some shortcomings in certain  safeguards the agencies rely on—shortcomings that somewhat limit the  utility of these safeguards in preventing duplicative research.
subsections: section_title: ARS and NIFA Generally Focus on the Same Broad Topic Areas
paragraphs: ARS and NIFA generally focus their work on the same broad topic areas.  Our analysis of each agency’s research spending for fiscal years 2002  through 2011 shows that during this period, ARS performed and NIFA  funded scientific activities in largely the same six broad topic areas:  (1) plant systems; (2) human nutrition; (3) animal systems; (4) natural  resources; (5) food and nonfood products; and (6) agricultural, natural  resource, and biological engineering. For example, in fiscal year 2011— the most recent year for which data from the agencies were available— both agencies performed or funded work on plant systems (ranging from  plant genetics to plant pests) more than on any other topic area,  representing about 41 percent of ARS’s total $1.1 billion research  spending and about 23 percent of NIFA’s total $1 billion fiscal year 2011  research spending (see table 1). ARS and NIFA both focused the next- largest shares of their spending in fiscal year 2011 on human nutrition,  animal systems, and natural resources. NIFA also funded activities in two  topic areas on which ARS did limited or no research from fiscal year 2002  through 2011: (1) economics, markets, and policy and (2) families, youth,  and communities. Similarly, within some topic areas, ARS conducted  research on certain issues that NIFA did not fund. For example, ARS  includes six human nutrition research centers that conduct studies on a  wide range of topics, such as the nutrient content of commonly eaten  foods and food choices in schools.
ARS and NIFA also share responsibility for many actions needed to  address REE goals as identified in its 2012 REE Action Plan. Our  analysis of the 2012 REE Action Plan shows that ARS and NIFA share  responsibility for two-thirds of the 139 planned actions delineated as  involving either agency or both. Specifically, the action plan identifies  ARS and NIFA as jointly responsible for 92 (66 percent) of the  139 planned actions. For the remaining 47 planned actions (34 percent),  ARS or NIFA is solely or partially responsible. These planned actions  align with strategies REE identified as needed to address its seven goals.  For example, to help address its nutrition and childhood obesity goal, the  action plan discussed a strategy to develop and extend approaches for  preventing obesity and related diseases, and it designated both ARS and  NIFA as responsible for determining the causes and consequences of  obesity and related conditions.
section_title: The Agencies Rely on Various Safeguards to Prevent Duplication of Research Projects
paragraphs: ARS and NIFA use various safeguards to prevent duplication of research  projects within and between the agencies. Specifically, each agency has  built its own safeguards against project duplication into processes for  developing and reviewing research proposals. The agencies also rely on  the scientific community’s professional norms, which discourage  duplicating the work of other scientists, even in similar topic areas.  Indeed, agency officials and stakeholders could provide no recent  examples of duplication within or between the two agencies, and our  review of 20 randomly selected agency research projects did not identify  duplicative projects.
ARS has developed several internal agency safeguards to help prevent  duplication, which operate at different stages in the agency’s 5-year  research cycle. The agency implemented this research cycle to help  ensure the relevance, quality, and impact of its research. As part of the  cycle, leaders from ARS’s 18 national programs assign research  objectives to scientists in the field, and to meet those objectives, these  scientists, in turn, work with national program and local area management  to develop project plans for the forthcoming 5-year period. Project plans  are reviewed by independent experts in the relevant field of science, ARS  managers and officials, including those in ARS’s Office of Science Quality  Review. The key safeguards ARS developed to help prevent duplicative  research projects include the following:
National program leaders set research project objectives. ARS’s  national program leaders set the objectives for all proposed research  under the agency’s 18 national programs and review each proposal to  ensure that research plans align with assigned objectives.  Specifically, national program leaders work with local managers to  translate the research priorities of each national program into  research objectives for the next 5 years. According to ARS officials,  this step serves to prevent research duplication because national  program leaders who write and assign the objectives can see if, for  two different research projects, they are writing objectives that are too  similar, and these national program leaders can rework objectives to  be distinct.
Scientists develop project plans using specific guidance. Teams of  ARS scientists then use the objectives approved by the national  program leaders to develop project plans—research proposals for  either a single assigned research objective or a group of assigned  objectives. These project plans are to discuss the nature of the  problem to be addressed and detail experimental approaches for  examining the problem. According to USDA officials, each national  program has multiple project plans, each addressing different  objectives within the national program. ARS guidance states that each  project plan must (1) discuss gaps in knowledge that proposed  research is intended to address and (2) describe how proposed  research relates to, and complements, other related research within  and outside of USDA. Together, these required discussions of gaps in  knowledge and relation to other research help the agency identify and  prevent approval of potentially duplicative research projects within and  outside of ARS.
National program leaders review project plans. Throughout  development of each project plan and after a plan is complete, ARS  national program leaders and local managers review the proposed  research to ensure that the final objectives are consistent with the  ones assigned. This process can identify when two research  proposals are too similar or when a proposal has the potential to  duplicate other proposed or existing work. For example, one scientist  told us that his national program leader noticed that his proposed  applied microbiology research was very similar to work already being  done at another ARS laboratory. As a result, this scientist ceded his  research to the other laboratory, thus averting duplication within ARS.
External peer review by independent experts. In response to the  Agricultural Research, Extension, and Education Reform Act of 1998,  which requires the establishment of procedures for external peer  review every 5 years of all research conducted by ARS, ARS  scientists now submit completed project plans for review by an  external panel of scientific experts before ARS approves funding for  any research. According to ARS officials, successful completion of  external peer review as a prerequisite for ARS’s funding of its own  intramural research was a significant change for the agency. ARS’s  Office of Science Quality Review coordinates this external peer  review, in which external peer review panels—comprising non-ARS  scientists selected for their expertise in a field pertinent to the  research proposals under review and coming from different places,  such as universities, industry, or other federal agencies including  NIFA—independently assess the quality of ARS’s proposed research,  including specific assessment of each proposal’s significance and  relevance. According to agency officials, these external peer review  panels each typically review from 4 to 12 ARS project plans and  evaluate individual project plans for adequacy of their experimental  approaches, probability of their success in achieving stated  objectives, and significance of the proposed research. On the basis of  recommendations from the peer review panels, officials from ARS’s  Office of Science Quality Review certify project plans that successfully  pass peer review or decline to certify plans that fail to adequately  address peer reviewers’ concerns. ARS’s best practices on peer  review state that the agency’s project plans must provide assurance  that proposed work addresses a unique gap in knowledge and is not  duplicative of ongoing or completed ARS work. Moreover, ARS  guidance states that ARS scientists are to demonstrate to peer  reviewers that they are aware of others performing similar research  and to show how ARS’s proposed research is unique. Successful  completion of ARS’s external peer review means that a proposed  project has scientific merit and addresses relevant gaps in  knowledge—including, in essence, that the proposed research is not  duplicative.
CRIS search for duplication. ARS guidance directs the agency’s  scientists to search CRIS for potentially duplicative projects when  preparing their project plans. As part of “due diligence,” ARS scientists  must show that research proposed in their project plans is relevant  and unique, and ARS guidance directs the scientists to survey  relevant literature, including searching CRIS. According to ARS  officials, the purpose of this CRIS search is to see if ARS or NIFA has  already funded the research the scientists are proposing. ARS  officials responsible for managing external peer review commented  that this CRIS search is typically the best method for identifying  potentially duplicative government research. In addition, the officials  told us that reviewers expect to see that such a CRIS search has  been done and will say if they feel that the search has not been done  adequately. ARS officials told us that this CRIS search generally  consists of using keyword lists associated with each project in CRIS  and names of scientists leading the work to identify projects that might  be too similar or have overlapping objectives. For example, an ARS  team proposing to work on avian influenza found through its CRIS  search that NIFA had funded a grant on various avian influenza  strains; after reading the objectives of that grant, however, the team  determined that, although similar, these objectives were not the same  and that the ARS team’s research would be substantively different  and not duplicative.
Like ARS, NIFA has also built several agency safeguards into its reviews  of proposed work, which help prevent the funding of potentially duplicative  research projects. These safeguards operate in NIFA’s competitive grants  programs and, to some extent, in the provision of formula funds to the  states. The key safeguards NIFA relies on to help prevent duplicative  projects include the following:
External peer review of competitive grant applications. NIFA uses  panels of external scientists—typically comprising 3 to 25 experts in  the field pertinent to the applications being reviewed—to assess the  strengths and weaknesses of all competitive grant applications and to  confirm that proposed work is original. These panels generally  review each application on the basis of scientific merit and how well  the application addresses, among other things, NIFA’s research  priorities. For example, criteria for the agency’s Agriculture and Food  Research Initiative include, among other factors, originality of  proposed research and relevance to NIFA priorities, including the  importance of the proposed topic to agriculture. As part of their  assessments, peer review panels check that the proposed work is not  already funded by other grant-making agencies. According to USDA  officials, panelists rely on their own familiarity with work in their fields  to determine whether a proposed project duplicates existing work in  those fields.
Application forms for competitive grants. According to agency officials,  they use the application process to help identify and discourage  potential duplication. NIFA requires competitive grant applicants to  submit two forms, each of which serves as a partial check against  potentially duplicative projects. The first form, called the Current and  Pending Support form, requires a project’s lead researcher to  document all sources of funding requested or received for his or her  active and pending projects. The second form requires applicants to  verify whether their application has also been submitted to other  agencies for funding and states that submitting the same application  in response to more than one NIFA request for application is not  permissible. According to NIFA officials, information in these forms  can alert the peer review panel or national program leader reviewing  applications to instances where the research proposed in an  application under review has already been funded, or is currently  being considered for funding, by NIFA as part of a separate  application and thus serves as a safeguard against duplication.
CRIS duplication check for competitive grants under the Agriculture  and Food Research Initiative. NIFA’s national program leaders direct  staff to conduct a CRIS duplication check, using relevant keywords  and the names of lead scientists, before awarding funds for any  application submitted to this key competitive grant program. NIFA  guidance on reviewing Agriculture and Food Research Initiative  applications states that national program leaders must check CRIS to  determine if the proposed work has already been funded by NIFA or  ARS and to ensure that it is not unnecessarily repeating work not yet  published. National program leaders must also check if the proposed  work is receiving funds from two different sources for essentially the  same work. NIFA guidance states that the Agriculture and Food  Research Initiative will not fund projects that overlap with work already  being funded by USDA or work that is unnecessarily duplicative.
National program leaders are to document this CRIS search and  attest to the proposed work’s originality and uniqueness.
Review of formula funds for innovation. Even though formula-funded  projects do not undergo external peer review, and NIFA’s national  program leaders are not required to check CRIS for duplication when  reviewing formula-funded projects, agency guidance emphasizes  that work proposed for formula funds be relevant and innovative. NIFA  guidance states that, as part of their general quality review, national  program leaders should consider whether the proposed research  addresses an important problem. In addition, NIFA guidance directs  national program leaders to assess whether the proposed work is  innovative, by asking whether it has the potential to add to the body of  scientific knowledge by, among other things, addressing new issues  or taking an innovative approach. This assessment in effect evaluates  whether the proposed project is potentially duplicative of other  research and thus serves to safeguard against duplication.
In addition to the agency safeguards they have developed, ARS and  NIFA rely on several of the scientific community’s professional norms to  help safeguard against duplicating research projects. According to  agency officials, scientists we spoke with, and the literature we examined  about science, the agencies rely on three such professional norms:
Scientists have a strong incentive to produce original research and to  make new discoveries. According to scientists and stakeholders,  scientists are rewarded for producing new or original research, not for  re-creating existing work; grants are not awarded for duplicative  research. Scientists we spoke with often told us that a scientist would  be penalized professionally for conducting research that  unnecessarily duplicated another’s work. According to the 1991  National Academies report, a premium is placed on making  breakthrough discoveries, and penalties may accrue for doing  unnecessarily duplicative work, including difficulty in obtaining federal  or private funding for future research.
Scientists must stay abreast of new research in their field to stay  competitive. According to numerous scientists and USDA officials we  spoke with, scientists must stay abreast of new research in their fields  to incorporate novel ideas into their research, compete for research  funding, and advance in their careers. Scientists keep current by  reading professional journals; attending scientific conferences and  related gatherings, where they and their peers present new findings;  and communicating with colleagues worldwide on a continual basis.  Moreover, scientists must successfully publish the results of their work  in scientific journals to be competitive for new funding and promotions  at the university level. This practice is also true for scientists at ARS,  according to several ARS officials we spoke with. In fact, several  scientists we spoke with echoed the expression “publish or perish,”  underscoring the need for scientists to publish new findings in journals  to remain competitive. According to USDA officials, the requirement to  publish in peer-reviewed journals is an element in every ARS  scientist’s annual performance plan, and a chronic failure to publish is  grounds for termination of employment. Staying in touch with other  scientists through online networks, conferences, and scientific  publications reduces the likelihood of duplication.
The peer review process used by scientific journals limits the  publication of unnecessarily duplicative research. As described in the  literature we examined and illustrated by scientific journals’  instructions to authors and reviewers, peer review is the means  journals use to vet the accuracy and quality of articles before they are  published. Scientists submit articles detailing their research findings to  a publication of their choice, which then sends the articles to several  scientists working in the research area discussed by the article. These  reviewers provide feedback on submitted articles and recommend to  the publication’s editor whether they think the research merits  publication. As explained in the literature we examined, only articles  that meet certain scientific standards—including building on and not  duplicating other work in the research area, as well as being  sufficiently original, significant, and valid to merit the attention of other  scientists—are accepted for publication. In effect, according to the  1991 National Academies’ report, the peer review process serves to  limit unnecessary duplication of research.
USDA officials we spoke with stated that duplication of research projects  had occurred in previous years, especially before ARS implemented its  external peer review process, and NIFA shifted to more competitive  grants. Moreover, in a March 2012 hearing, the Secretary of Agriculture  testified before Congress that some duplication in research projects may  exist today. During our review, however, USDA officials and stakeholders  could provide no recent examples of duplication within or between ARS  and NIFA, and many stakeholders told us that duplicative research  projects were not a problem for ARS and NIFA today. In addition, we did  not identify duplicative projects within or between the two agencies during  our review of selected research projects. Some stakeholders and USDA  officials called our attention to instances where they believed ARS or  NIFA supported research that appeared duplicative but that, upon further  investigation, was found to be complementary. For example, ARS  researchers who used the agency’s base funding to study the potential of  ultraviolet processing to improve the healthfulness of foods received a  NIFA grant under the Agriculture and Food Research Initiative to  investigate whether ultraviolet light improved the nutritional properties of a  wide range of specialty crops, such as berries and apples. We found that  this work built on previous ARS research in mushrooms in a way that  extended ARS’s research, rather than duplicating it. Other instances  called to our attention were, in fact, cases where industry intentionally  funded research duplicating what ARS or NIFA was already funding to  see which team would find the better solution. Further, we found no  instances of actual duplication within or between ARS and NIFA during  our review of 10 ARS and 10 NIFA research projects (see app. I for more  information).
section_title: Some Shortcomings May Limit the Utility of Certain Agency Safeguards
paragraphs: Our review also identified some shortcomings with certain agency  safeguards, which may somewhat limit their usefulness and increase the  potential risk of duplication between or within the two agencies. Three of  these shortcomings limit CRIS’s utility to guard against duplication of  research projects, specifically, (1) ARS data in CRIS are not current;  (2) NIFA directs staff to conduct a CRIS duplication check for what  amounts to two-thirds, but not all, of its competitive grants; and (3) CRIS  searches are not user friendly.
According to USDA officials, the effectiveness of CRIS in preventing  duplicative research projects depends on the system’s information on  new, ongoing, and completed research. Our analysis of agency  safeguards, however, showed that project information in CRIS may not be  current because ARS has not always sent its project information in a  timely manner to be entered into CRIS by NIFA, which maintains the  system. ARS sends information in batches once or twice per fiscal year  on its newly started or revised projects for uploading into CRIS. In  contrast, NIFA officials stated that they upload information on new and  revised NIFA projects continually throughout the year. According to ARS  officials, the agency sent its project information for uploading only once in  fiscal year 2011 (June), twice in fiscal year 2010 (February and July),  and twice in fiscal year 2012 (February and June). According to ARS  officials, any given batch contains data on all research projects started or  revised since ARS sent its last batch of project data. For example, the  July 2010 batch contained information on all ARS projects started or  revised since March 1, 2010 (see fig. 4). All together, we found that  information in CRIS about ARS projects started from February 2010 to  December 2012 was typically at least 6 months out-of-date; at least one  new project we identified was nearly 21 months old when ARS sent  information about it to NIFA for uploading into CRIS.
ARS project information that is typically at least 6 months out-of-date  when uploaded limits CRIS’s utility as a safeguard to prevent duplication  of research projects and increases the risk of inadvertent project  duplication at the two agencies. For example, a university scientist or  NIFA national program leader searching CRIS for possible duplication  could inadvertently propose or approve a project that unnecessarily  duplicates an ARS project that might have started nearly a year before  but had not been posted in CRIS. Agency information technology staff  and managers agreed that sending ARS project data more often—such  as quarterly—would be feasible, cost no more than 1 additional staff day  of work, and increase CRIS’s utility as a safeguard to prevent project  duplication by keeping ARS’s project information up-to-date. NIFA and  ARS officials also acknowledged that requiring submissions of project  information at least quarterly would help prevent other workload priorities  from delaying ARS submissions. After discussing this topic with us,  agency officials said that ARS now expects its staff to send ARS project  information on a quarterly basis to NIFA for uploading. This expectation,  however, is not documented in guidance. Under federal standards of  internal control, agencies are to clearly document internal control in  writing, in the form of management directives, administrative policies, or  operating manuals.
NIFA directs its staff to conduct a CRIS duplication search for certain  projects accounting for about two-thirds of the funding it awarded for  competitive grants over the last decade. In response to a 2008 USDA  Inspector General’s recommendation specifically directed at grant  applications for the predecessor to the Agriculture and Food Research  Initiative, NIFA developed written guidance directing that such a  duplication check specifically be done for Agriculture and Food Research  Initiative grants. That recommendation advised the agency to develop  written guidance—rather than rely on informal, unwritten guidance—for  these types of grants. As a result, NIFA guidance states that its national  program leaders must search CRIS for possible duplication when  reviewing project applications recommended for funding under its  Agriculture and Food Research Initiative; these projects, however,  accounted for about two-thirds, but not all, of the agency’s competitive  grant funding. The duplication search directive does not apply to projects  funded by NIFA’s other competitive grants or to formula-funded grants.  The amount of funding for competitive grants not subject to this directive  has increased each year from about 31 percent of the funds the agency  awarded for competitive grants in fiscal year 2002 to more than one-third,  or about 37 percent, of the funds the agency awarded for competitive  grants in fiscal year 2011. Thus, an increasing amount of the agency’s  competitive grant funding has been awarded to projects without the  benefit of this safeguard. In contrast, ARS’s CRIS search directive  applies to all project plans proposed by ARS.
Although NIFA responded to the 2008 USDA Inspector General’s  recommendation by issuing written guidance directing staff to conduct  CRIS duplication checks for the Agriculture and Food Research Initiative,  it has not issued similar written guidance for the rest of its competitive  grant programs. According to NIFA officials, many but not all national  program leaders informally conduct CRIS duplication checks when  reviewing competitive grant applications that are not part of the initiative,  even though no written guidance explicitly directs them to do so. The  agency recently convened a task force to study, among other issues,  whether this directive should be extended to other competitive grants. At  the time of our report, the task force was still studying expanding the  CRIS duplication search to all competitive grants and had not yet made a  decision. Under federal standards for internal control, agencies are to  clearly document internal control in writing, and this documentation  should be readily available for examination. Without written guidance  directing staff to conduct CRIS duplication checks for all of NIFA’s  competitive grant programs, the agency does not have an internal control  in place to ensure consistency among national program leaders in  checking for potential duplication when reviewing applications. As a  result, NIFA cannot be assured that grant funds are consistently used for  supporting unique work.
In addition, NIFA does not conduct CRIS duplication checks on formula- funded grants, meaning that research and related work proposed in the  states’ plans of work are also not subject to this safeguard. According to  NIFA officials, the agency’s formula funds are often used for researcher  salaries and preliminary research, and CRIS duplication checks are not  typically applicable. For example, the officials said, enough detail is  seldom contained in these high-level plans of work to check CRIS for  potential duplication on research objectives. Moreover, many projects  proposed for formula funds, such as component parts of multistate  projects, are all required to have the same title; consequently, a search in  CRIS on any one of these would flag the others as being potentially  duplicative, even though they are complementary and intentionally titled  the same.
USDA officials, including those responsible for system management,  acknowledge that CRIS is not user friendly, especially for project  searches. Specifically, they reported that CRIS is not straightforward to  search; links to authors, publications, and patents are limited or  nonexistent; and researcher names are listed inconsistently. For example,  as explained by agency officials, one must go through several nonintuitive  steps to search all text in the system for specific keywords or subjects,  such as “bark beetles” or “bioenergy.” In addition, because CRIS does not  have a standard way of listing researcher names, someone using CRIS  must enter a single researcher’s name into the system in multiple ways— such as Doe, J., and Doe, J. A.—to be certain of finding all projects by  that one researcher. Consequently, CRIS searches are not always  efficient, which, according to some scientists, has discouraged or kept  researchers from using the system for duplication searches. NIFA has  been trying to upgrade CRIS to make it less cumbersome but, according  to USDA officials, it has not secured the resources to do so.
To provide additional capabilities for accessing and disseminating federal  agricultural research, USDA’s Chief Scientist is implementing a new web  platform called VIVO, which, according to USDA officials, has powerful  search functionality for locating scientists and their research. According to  USDA officials, a key purpose of VIVO—which is expected to become  operational in 2013—is to serve as a one-stop source for USDA science  expertise by integrating project information from all five USDA science  agencies, thereby allowing USDA scientists to find possible research  partners and allow those outside USDA to easily find USDA expertise in  specific scientific topics (e.g., honeybees) or geographic areas (e.g., the  Pacific Northwest). Unlike CRIS, VIVO is expected to contain not only  information on ARS and NIFA projects but also active links to scientists  and associated publications, organizations, patents, projects, and  locations and to scientists’ contact information. VIVO also lists  scientists—authors and coauthors—in a standardized format. Further,  USDA officials stated that VIVO will reflect any new improvements made  in the quality of project information fed into CRIS from other USDA project  information systems because these systems of record are the source of  VIVO’s data. For example, VIVO is expected to reflect improvements  under way at NIFA to standardize researchers’ names in CRIS. In  addition, USDA officials stated that they expect a related initiative known  as STAR METRICS, a collaboration between federal science agencies  and research institutions to coordinate federal investments in science, to  build on VIVO and use the data to, among other things, illuminate  research gaps and potentially duplicative research projects at ARS and  NIFA, as well as at other federal science agencies. USDA officials agreed  that VIVO has the potential to be a more robust and user-friendly system  than CRIS in helping to identify USDA scientists and their work and, thus,  in identifying potentially duplicative research projects.
Under policy issued by the Office of Management and Budget in 2000,  agencies are to ensure that the information systems they select  (1) maximize the usefulness of information and minimize any burden on  the public and (2) do not unnecessarily duplicate existing information  technology capabilities within the same agency, from other agencies, or  from the private sector. USDA officials have identified potential benefits of  VIVO for both elements of this 2000 policy because of VIVO’s more  robust search capabilities. They said, however, that the department has  no plans to assess whether VIVO significantly improves search  capabilities compared with CRIS or is more effective or efficient than  CRIS in identifying potentially duplicative research projects within and  between ARS and NIFA. As a result, USDA has not assessed its  information systems in a way that would allow it to (1) maximize the  usefulness of information and minimize the burden on the public and  (2) avoid unnecessarily duplicating existing information technology  capabilities, as described in the 2000 Office of Management and Budget  policy. Unless USDA officials assess VIVO’s effectiveness and efficiency,  and unless USDA officials establish written procedures for staff to use the  best method when searching for duplicative research projects, ARS and  NIFA scientists and managers could remain dependent on a less robust  safeguard.
section_title: High-Level Collaborative Planning Has Increased, but Agency-Level Collaborative Planning between ARS and NIFA Has Not Been Systematic
paragraphs: Over the past few years, USDA’s Chief Scientist has facilitated increased  collaborative planning within the REE mission area, particularly between  ARS and NIFA, but agency-level collaborative planning between these  two agencies has not been systematic. Specifically, the Chief Scientist  and her staff have led several REE-level planning efforts that, over the  past several years, have brought together staff from the two agencies and  generated key products. Nevertheless, six USDA officials with planning  responsibilities said that collaborative planning between ARS and NIFA  national program leaders has not been systematic for topic areas the  agencies have in common, and 20 officials and stakeholders said that  ARS and NIFA should systematically hold joint planning and stakeholder  input meetings to make the best use of limited agricultural research  resources.
subsections: section_title: High-Level Collaborative Planning Has Generated Key Products
paragraphs: Over the past few years, the Chief Scientist, her senior advisors, and  other staff have led efforts that have facilitated REE-level collaborative  planning among ARS, NIFA, and the other REE mission area agencies,  and these efforts have resulted in several key products. First, the Chief  Scientist led a collaborative planning effort that involved the four REE  agencies to develop the REE Action Plan. The Chief Scientist and her  staff established seven teams, focused on the action plan’s seven goals,  which held planning meetings from November 2010 to February 2012 and  involved staff from multiple agencies in developing the subgoals,  strategies, planned actions, and agency responsibilities corresponding to  the action plan’s goals. For each goal and subgoal, the teams also  identified specific challenges facing agriculture, natural resources, and  conservation to be addressed by the planned actions. Several USDA  officials told us that because the meetings led to more collaboration  across REE agencies, these planning meetings were a major benefit of  REE Action Plan development. For example, according to one USDA  official, discussions among USDA officials during meetings about a  subgoal related to energy and biofuels research highlighted the fact that  research areas overlapped between USDA and the Department of  Energy, creating the potential for duplicative work. Such discussions  suggested a solution—a decision to focus USDA research efforts on  developing biofuel feedstocks (i.e., materials such as perennial grasses  from which biofuels are generated) and to move away from technology for  converting materials into biofuels, a research area for which the  Department of Energy has predominant responsibility. Further, according  to this official, the discussions between ARS and NIFA officials during the  development of the REE Action Plan clarified each REE agency’s role in  each of the goals, strategies, and planned actions. Overall, this planning  effort made use of practices consistent with two we identified in our  October 2005 report as helpful to enhance and sustain agency  collaboration, in particular, establishing mutually reinforcing or joint  strategies and agreeing on roles and responsibilities.
Second, the Chief Scientist’s senior advisors led a collaborative effort  from early 2011 to mid-2012 that resulted in six specific “white papers,”  issued by the Office of the Chief Scientist, on the topics of global food  security, bioenergy, nutrition and childhood obesity, climate change, food  safety, and sustainable agricultural systems. These white papers were  developed through a series of meetings and discussions with multiagency  teams from ARS, NIFA, and other USDA agencies, and each white paper  was independently peer reviewed. The senior advisors worked with these  teams to describe in more detail challenges and strategies identified in  the REE Action Plan, including (1) roles to be played by the agencies in  addressing each challenge, (2) key strategies to focus on in fulfilling those  roles, (3) USDA’s current actions, (4) USDA’s future plans, and (5) the  research outcomes that USDA anticipates. Like the collaborative planning  that led to the REE Action Plan, the white paper planning effort also made  use of collaboration practices consistent with several we previously  identified, particularly, defining and articulating a common outcome,  establishing mutually reinforcing or joint strategies, and agreeing on roles  and responsibilities.
Third, the Chief Scientist’s senior advisors stated that in 2010 and 2011  they coordinated input from ARS, NIFA, and other agencies to help  develop a number of products, including Feed the Future: Global Food  Security Research Strategy. Issued in May 2011, this global research  portfolio serves as an integral part of the President’s broader Feed the  Future initiative. Specifically, the senior advisors said that they  established multiagency working groups of USDA officials, who worked  with their counterparts in the Agency for International Development to  identify research priorities—aimed at sustainably advancing productivity,  transforming production systems, and enhancing nutrition and food safety  around the world—to help implement this strategy.
The Chief Scientist’s senior advisors also facilitated collaborative planning  efforts not resulting in products. For example, in September 2012, one  senior advisor began an effort to better coordinate ARS’s intramural and  NIFA’s extramural research on nutrition and obesity. In this instance, the  Chief Scientist provided guidance for the initial joint meeting, and the  senior advisor scheduled follow-up meetings to address unresolved  issues.
section_title: Several USDA Officials Said That Collaborative Planning between ARS and NIFA Has Not Been Systematic
paragraphs: Program leaders in ARS and NIFA meet throughout the year for various  purposes, but, according to six USDA officials who have planning  responsibilities, the program leaders have not systematically held joint  planning meetings to help determine research priorities for topic areas the  agencies have in common or their roles and responsibilities for  addressing these priorities. According to our review of meeting agendas  and other agency documents, ARS and NIFA program leaders meet with  stakeholders—such as agricultural producers and industry  representatives—at scientific society, commodity group, stakeholder  input, and other meetings throughout the year. The six USDA officials  acknowledged, however, that the two agencies do not systematically plan  or hold joint stakeholder input meetings, nor do they hold joint planning  meetings. Rather, each agency generally holds separate stakeholder  input meetings, even when agency research topics and stakeholders are  the same or similar. For example, according to our review of meeting  documentation, ARS held most of its recent stakeholder input meetings  for its national programs independently. Even though ARS and NIFA  generally hold such meetings independently, ARS’s and NIFA’s websites  both identify collaborative ARS-NIFA stakeholder input meetings for the  animal sciences programs. According to these websites and agency  documents, animal sciences is the one broad topic area in which the  agencies have for several years systematically held joint ARS-NIFA  national stakeholder input meetings. The animal sciences team also holds  joint ARS-NIFA planning meetings afterward to address stakeholder input  and help each agency determine its research priorities as well as roles  and responsibilities. In contrast, according to the six officials, national  program leaders have not systematically held joint planning meetings to  (1) help the agencies determine their research priorities for topic areas  they have in common or (2) help each agency determine its roles and  responsibilities for addressing these priorities.
According to USDA officials and stakeholders we interviewed,  collaborative agency planning is important because the collective  knowledge of both ARS and NIFA is needed to ensure that the agencies’  research areas and projects address top priorities and that research is  complementary and not duplicative. In addition, these officials and  stakeholders said that collaborative planning helps identify the  appropriate entity—whether ARS itself, universities, or other  organizations—to address a research problem. Each entity, they said,  may have unique expertise, facilities, and equipment, and no one agency  or official has all the relevant knowledge.
Over the last decade, USDA and others have emphasized the need for  enhanced collaborative planning and joint stakeholder input, but USDA  has not implemented key recommendations. USDA’s 2010 Roadmap  states that solutions to many of the nation’s most serious problems  demand change that USDA will bring about, in part, by better coordinating  its science planning among and between REE science agencies and with  other federal science agencies and by listening to the needs of  stakeholders. Also, a 2010 annual report in the plant sciences area cited  the lack of strategic planning for research across federal agencies and it  stated that regular team meetings held quarterly would help improve  collaboration within NIFA.  In addition, reports by USDA and others  made specific recommendations to enhance collaborative planning and  joint stakeholder input. USDA, however, has not implemented these  recommendations. For example, USDA has not implemented the  recommendation from an ARS-NIFA-sponsored study that the agencies  develop a unified approach to the stakeholder community to address and  solve critical problems. Neither has USDA implemented the  recommendation in a 2003 National Research Council report that REE  hold a national summit every 2 to 3 years to engage the agencies and a  broad spectrum of national, regional, and local stakeholders.
The Chief Scientist told us that she envisions greater collaborative  planning between the agencies in the future through, in particular, more  joint meetings for obtaining stakeholder input and discussing agency-level  research plans to support the REE Action Plan. Several USDA officials  and stakeholders told us that joint ARS-NIFA stakeholder input should be  sought at least every 5 years, to coincide with ARS’s 5-year planning  cycle, and suggested that the two agencies hold joint planning meetings  annually. In addition, two industry stakeholders said that joint meetings  would benefit them by increasing their contact with NIFA national program  leaders, whom they may not know as well as they know ARS national  program leaders.
Promising examples of systematic collaborative stakeholder input and  planning, initiated independently by officials and individual teams within  ARS and NIFA, already exist in one broad topic area—animal sciences— and in more limited areas that may fall within a national program or broad  topic area. Specifically, for more than a decade, an animal sciences team  of ARS and NIFA national program leaders has jointly sponsored large  stakeholder meetings led by both agencies every 5 years, with smaller,  annual stakeholder meetings on more specific topics (e.g., swine). The  team started holding these meetings after a stakeholder requested such  collaboration to save time spent attending two sets of similar meetings.  The national program leaders meet after the stakeholder meetings to  discuss ongoing research projects and associated resources, gaps in  addressing stakeholders’ needs and priorities, and their respective  agency’s potential roles and responsibilities; the program leaders also  meet monthly by telephone or in person. Collectively, these meetings  facilitate development of specific ARS and NIFA plans or requests for  application. Several USDA officials and stakeholders we interviewed  praised the animal sciences team’s collaborative effort as a best practice,  and the team was nominated for a USDA-wide award for sustained  collaboration that “has facilitated strong engagement with stakeholders  and leveraged intramural and extramural programs to benefit U.S. animal  agriculture.” Two industry stakeholders we spoke with who participate in  these joint meetings said that the meetings assure them that the two  agencies are coordinating their research plans.
According to USDA officials, many benefits have come from the animal  science team’s collaboration. For example, according to USDA officials,  this team’s collaborative planning led to the development of a network of  researchers working to enable development of veterinary vaccines for  cattle, swine, poultry, and fish. This network now includes scientists from  ARS, several universities, and industry and coordinates with similar  endeavors internationally. According to a USDA official and the network’s  website, products developed from these collaborative efforts benefit a  large group of researchers, including veterinary clinicians, immunologists,  pathologists, and microbiologists. Additionally, joint stakeholder and  planning meetings hosted every 5 years by the aquaculture subgroup of  the animal sciences team have led to NIFA funding for a 2008 Regional  Collaborative Research and Extension Catfish Forum, linking ARS, NIFA  multistate committees, and land-grant college and university programs.  According to NIFA and ARS aquaculture program leaders, the  collaboration helped bring about the development of an innovative catfish  production system that is now used commercially.
USDA officials also provided examples of other ARS-NIFA collaborative  stakeholder input meetings and planning efforts that are more limited in  scope or fall within a national program or topic area. For example, the  National Plant Germplasm Coordinating Committee was established in  2006. Largely funded by ARS, with support from NIFA and universities,  the National Plant Germplasm System protects plant hereditary material,  or germplasm, including seeds, to preserve the genetic diversity of our  nation’s crops. The coordinating committee, comprising ARS and NIFA  officials and academic scientists, meets annually to review and set  national scientific priorities for the system. The committee also initiates a  strategic planning effort to better define and communicate a vision,  mission, and short- and long-term goals for the system. In addition, in the  area of food safety, ARS and NIFA national program leaders hold a joint  annual meeting with a group of key stakeholders—three federal agencies  involved in regulating food safety and preventing foodborne illness—to  identify the research needs of these stakeholders. ARS and NIFA then  use this information to help coordinate each agency’s research plans.
Even though the benefits of collaboration are widely recognized, part of  the reason that ARS-NIFA collaborative planning may not be systematic,  agency officials told us, is that REE does not require its agencies to  systematically hold joint planning meetings or jointly meet with  stakeholders. Moreover, some USDA officials we interviewed stated that  it is a challenge for national program leaders—at both ARS and NIFA,  where many national program leaders have heavier workloads because  of recent retirements—to make systematic collaboration a high priority.  One NIFA official stated that without a requirement for systematic  collaborative planning, such planning is unlikely to increase. By  enhancing collaborative planning for topic areas that ARS and NIFA have  in common, as the animal sciences area has done, the agencies can take  full advantage of their collective knowledge and expertise in setting their  research priorities and determining their roles and responsibilities for  carrying out these priorities.
section_title: Conclusions
paragraphs: Agricultural science, like science in general, advances through research  that may at first seem duplicative but may (1) actually replicate, and  thereby validate or disprove existing research, or (2) complement and  extend existing research. Recognizing the importance of both replication  and complementarity in science—and in light of some duplication at  USDA’s largest research agencies in the past and the potential for some  duplication in the future—ARS and NIFA have each developed important  agency safeguards for reviewing projects, which work in tandem with the  scientific community’s professional norms, to prevent duplicative research  projects within and between the agencies. We commend the agencies for  developing multitiered safeguards. Modifications to these safeguards,  however, could better position both agencies to further reduce the risk of  duplicative research projects in the future. For example, information on  ARS projects is not current in CRIS because ARS has no written  guidance for how often to send its project information to NIFA for  uploading into the system; as a result, ARS has not always sent its  project information in a timely manner. Thus, ARS project information that  is typically at least 6 months out of date limits CRIS’s utility as a  safeguard against duplicative research. In addition, since NIFA does not  direct that all competitive grant applications recommended for funding be  checked against CRIS for potential duplication, about one-third of the  funds NIFA awards in competitive grants go toward projects that are, in  effect, not subject to a key agency safeguard to prevent potential  duplication and are thus at increased risk of inadvertent duplication. NIFA  recently convened a task force to study, among other issues, whether this  directive should be extended to other competitive grants. At the time of  our report, the task force was still studying expanding the CRIS  duplication search to all competitive grants and had not made a decision.  Moreover, without an assessment of whether the search functionality of  the forthcoming VIVO platform is more effective or efficient than CRIS at  finding duplicative research projects, and until assessment results are  used to drive procedures for identifying potentially duplicative projects,  scientists and national program leaders cannot be assured that they are  using the best method or system for detecting duplication. Finally, by  successfully engaging ARS, NIFA, and other REE agencies, USDA’s  Chief Scientist has enhanced high-level collaborative planning. In  addition, some collaborative planning initiated jointly by ARS and NIFA  has occurred across national program areas, such as in the animal  sciences area. By enhancing collaborative planning—including planning  that involves key stakeholders—for topic areas the agencies have in  common, ARS and NIFA national program leaders can consistently take  full advantage of their collective knowledge and expertise to help set their  research priorities and help the agencies determine their roles and  responsibilities for carrying out these priorities.
section_title: Recommendations for Executive Action
paragraphs: To help further reduce the risk of duplicative research within and between  ARS and NIFA and to improve the agencies’ collaborative planning, we  recommend that the Secretary of Agriculture direct the Under Secretary of  Research, Education, and Economics to take the following four actions:
Require that ARS issue written guidance specifying that it should  update its research project data in CRIS at least quarterly and ensure  that this requirement is also applied to any new systems that receive  these project data.
Require that NIFA issue written guidance that its national program  leaders ensure that staff check all of NIFA’s competitive grant awards  against CRIS for potential duplication.
Require, as USDA identifies promising systems for searching and  disseminating research project information, the appropriate entity to  (1) determine whether VIVO or other systems are more effective or  efficient than CRIS in identifying potentially duplicative research and  (2) revise guidance to reflect the best system and methods for  identifying potentially duplicative research projects.
Enhance ARS’s and NIFA’s collaborative approach across all topic  areas they have in common—including holding jointly sponsored  meetings with stakeholders—to help set their research priorities and  help them determine their roles and responsibilities for carrying out  these priorities.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a copy of the draft report to the Department of Agriculture for  its review and comment. In written comments from the department, which  are reproduced in appendix II, the Chief Scientist stated that the  department generally agreed with our findings. Nevertheless, USDA  responded to the statement in our report that “collaborative planning  between ARS and NIFA has not been systematic.” In our report, we  define the term collaborative planning as the act of bringing research  agencies and stakeholders together to discuss priorities, roles, and  responsibilities, and we use the term systematic to mean marked by  thoroughness and regularity. In its comments, USDA outlined six steps  within REE that together it states constitute a system for collaborative  planning, noting that the report accurately describes these steps. As we  describe in the report, some of these steps have contributed to  collaborative planning conducted jointly by ARS and NIFA across national  program areas, such as in the animal sciences area. We concluded,  however, that by enhancing collaborative planning—including planning  that involves key stakeholders—for all topic areas the agencies have in  common, ARS and NIFA national program leaders can consistently take  full advantage of their collective knowledge and expertise to help set their  research priorities and help the agencies determine their roles and  responsibilities for carrying out these priorities. Such an approach would  be more systematic in carrying out collaborative planning.
In addition, while neither explicitly agreeing nor disagreeing with our  recommendations, the Chief Scientist stated that the department will give  strong consideration to our recommendations as it seeks to improve how  it identifies fruitful approaches to address the highest priority food and  agricultural research needs. In these written comments, the Chief  Scientist did not comment specifically on one of our recommendations:  that USDA require that NIFA issue written guidance that its national  program leaders ensure that staff check all of NIFA’s competitive grant  awards against CRIS for potential duplication. Nonetheless, the written  comments cite the benefits of implementing the three other  recommendations. The following list summarizes the department’s  comments on our recommendations:
For our recommendation that USDA require that ARS issue written  guidance specifying that it should update its research project data in  CRIS at least quarterly and ensure that this requirement is also  applied to any new systems that receive these project data: In the  written comments, the Chief Scientist stated that such written  guidance to direct more frequent updates of ARS data “to common  data systems could benefit the use of these systems as decision- support tools.” The Chief Scientist also stated that such a “benefit  extends to several efforts under way in REE to develop and improve  common data systems,” including VIVO and others mentioned in the  report.
For our recommendation that USDA require, as it identifies promising  systems for searching and disseminating research project information,  the appropriate entity to (1) determine whether VIVO or other systems  are more effective or efficient than CRIS in identifying potentially  duplicative research and (2) revise guidance to reflect the best system  and methods for identifying potentially duplicative research projects:  In the written comments, the Chief Scientist stated that, as the  initiatives “mature and provide validated improvements over existing  systems, guidance for use of the best available tools for safeguarding  against duplication” could “enhance this benefit.”
For our recommendation that USDA enhance ARS’s and NIFA’s  collaborative approach across all topic areas they have in common— including holding jointly sponsored meetings with stakeholders—to  help set their research priorities and help them determine their roles  and responsibilities for carrying out these priorities: While USDA did  not expressly agree or disagree with this recommendation, it stated in  its comments that it expects ARS and NIFA to “continue to increase  the number of jointly sponsored planning activities” and that ARS- NIFA joint stakeholder meetings in Plant Sciences and Aquaculture  are already planned for 2013.
We continue to believe that all four recommendations would enhance  safeguards against project duplication and strengthen collaborative  planning.
USDA’s Chief Scientist also provided technical comments, which we  incorporated into the report as appropriate.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of  Agriculture, the appropriate congressional committees, and other  interested parties. In addition, the report will be available at no charge on  the GAO website at http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at (202) 512-3841 or garciadiazd@gao.gov. Contact points for   our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. Key contributors to this report are listed in  appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to examine (1) the topics the Agricultural Research  Agency (ARS) and the National Institute of Food and Agriculture (NIFA)  focus on and the safeguards they use to prevent duplication of research  projects, along with any shortcomings in those safeguards, and  (2) collaborative planning ARS and NIFA engaged in and how, if at all,  such planning could be enhanced.
For both objectives, we collected and reviewed relevant law and agency  guidance, as well as reports from the National Academies; the Inspector  General of the U.S. Department of Agriculture (USDA); the National  Agricultural Research, Extension, Education, and Economics Advisory  Board (REE advisory board); and others. We also interviewed and  gathered relevant documents from headquarters and field officials of  USDA, including the Office of the Under Secretary for Research,  Education, and Economics (REE); Office of the Chief Scientist; ARS;  NIFA; and the Economic Research Service. We interviewed officials on  the telephone and during our two site visits—one to ARS and NIFA  headquarters offices in the Washington, D.C., area and the other to the  University of California, Davis, and ARS’s field office in Albany, California.  These officials included program leaders and other managers and  scientists. In addition, we interviewed 16 stakeholders from the REE  advisory board; industry organizations; and academic organizations,  including land-grant colleges and universities. Stakeholders representing  industry organizations came from the Florida Citrus Processors  Association, American Dairy Science Association, National Pork Board,  Iowa Soybean Association, and the National Grape and Wine Initiative.  Stakeholders representing academic organizations were current or former  officials of the Association of Public and Land-grant Universities; the  University of Arkansas; the University of California, Davis; the University  of Georgia Agricultural Experiment Stations; and Ohio State University.  We selected industry and academic stakeholders on the basis of input  from officials with USDA and the National Academy of Sciences’ Board  on Agriculture and Natural Resources.
To examine the topics ARS and NIFA focus on, we collected and  analyzed agency funding, spending, and planning data related to the  topics on which ARS and NIFA focus their work. We collected and  analyzed the President’s budgets to identify total ARS and NIFA funding  for fiscal years 2002 through 2011 and NIFA budget data on the  breakdown of its funding by mechanism (formula funding, competitive  grants, and noncompetitive grants) for fiscal years 2001 through 2011.  We assessed the reliability of these data and found them to be sufficiently  reliable for our purposes. We also collected and analyzed spending by  ARS and NIFA on research from fiscal year 2002 through fiscal year  2011. To assess the reliability of spending data provided from NIFA’s  Current Research Information System (CRIS)—USDA’s primary database  of completed and ongoing projects, which includes both financial and  project information—we interviewed knowledgeable officials about the  data, possible limitations of the data, and internal controls for the system  containing the data. We found these data to be sufficiently reliable for our  purposes. Nevertheless, our ability to quantify and compare the agencies’  spending on research topics was limited because, unlike ARS, NIFA  could not isolate its research spending. Consequently, NIFA could only  provide research spending data that also included an unknown amount of  spending on education and extension activities. Moreover, we could  compare ARS and NIFA research topics only at a high level and could not  comparatively assess in detail the agencies’ spending on subtopic areas,  such as which agency spent more money or a greater share of its money  on childhood obesity or colony collapse disorder in honeybees. We also  collected and analyzed estimates provided by NIFA on its research  spending for fiscal years 2002 through 2011. These estimates are the  same as those provided annually to the National Science Foundation. We  assessed the reliability of these estimates by interviewing knowledgeable  officials and comparing these estimates against data on spending and  funding, and we determined them to be sufficiently reliable for our  purposes.
To examine the safeguards used by the agencies to identify and prevent  duplication within and between ARS and NIFA, we interviewed current  and former scientists and agency officials, reviewed relevant scientific  literature discussing professional norms and practices, and analyzed  documentation of safeguards used at each agency to identify and prevent  duplicative research. To identify shortcomings in those safeguards, we  interviewed agency officials and analyzed relevant guidance. Also, we  used agency guidance on searching CRIS to review for possible  duplication a total of 10 ARS and 10 NIFA projects started in 2011,  randomly sampled from the animal sciences and human nutrition areas.  This review did not assess the extent of duplication across all NIFA and  ARS projects. To ensure the reliability of the review, we also worked with  knowledgeable USDA officials to develop the formulas used to query  CRIS and obtain a master list of ARS and NIFA projects started in 2011  from which to derive a sample. As part of this review, we followed ARS  and NIFA guidance to search CRIS by keywords and scientists’ names  for potentially duplicative research projects within and between the two  agencies. One analyst reviewed all 20 selected projects to determine  whether they were potentially duplicative or, rather, complemented or  replicated existing work. A second analyst then reviewed the first  analyst’s initial determinations. We resolved any disagreement between  the analysts through discussion and afterward obtained additional input  from relevant ARS and NIFA scientists and officials for any project that  either analyst determined to be potentially duplicative. Our review of  selected projects flagged one ARS project and one NIFA project as  potentially duplicative; after discussion with agency officials, however, we  concluded that neither project was in fact duplicative. In ARS’s case, the  same project was recorded twice because of two external funding  sources, and in NIFA’s case, an existing formula-funded project  supported the collection of preliminary data for a related competitive- grant-funded project.
To examine collaborative planning engaged in by ARS and NIFA and  how, if at all, such planning could be enhanced, we analyzed the Chief  Scientist’s actions aimed at increasing collaboration and the products  resulting from these actions. We also interviewed USDA officials and  stakeholders regarding the effects of these actions. To examine the  collaborative planning ARS and NIFA engaged in, we reviewed agency  plans and agendas and other documents related to stakeholders’ input  into research priorities. In addition, we examined our own and USDA’s  documents, issued from 1996 to 2010, discussing the need for improved  collaboration and planning. For example, we reviewed two of our reports  on barriers agencies face when attempting to collaborate on selected key  practices that can help enhance and sustain collaboration among federal  agencies. To identify practices that can enhance and sustain agency  collaboration, we also reviewed a 2005 USDA-sponsored study on  improving ARS-NIFA collaboration, two reports we issued on federal  agency collaboration, other reports of ours on agricultural research  planning, and National Research Council studies. Finally, we analyzed  our interviews with USDA officials and stakeholders to report their views  on whether collaborative planning—including stakeholder input meetings  and priority-setting meetings—was sufficient and whether they had any  recommendations that could enhance such collaborative planning. We  also examined whether the agencies’ collaborative efforts reflected key  practices we identified in October 2005 that can help enhance and  sustain collaboration among federal agencies.
We conducted this performance audit from January 2012 through April  2013 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the U.S. Department of Agriculture
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
paragraphs: Daniel Garcia-Diaz, (202) 512-3841 or garciadiazd@gao.gov.
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Karen A. Jones (Assistant  Director), Kevin Bray, Tony Calero, Ellen W. Chu, Paula Moore, Beverly  Peterson, Dan Royer, Michael Silver, and Tama Weinberg made  significant contributions to this report.