National Institutes of Health: Research Priority Setting, and Funding Allocations across Selected Diseases and Conditions
Released: 2014-04-30T00:00:00

Letter

Background

NIH’s Organization

As the central office for NIH, OD establishes NIH policy and broad themes for the agency to pursue, such as ensuring a sustainable scientific workforce, based on national needs and scientific opportunities. In addition, the OD is responsible for coordinating the programs and activities that span NIH components, particularly research initiatives and issues involving more than 1 of the 27 ICs. OD is also responsible for ensuring that scientifically-based strategic planning is implemented in support of research priorities, and that NIH’s resources are sufficiently allocated for research projects identified in strategic plans. NIH conducts and sponsors biomedical research through its ICs, each of which is charged with a specific mission. ICs’ missions generally focus on a specific disease, a particular organ, or a stage in life, such as childhood or old age. The ICs support, plan, and manage their own research programs in keeping with OD policy and priorities. Within an IC, there can be a number of offices, centers, and divisions that focus on specific aspects of the IC’s mission. For example, NCI has a Division of Cancer Epidemiology and Genetics, as well as a Division of Cancer Treatment and Diagnosis.

Extramural and Intramural Research Supported by NIH

ICs accomplish their missions chiefly through extramural and intramural research. Extramural research accounted for more than 80 percent of NIH’s budget in fiscal year 2012. This research is conducted at 2,500 universities, medical schools, research organizations, and companies who are awarded extramural research grants or extramural research and development contracts through NIH’s competitive process. Twenty-four of the 27 ICs fund extramural research projects, and these ICs make final decisions on which extramural research projects to fund following a standard peer review process.what NIH considers “unsolicited” research and research training projects for which applications are submitted in response to broad funding opportunity announcements that span the breadth of the NIH mission. In addition, to encourage and stimulate research and the submission of research applications in specific areas, many ICs will issue solicitations that are more narrow in scope in the form of , requests for applications, and requests for proposals. When reviewing applications for extramural research projects, NIH follows a process of peer review, established by law.sequential levels of peer review. According to NIH officials, the first level involves panels of experts to assess the scientific merit of the proposed science. The second level involves panels of experts and leaders of non- science fields including patient advocates that, in addition to scientific merit, also consider the IC’s mission and strategic plan goals, public health needs, scientific opportunities, and portfolio balance. After NIH’s Most IC extramural funding is provided for This peer review system has two peer review process is concluded, IC directors make extramural funding decisions.

Intramural research, which accounts for approximately 10 percent of NIH’s budget, is conducted by NIH scientists in NIH laboratories. This includes about 5,300 scientists and technical support staff who are employees, and another 5,000 young scientists at various stages of research training who come to NIH for a few years to work as non- employee trainees, including 3,800 postdoctoral fellows. All but 3 of the 27 ICs have an intramural research program, but the size, structure, and activities of the programs vary greatly.

NIH’s Research, Condition, and Disease Categorization System

In January 2007, Congress directed NIH to establish an electronic system to categorize the research grants and activities of OD and all the ICs. In response, NIH created RCDC. Implemented in February 2008, RCDC uses a computer-based text-mining tool that recognizes words and phrases in project descriptions in order to assign NIH projects to at least one of 235 categories of diseases, conditions, and research areas that were developed for reporting to Congress and the public. NIH officials said that RCDC serves as NIH’s primary computerized reporting process to categorize its research funding. The system includes reporting tools that can be used to generate publically-available, web-based reports on total funding amounts for the research projects related to each RCDC category.

Individual Institutes and Centers Set Their Own Research Priorities Using a Variety of Approaches, Including Strategic Planning, Annual Planning, and Review of Research Portfolios

Individual ICs at NIH set their own research priorities, and we found that the five selected ICs we reviewed did so considering similar factors and using various priority-setting approaches. Officials at all five of the ICs stated that their mission and available appropriations inform priority setting approaches. Officials at one IC noted that their IC’s mission provides context related to why the IC was developed initially and insight into the emerging areas of research. Officials stated that an IC’s appropriations not only set funding parameters, but may also influence priority setting if the appropriations include mandated spending by Congress for a specific disease. Some IC officials noted that because the costs of potential research projects generally exceed the available appropriation, the ICs generally must prioritize among research projects. In priority setting, IC officials also reported taking into consideration scientific needs and opportunities, gaps in funded research, the burden of disease in a population, and public health need, such as an emerging public health threat that needs to be addressed, like influenza.

While individual ICs used different approaches to priority setting, all five selected ICs we reviewed reported using some combination of strategic planning, annual planning, and periodic review and evaluation of their research portfolios as part of their approach to priority setting.

Strategic planning: All five ICs we interviewed developed strategic plans—consistent with law and NIH policy—to set priorities and goals for research funding. According to officials at selected ICs, the development of these plans is guided by various processes at each IC. Although these processes vary by IC, they include an opportunity to solicit input from stakeholders, including the scientific community, as well as review by IC staff and leadership. Examples of IC strategic planning activities include the following:

NIAID used a framework organized around various scientific areas that encompassed its research to develop the strategic plans it published in 2000, 2008, and 2013. When developing the plans, NIAID convened groups of NIAID subject matter experts for each scientific area and considered input offered by external organizations and scientists to guide the process. The experts deliberated priorities identified for each scientific area and suggested revisions to the draft plan. For example, officials stated that NIAID has seen a shift in the emphasis on research efforts related to human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) from studying single-drug therapy to the current focus on finding improved HIV treatments and tools for preventing infection. This new focus was then incorporated as part of the strategic plan published in 2013. Draft strategic plans were reviewed by the NIAID Advisory Council. After a final review by the director of NIAID, the finalized plans were published on NIAID’s web site.

NHLBI officials said that, as of November 2013, the IC was revising the strategic plan it published in 2007 and part of its strategic planning process under the direction of a new director appointed in 2012. As part of the revised process, NHLBI created the Strategic Investment and Management Steering Committee that will help establish strategic goals as well as advise leadership about whether proposed activities are consistent with the IC’s strategic plan. Officials stated that the strategic planning process will engage both NHLBI scientific staff as well as the larger extramural community because they are knowledgeable about research gaps within their respective fields. Further, NHLBI officials stated that input provided by stakeholders also helps to inform development of disease-specific planning documents, such as one for asthma and another on sleep disorders.

Annual planning activities: In addition to the development of strategic plans, some IC officials we interviewed told us that they conduct annual planning activities as a part of their process to set priorities for research funding. They told us that these annual planning activities address ongoing and emerging needs. While the processes vary across the ICs in terms of the level of structure and formality, officials stated that during the annual planning process these ICs typically consider factors such as scientific needs, research gaps, scientific opportunities, public health needs, and the need to balance the various types of research conducted to address all areas of the mission. Examples of annual planning activities include the following:

NIAID uses an annual planning process to develop and select initiatives that address special needs, gaps, and opportunities in relevant research areas. According to NIAID officials, these initiatives are developed through a structured process centered around two primary events—the annual Summer Policy Retreat and the annual Winter Program Review. NIAID staff also review their scientific areas for the latest information on burden of illness and state of scientific progress, which is used to inform the development of future initiatives. According to agency officials, NIAID staff use this information, as well as input from the scientific community, to prepare and present a concept of an initiative to NIAID’s National Advisory Council, which reviews, comments on, and decides whether to approve the initiative. Once an initiative has been approved, NIAID solicits research applications through a request for application, a program announcement, or a request for proposal.

NIDDK uses an annual process that requires NIDDK scientific program staff within the IC’s divisions to review the research portfolio and to identify research opportunities. According to NIDDK, this effort is informed by a number of factors, such as ongoing input from the extramural research and health advocacy communities and recent research advances. They said that each division then presents initiative concepts built on these research opportunities at NIDDK senior leadership meetings. The NIDDK director prioritizes the initiative concepts and determines which will move forward based on consideration of merit and other factors. These concepts go through a clearance process, including review from the NIDDK National Advisory Council. NIDDK develops funding opportunity announcements based on these identified research opportunities, such as requests for application that address specific scientific questions or diseases. In addition, NIDDK conducts divisional planning and prioritization activities during monthly meetings, retreats, and other activities, including consideration of opportunities for funds to support targeted emphasis areas.

Review and evaluation of research portfolio: Officials from the five selected ICs we interviewed stated they conduct reviews of their research portfolio to help ensure existing priorities reflect and align with current scientific opportunities, research gaps, and emerging science. This includes periodic formal program assessments of their research portfolio, which the ICs used to determine if the IC is meeting its overall priorities and goals, to maintain portfolio balance, and to make any changes to priorities over time as science evolves. Examples of portfolio review and evaluation activities include the following:

NCI officials stated that the IC conducts portfolio analyses, which involve examining research areas NCI has funded to identify opportunities for research. Officials stated that the portfolio analysis is an iterative process that occurs continually throughout the year. One example of this portfolio analysis has involved publishing broad questions related to five themes for the research community on NCI’s website and using the responses received to inform development of requests for application and program announcements published by NCI. Officials said they aim to generate about four new questions each year and that the questions are generally very broad and do not usually address specific diseases. For example, one question recently asked how the level, type, or duration of physical activity influences cancer risk and prognosis.

NIGMS conducts formal program evaluations to determine what benefits the program has produced overall, including scientific advances, new technologies, and cutting-edge paradigm shifts, such as changes in the understanding of human biology. In addition, officials told us that NIGMS divisions conduct an annual review, which is generally prepared for NIGMS’s director, and provides information about the scientific advances that can be attributed to the work conducted in each division. This includes items such as the number of grants funded, the number of new principal investigators, the types of research conducted, and an overview of any new science performed.

While the individual ICs set their own priorities, according to NIH officials, the OD provides leadership to the ICs and helps coordinate priority setting activities, especially for efforts that involve multiple ICs. For example, NIH officials reported that the director of NIH meets with all IC directors weekly to discuss research priorities, investments, and concerns that may affect an IC or NIH overall. In addition, the Advisory Committee to the Director assists the director of NIH with making major planning and policy decisions, including those related to research priorities and the allocation of funds for major, NIH-wide projects and identifies areas that have the potential for significant improvements both within NIH and within the scientific community at large. For example, the NIH Director charged the committee with examining NIH’s data needs to determine whether the agency is positioned to handle significant amounts of data and how to make these data accessible to researchers. In addition, NIH established a special division within the Office of the Director—the Division of Program Coordination, Planning, and Strategic Initiatives—to help manage large and complex research portfolios across NIH. The offices within this division organize formal discussions to obtain input from all of the IC directors as well as members of the scientific community to identify areas of significant scientific interest or need that typically span the interests of multiple ICs. NIH officials also stated that although each IC has at least one strategic plan in place, in some instances NIH has developed strategic plans for particular disease or topic areas that cross multiple ICs, such as one related to women’s health research.

NIH Funding Levels Varied Widely For Selected Categories of Research, Conditions, and Diseases Corresponding to the Leading Causes of Death and Chronic Conditions

In fiscal year 2012, NIH reported funding levels that ranged widely—from $13 million for projects in one RCDC category to more than $5.6 billion for another—for the 40 different RCDC categories we examined. (See table 1.) We determined that these RCDC categories were the best match for the most frequent causes of death in the United States, the most frequent causes of death globally, and the most prevalent chronic conditions in the United States. (See app. II for an explanation of how we identified these diseases and conditions and matched them with RCDC categories.)

NIH officials said that RCDC is the agency’s official system for reporting research funding across the agency’s ICs and it provides a method for reporting consistent information about NIH funding. As noted by the agency, RCDC produces standard reports across all ICs using a process that is reproducible. To facilitate this, all ICs are to use the same definitions of research, diseases, and conditions for the RCDC categories. Further, the system allows for detailed reports, including the total funding for a category as well as specific information about the projects under each category, such as the title of the project, the IC supporting the research, and the project number.

Agency officials said that the system was not designed to be able to estimate a total, non-duplicated amount of funding specific to a given disease or condition because RCDC categories are neither mutually exclusive nor exhaustive. Specifically:

Projects may be reported in multiple categories. NIH officials said that the categories within the system are not mutually exclusive and therefore a project may be included in multiple categories. Officials told us that, on average, a single project may fall into five or six RCDC categories.

Categories may be contained in another category. Some categories are inherently related and therefore an entire RCDC category can also be contained in another category. For example, the category of cancer includes other cancer categories, such as breast cancer, lung cancer, or prostate cancer, and the research funding reported in each of these categories will also be reported in the cancer research total.

Categories do not exist for all diseases. RCDC also does not capture the funding for all diseases that NIH studies. As of November 2013, RCDC publicly reported on 235 different categories, but agency officials told us that there are an additional 30 to 40 categories used for congressional reporting only, and about 50 categories on a waiting list to be developed for inclusion in the system.

Not all projects that NIH funds are included. The protocols that NIH develops for tracking funding include certain thresholds for inclusion. Therefore, a project that is only minimally related to a particular category may not be included in the funding for that category because the description of the project does not adequately match the terms defining that category. NIH officials told us that 3 to 5 percent of NIH funded research projects do not appear in any RCDC category.

Some ICs have their own systems that track and report funding within their research portfolios. Of the five selected ICs we reviewed, two had their own systems for tracking their funding.

NCI maintains a publicly available website that communicates funding decisions across the research supported by NCI. The system was developed in 1998 and categorizes research into more than 40 specific cancer types, as well as almost 50 research topics that are not disease-specific. Funding for individual projects may be separated and reported in the NCI system by the specific types of cancers being studied. According to NIH officials, the system is limited to those research projects funded by NCI and therefore does not include information about research studies addressing cancer that are funded solely by other ICs. NCI officials told us that they use the system to report data to stakeholders, including Congress, and that the analyses they conduct based on these data enhance NCI’s ability to plan and monitor its scientific investment.

NIAID also has an internal coding system which, according to agency officials, was instituted in 1979 and codes each individual research project it funds by areas of study. Staff members manually assign codes. NIAID officials noted that projects may have multiple codes depending on the specific goals of the project, and these codes reflect the percentage of relevance to that specific goal, such that the total values add to 100 percent. Officials noted that the system helps NIAID respond to requests for funding information that is not included in RCDC. For example, when addressing a question about how much NIAID has invested in influenza research, officials said they examine research project funding for the various subtypes of influenza such as H1N1 and avian influenzas (including H5N1 and H7N9). These subcategories are included in the internal coding system, but are not covered by RCDC. Officials noted that NIAID uses RCDC for the official budget numbers that NIH reports to Congress.

Agency Comments

We provided a draft of this report to HHS. The Department provided technical comments, which we incorporated as appropriate.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of the Department of Health and Human Services, the Director of the National Institutes of Health, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7114 or kohnl@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Major contributors to this report are listed in appendix III.

Appendix I: Studies about the Relationship between Research Funding and Burden of Disease

To provide information on studies about the relationship between research funding and burden of disease at the National Institutes of Health (NIH) and in other countries, we identified such studies relevant to NIH through on-line searches and we provide information on the two most recent of the studies. Additionally, for other countries, we identified published studies through a search of various databases, and we provide information from two of the more recent such studies. We did not identify research funding levels in other countries.

Studies have been conducted that examine the relationship between research funding and burden of disease in the United States and other countries. Some researchers have reported that no single measure of disease burden captures the impact of various diseases and that different measures of burden may result in different conclusions about funding.

Studies on the relationship between research funding by NIH and burden of disease found varied relationships depending upon the burden of disease measures used. For example,

A 2013 study assessed the allocation of research funds across 107 diseases using NIH’s Research, Condition, and Disease Categorization system (RCDC) funding data and found a strong and statistically significant relationship between NIH funding and deaths and hospitalizations. As a result, according to the study, the data suggest that NIH funding is responsive to these two measures of disease burden. Further, the study noted that the data were consistent with the argument that it is more feasible for NIH to respond to disease burden considerations through directed or applied research funding, such as funding for clinical trials, than through investigator- initiated basic research.

A 2011 study to assess the correlation between NIH research funding and burden of disease using data from 2004 and 2006 found that current levels of NIH disease-specific research funding correlated modestly with U.S. disease burden.

The study noted that there could be a number of reasons that alignment of funding and burden of disease were not better correlated, including that basic science research has consistently accounted for 55 percent of NIH spending and it is difficult to attribute to a specific disease, contributing uncertainty to the analysis and reducing correlations. Measures of burden of disease for this study were incidence, prevalence, mortality, years of life lost to premature death (YLL), and disability-adjusted life years (DALY).

Studies of research funding in other countries have also found variations in relationships between funding and burden of disease. For example,

A 2012 study found that research expenditures by governmental agencies and charities for four of the foremost chronic diseases in the United Kingdom—cancer, coronary heart disease, stroke and dementia—was not aligned with burden of disease. Specifically, research funding for dementia and stroke was disproportionately small in comparison with funding for cancer and coronary heart disease. In this study, burden of disease was measured by prevalence and DALYs.

A 2004 study found a significant relationship between research funding by the National Health and Medical Research Council in Australia and several measures of burden of disease, variations over the 6-year span reviewed in the study using different measures. Measures of burden of disease for this study were incidence, mortality, YLLs, years of life lost to disability, and DALYs.

The National Health and Medical Research Council is the main Australian body charged with the responsibility of supporting medical and public health research and training in Australia.

Appendix II: Leading Causes of Death and Chronic Conditions and their Corresponding National Institutes of Health Categories

To determine the leading causes of death in the United States, we reviewed data on causes of death in 2011 that were reported by the Centers for Disease Control (CDC). First, we identified the 15 leading causes of death reported by CDC. Then, we identified subcategories of those leading causes of death. To do so, we used as the cut off the number of deaths for the 15th leading cause of death—which was 18,090 deaths from pneumonitis due to solids and liquids. We included those sub-categories of causes of death, for example pneumonia, where the number of deaths reported was greater than 18,090. See table 2 for the leading causes of deaths in the United States in 2011, including the sub- categories with at least 18,090 related deaths in 2011, and the number of deaths attributed to these causes.

To determine the leading causes of death globally, we reviewed the analysis in the Global Burden of Disease Study. We identified the 15 leading causes of death globally in 2010 that were reported in this study. See table 3 for the leading causes of deaths globally.

To determine the most prevalent chronic diseases and conditions in the United States, CDC provided us with a list of the 13 most prevalent chronic diseases and conditions for adults, and four subcategories. See table 4 for a list of the most prevalent chronic diseases and conditions for adults identified by CDC.

To determine the corresponding categories from the National Institutes of Health’s (NIH) Research, Condition, and Disease Categorization system (RCDC) for the diseases and conditions that are the leading causes of death in the United States, the leading causes of death globally, and the most prevalent chronic diseases and conditions in the United States, we identified the International Classification of Diseases (ICD-10) codes associated with each of these diseases and conditions and, using their related descriptions, compared them with the RCDC category descriptions. We then provided our list of diseases, conditions, and ICD- 10 codes with the corresponding RCDC categories to NIH for the agency’s review and concurrence. NIH officials confirmed most of our matches. For some of the diseases and conditions, officials noted the RCDC category that we identified was the closest match, but the category was substantially broader than the RCDC category it was selected to represent or it was substantially narrower than the RCDC category it was selected to represent. In other cases, NIH officials noted that there was not an RCDC category that was a close enough fit to the disease category we were trying to represent. See table 5 for the leading causes of death in the United States and globally, as well as the most prevalent chronic diseases and conditions, and their corresponding RCDC category or categories.

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact name above, Karen Doran (Assistant Director), George Bogart, Adrienne Daniels, Carolyn Feis Korman, Cathy Hamann, Natalie Herzog, Amy Leone, and Andrea Richardson made key contributions to this report.