Clinical Research: NIH Has Implemented Key Provisions of the Clinical Research Enhancement Act
Released: 2002-09-18T00:00:00

Letter

Background

In the 1990s, a number of influential studies sponsored by NIH, IOM, and AAMC and the American Medical Association (AMA) identified some major problems in clinical research and highlighted NIH’s role in addressing some of these problems. First, there was a general concern that clinical research was receiving substantially less support than basic research at NIH, yet there was little systematic data to document how much, in fact, NIH was spending on clinical research. In an analysis of NIH investigator-initiated extramural grants active in 1991, an IOM committee found that 16 percent involved human research. A few years later, a panel appointed by the NIH director known as the “Nathan Panel,” developed a broad definition of clinical research (the definition NIH now uses) and, applying this definition to all NIH competing extramural research grants in fiscal year 1996, found that 27 percent of grants and 38 percent of dollars were devoted to clinical research. The Nathan Panel believed that this fraction of the extramural budget devoted to clinical research was reasonable and should remain about the same, as efforts to increase the NIH budget as a whole were pursued. The studies sponsored by NIH, IOM, and AAMC/AMA recommended that NIH monitor and track its expenditures on clinical research.

A second concern was that clinical research proposals, especially those from individual investigators, did not fare as well as basic research proposals in peer review at NIH. Grant applications for clinical trials, clinical research centers, and clinical research training are typically reviewed by the sponsoring institute; however, the peer review of individual investigator grant applications usually takes place centrally, within CSR. CSR has approximately 65 study sections that review research. A study section is a panel of experts established according to scientific disciplines or research areas for the purpose of evaluating the scientific and technical merit of grant applications. In 1994 an NIH- commissioned study reported that patient-oriented research applications were less likely to receive favorable reviews in CSR than laboratory- oriented research applications when reviewed in study sections with less than 30 percent patient-oriented research applications. However when patient-oriented research applications were grouped in study sections with greater than 50 percent patient-oriented research, they fared as well as laboratory-oriented research applications. Consequently, this report recommended that study sections reviewing patient-oriented research should have at least 50 percent of such applications and that a means should be developed and implemented to collect and track data prospectively on research applications that are predominantly patient- oriented, laboratory-oriented, mixed, or clinical epidemiology and behavioral research. Similarly, the Nathan Panel recommended that panels that review clinical research must include experienced clinical investigators and that at least 30 to 50 percent of the applications reviewed by these panels must be for clinical research. The IOM committee also recommended more oversight of study section composition, functions, and outcomes pertaining to human research.

A third problem identified in these studies was the adequacy of support for the infrastructure (that is, facilities, equipment, data systems, and research personnel) for the conduct of clinical research. Since the late 1950s, NIH has funded GCRCs across the U.S to provide clinical research infrastructure—facilities, equipment, and personnel—for NIH-funded investigators as well as non-federally funded investigators conducting patient-oriented research. Interdisciplinary and collaborative research is encouraged at these centers. The Nathan Panel, the IOM committee, and others recommended increasing financial support for GCRCs and broadening their leadership role in clinical research and research training.

A fourth concern was the decline in the number of physicians conducting clinical research. According to data collected by the AMA, the number of physicians reporting research as their primary career activity fell by 6 percent from 1980 to 1997 (from 15,377 to 14,434), while the number reporting patient care as their primary career activity almost doubled (376,512 to 620,472). Observers identified a variety of challenges in pursuing a career as a clinical investigator, including the indebtedness of medical students, the length of time a clinical scientist must train, the culture of academic medicine, as well as the competition from other career options. For many years NIH has supported the training of investigators through extramural and intramural predoctoral, postdoctoral training and career development awards. However, there was concern that these awards were being directed toward basic research and were not sufficiently supporting the training and development of clinical investigators. The IOM committee, the Nathan Panel, and the AAMC/AMA reports recommended that NIH provide substantial new support for clinical research training, career development, and debt relief.

Growth in NIH’s Spending on Clinical Research Has Kept Pace with Total Spending

NIH reports that it increased its funding of clinical research and expanded its clinical research activities in response to CREA. NIH estimates that it spent about one-third of its budget, or approximately $6.4 billion, on clinical research in fiscal year 2001. Based on these estimates, the proportion of the NIH budget spent on clinical research has remained fairly constant since fiscal year 1997. NIH’s estimates of clinical research expenditures represent the best available indications of financial trends over time, but they are not precise figures because the process of counting clinical research dollars varies widely across ICs. Finally, in response to CREA, some NIH ICs have developed specific clinical research initiatives.

NIH Estimates That It Spends about One-Third of Its Budget on Clinical Research

In fiscal year 2001, NIH estimated that it spent approximately $6.4 billion on clinical research, which represented about 32 percent of total research spending (see table 1). The institutes that spent the most on clinical research in fiscal year 2001 were the National Cancer Institute (NCI); the National Heart, Lung, and Blood Institute (NHLBI); and the National Institute of Mental Health (NIMH) (see app. I). NIH’s estimated expenditures on clinical research have kept pace with the overall growth in NIH’s budget. As NIH’s reported clinical research expenditures increased by 44 percent (adjusted for inflation) from fiscal year 1997 to fiscal year 2001, the proportion of research dollars spent on clinical research remained constant, at 32 percent, each year.

NIH estimates that in fiscal year 2001, it spent approximately $5.9 billion on extramural clinical research, about 35 percent of its total extramural research expenditures. NIH’s extramural clinical research dollars were spent through a variety of funding mechanisms in fiscal year 2001. About 40 percent of the awarded dollars were grants to individual investigators, followed by other funding mechanisms, center grants, cooperative agreements, research program projects, and research and development contracts (see fig. 1). Of NIH’s total extramural research expenditures for cooperative agreements and center grants, the majority of dollars were spent on clinical research in fiscal year 2001.

In fiscal year 2001, NIH estimated that it spent about $529 million, or 27 percent of its intramural research expenditures, on clinical research. NIH’s intramural clinical research activities include research at the Clinical Center on NIH’s Bethesda, Maryland, campus, as well as research by individual institutes. The Clinical Center’s budget represents more than half of the intramural clinical research expenditures. The budget of the Clinical Center increased from approximately $204 million in fiscal year 1997 to an estimated $303 million in fiscal year 2002. This budget increase supported an increase in admissions, inpatient days, and outpatient visits.

Flaws Exist in NIH’s Process of Counting Clinical Research Expenditures

NIH’s reports of clinical research expenditures represent the best available indications of financial trends, but are not precise figures. The methods NIH uses to count clinical research dollars are inconsistent across ICs, potentially underestimating or overestimating its actual clinical research expenditures. Since fiscal year 1997, the Office of Budget, within the Office of the Director, has collected information from each IC on its extramural and intramural clinical research expenditures. The ICs use the NIH definition of clinical research (described earlier), but they count the dollars in very different ways. The 20 ICs that fund clinical research reported three different ways of counting clinical research dollars. First, 12 ICs count 100 percent of the grant dollars of research projects that include any clinical research. Second, one institute, NCI, codes a research project’s “percent relevance” to clinical research. Projects are coded as 100 percent, major, minor, or 0 percent clinical research. If they are classified as “major,” they are assigned a percentage relevancy of 50 percent, and 50 percent of the dollars are counted. If they are classified as “minor,” they are assigned a percentage relevancy of 5 percent, and 5 percent of the dollars are counted. Third, 7 ICs either attempt to estimate the dollars of a research project spent on clinical research or the percentage of a project that is clinical research and apply that percentage to the total grant dollars.

These different methods of counting clinical research dollars can produce very different results. For example, given a hypothetical grant to an investigator of $300,000 for which an IC has estimated that $50,000 of the budget would be spent on clinical research, some ICs would report that $300,000 was spent on clinical research; NCI could conclude that this grant has only minor relevance to clinical research and therefore would count 5 percent, or $15,000, as clinical research dollars; the rest of the ICs would estimate that this project is about 17 percent clinical research and therefore count $50,000 of the grant as clinical research dollars.

The Office of Budget said that the reason the ICs count clinical research dollars differently is that each developed its own methods over time, and for historical consistency, they are reluctant to change. One IC director, who heads an NIH Director’s committee concerned with clinical research spending told us that NIH is working on ways to make its process of tracking and reporting clinical research dollars more consistent and accurate.

NIH Institutes Have Developed New Clinical Research Initiatives

In response to CREA, some institutes have developed new clinical research initiatives. For example, since the passage of CREA, NCI has funded two new clinical cancer centers and funded 22 new Specialized Programs of Research Excellence for different types of cancer, all of which involved early phase clinical trials. NHLBI is establishing new clinical research centers to study ways to reduce racial and economic disparities in asthma prevalence, treatment, and mortality and is funding trials to assess innovative strategies to improve the implementation of clinical practice guidelines for heart, lung, and blood diseases. The National Institute of Arthritis and Musculoskeletal Diseases has a new osteoarthritis initiative; funds multidisciplinary clinical research centers in arthritis, musculoskeletal, and skin diseases; and plans to enhance its translational research projects in children’s diseases. The National Institute of Allergy and Infectious Diseases (NIAID) has continued to fund large clinical trial networks such as the AIDS Clinical Trials Group, a $120 million per year initiative that involves research on pediatric and adult AIDS.

NIH Has Taken Steps to Improve Its Peer Review of Clinical Research

Since passage of CREA, NIH has acted to strengthen its peer review of clinical research applications. CSR established two new study sections in the areas of clinical oncology and clinical cardiovascular sciences. In study sections with a mix of clinical and basic proposals, CSR tries to group clinical research applications and reviewers, but officials could not provide data to determine how successful it has been in achieving this goal. NIH has established peer review mechanisms at the institutes for the review of career development and training awards established under CREA.

CSR Has Established Two New Clinically Focused Study Sections for Peer Review

In response to concerns that clinical research proposals are not fairly reviewed in its study sections, CSR has established two new clinically oriented study sections, Clinical Oncology and Clinical Cardiovascular Sciences. In these scientific areas, CSR found that there were a sufficient number of clinical research applications to justify separate study sections. Although the two new clinical research study sections have been welcomed by the research community, some concerns remain among clinical investigators about the fairness of the review of clinical research by other study sections that have a mix of clinical and basic research. In these study sections, CSR officials told us they try to group clinical research applications and clinical research reviewers. CSR officials told us that it is their general goal to review clinical research applications in study sections in which at least 30 percent of the applications involve clinical research and in which at least 30 percent of the reviewers are themselves clinical investigators. CSR officials also explained that this goal cannot always be achieved because if the number of clinical research applications in a specific scientific area is small, it may not be possible to group the applications to 30 percent and still review them in a study section that provides the appropriate scientific context for review. They emphasized that reviewing applications in the appropriate scientific context is given priority over quantitative targets for grouping. CSR officials could not provide data on the extent to which they have been able to group clinical research applications and have very limited data on which reviewers are clinical investigators. The officials told us that, to date, they do not have reliable and accurate methods for identifying and tracking clinical applications or clinical reviewers.

CSR officials told us they are in the process of a broader review and restructuring of their peer review system, with input from the scientific community, to account for new developments in science. According to CSR, one of the goals of this reorganization is grouping applications and reviewers at 30 percent so that there is a “density of expertise” in review sections. In addition, CSR has recently appointed a special advisor on clinical research review to serve as a liaison with the clinical research communities.

Clinical Research Career Development and Training Applications Are Reviewed by ICs

To determine NIH’s response to CREA’s requirement that NIH establish appropriate mechanisms for the peer review of clinical research career development and training applications, we surveyed nine ICs that sponsored the highest number of clinical research career development awards in fiscal year 2001. We found that three ICs used a Special Emphasis Panel, while the six others used established committees or subcommittees to review clinical research career development and training applications. In addition, the ICs reported that most of the reviewers of these applications have clinical research experience, and some are involved in clinical research training. One institute brings in temporary reviewers to augment its committee if special expertise is needed. NCRR uses CSR for peer review of some career development applications that require very specific scientific expertise and therefore require review by the discipline-specific study sections of CSR.

NIH Has Increased Its Support and Scope of GCRCs

NIH has increased its support of GCRCs and GCRCs’ scope of work, as required by CREA. The GCRC budget has grown over time, although more slowly than NIH’s estimates of clinical research spending. Adjusted for inflation, the funding for GCRCs increased by 24 percent from fiscal year 1997 to fiscal year 2001, compared to a 44 percent estimated increase in clinical research spending at NIH during that same period. Although NIH has stopped funding some GCRCs, there has been a gradual increase in the number of GCRCs over time, from 74 in fiscal year 1997 to 79 in fiscal year 2001. There has also been an increase in the activities of GCRCs and some expansion in their scope since passage of CREA.

Funding, Number, and Activities of GCRCs Have Increased

NIH has increased funding for the GCRC program, although funding for the GCRCs has grown more slowly than NIH’s estimate of overall expenditures on clinical research. From fiscal year 1997 through fiscal year 2001, funding for the GCRCs increased from $153,521,000 to $220,824,000 (see table 2). Adjusted for inflation, this represents an increase of 24 percent, compared to the 44 percent estimated growth in total clinical research expenditures during this period. The number of GCRCs gradually increased during this period, from 74 to 79. Funding levels for individual GCRCs in fiscal year 2001 ranged from $712,339 to $6.2 million, with an average funding level of about $2.8 million. NIH officials told us that in fiscal year 2002, they are opening two new GCRCs, one at the University of Maryland and one at the University of Miami. Establishing a new GCRC costs about $2.5 million and requires a certain threshold of investigators. Once a GCRC is set up, attracting additional investigators and research activities is easier, according to NIH officials.

Also shown in table 2, some activities of GCRCs have increased in recent years. For example, the number of research protocols and investigators supported by GCRCs increased from fiscal year 1997 through fiscal year 2001. While the number of inpatient days funded by GCRCs declined from 70,814 in fiscal year 1997 to 62,769 in fiscal year 2001, the number of outpatient visits increased from 282,125 to 334,828 during the same period.

Scope of GCRC Activities, Including Telecommunications, Has Expanded

Since passage of CREA, NIH officials told us there has not been a change in the mission of GCRCs, but there has been an increase in the scope of GCRC activities. For example, in fiscal year 2002, 27 GCRCs have funded Clinical Research Feasibility pilot projects to support the research of beginning investigators. In addition, 76 GCRCs now each have a Research Subject Advocate who helps ensure that GCRC research is conducted safely and protects human research subjects.

CREA required that NIH expand the activities of the GCRCs through increased use of telecommunications and telemedicine initiatives. In response, NIH officials told us they increased their support of specialized bioinformatics networks that electronically link research data across GCRCs. Specifically, NCRR established a Biomedical Informatics Research Network, a computerized network that allows investigators affiliated with GCRCs to share high-resolution images of human brains and large volumes of complex data and conduct remote analysis of the data. In fiscal year 2001, NCRR funded five bioinformatics centers at $2.1 million, and a coordinating center at $1.6 million, spending a total of $3.7 million on this initiative. In fiscal year 2002, $6 million has been set aside to extend this network. NCRR also funded a collaborative pilot project between the Cystic Fibrosis Foundation and several GCRCs, called CFnet, to assess whether clinical trials could be facilitated across GCRC sites with Web- based data handling. Based on the success of this pilot, NCRR plans to extend CFnet to 20 GCRCs and also establish a comparable network among the eight U.S. medical schools that have a high proportion of minority students to facilitate the schools’ participation in clinical trials that relate to health disparities.

NIH Has Increased Its Support of Clinical Research Career Development and Training

NIH has established the four new career development award programs required by CREA. Three of these have been implemented, and the fourth is just beginning. NIH has also established intramural and extramural clinical research training programs for medical and dental students and clinical research continuing education programs as required by CREA.

NIH Has Implemented Three Award Programs for Career Development of Clinical Investigators

NIH recently established three new clinical research career development award programs for individuals and institutions outside government that are designed to increase the supply and expertise of clinical investigators (see table 3). NIH used its K award mechanism, its usual method for providing support for career development of investigators, to establish these programs. In fiscal year 1999, NIH implemented the Mentored Patient-Oriented Research Career Development Award (K23) to support investigators who are committed to conducting patient-oriented research for 3 to 5 years. In the same year, NIH implemented the Mid-Career Investigator Award in Patient-Oriented Research (K24) to provide support for more senior clinicians to relieve them of patient-care duties and administrative responsibilities so that they can conduct patient-oriented research and serve as mentors for beginning clinical investigators. The Clinical Research Curriculum Award (K30), also implemented in fiscal year 1999, supports the development and expansion of clinical research teaching programs at institutions. About half of the K30 programs offer graduate degrees in clinical research (for example, masters or doctorate).

The response to these new award programs was substantial, and NIH funded more awards than originally planned. NCRR and the largest institutes (for example, NCI, NHLBI, and NIMH) sponsored the highest number of the new K23 and K24 awards. NHLBI is administering the majority of the K30 awards. Although NIH has received applications for K23 and K24 awards from a variety of clinical investigators, most applicants and awardees are physicians. The K30 awards have primarily gone to academic medical centers. The new awards combined represent 25 percent of expenditures NIH allotted for all K awards under its Career Development Program in fiscal year 2001 (see fig. 2).

NIH officials told us that they are initiating plans to evaluate the new clinical research career development awards and track career outcomes. The design of this assessment will be based on previous studies of training award recipients, specifically NIH’s study of the outcomes of the National Research Service Awards (NRSA) and will rely on NIH’s new electronic grant application.

Size and Scope of New Award Program for Graduate Training in Clinical Research Is Limited

In 2001 NIH announced a fourth new clinical research career development award, the Mentored Clinical Research Scholar Program (K12). This award program, sponsored by NCRR and linked to the GCRCs, is NIH’s response to CREA’s directive to support graduate training in clinical research. NCRR decided to start the K12 program as a small pilot project and then expand it later if successful. In fiscal year 2002, NCRR received 43 applications for this award and expects to fund 10 of these. In the first year of the program, each funded award may enroll three clinical research scholars, for a total of 30 scholars. NIH projects that the number of scholars could grow to 120 in 5 years.

We interviewed several K30 program directors who indicated that obtaining graduate tuition and stipend support for their students and prospective students was a major constraint. The K30 award, which has been well received in the research community, funds curriculum, staff, as well as tuition and other costs in special circumstances, but generally does not directly support students. Instead, students must seek funding from other NIH, federal, or private sources. An NIH official estimated that the number of formal trainees in individual K30 programs ranges from several to three dozen. This official was not able to provide data on whether these students had tuition support and what kind of support. However, the K30 program directors we talked to said some of their students had tuition support from other NIH funding mechanisms; others had support from their university. Although the new K12 program is consistent with the requirements of CREA, some K30 program directors and other experts believe the size and scope of the program will be too small to meet the need for graduate training support for clinical investigators.

Other NIH Clinical Research Training Programs Targeted toward Medical and Dental Students

In terms of fellowships for clinical research training, in fiscal year 2001, NCRR announced a new mentored medical student clinical research program that will support a small number of medical and dental students at GCRCs. This program provides supplemental grants to GCRCs to offer 1 year of support for medical and dental students, usually from their third through fourth year of school, in the form of salary, supplies, and tuition assistance. A total of five students may eventually be supported at each GCRC site annually, although NCRR plans to provide support for only one medical student per GCRC in fiscal year 2002.

Since 1997, NIH has also trained medical and dental students at its campus in the area of clinical research. In this program, partially supported by a pharmaceutical company, 15 to 20 students are selected each year and are each paired with a mentor for a year of academic study and clinical research experience.

NIH Has Established an Extramural Loan Repayment Program for Clinical Investigators

NIH has launched an extramural loan repayment program for clinical investigators as required by CREA, and most of NIH’s ICs participate in the program. In the first year of implementation, eligibility for the loan repayment program was tied to receipt of NIH funding. However, in fiscal year 2003, NIH plans to extend eligibility to allow clinical investigators who receive funding from other sources, such as other federal agencies and nonprofit foundations, to apply.

New Extramural Loan Repayment Program Established by NIH

In response to CREA, NIH established an extramural Clinical Research Loan Repayment Program. This new loan repayment program joins four other extramural loan repayment programs and four intramural loan repayment programs that are administered by NIH’s Office of Loan Repayment and Scholarship. The new extramural Clinical Research Loan Repayment Program was implemented on December 28, 2001, and a total of 456 applications were received by February 28, 2002. NIH plans to fund 396 loan repayment contracts for a total of $20.2 million by the end of fiscal year 2002. The program provides for the repayment of up to $35,000 per year of the principal and interest of an individual’s educational loans for each year of obligated service. These individuals are obligated to engage in clinical research for at least 2 years. The clinical research loan repayment program represents a sizeable proportion (almost two-thirds) of the total extramural loan repayment program budget.

To be eligible for the clinical research loan repayment program, a clinical investigator must have received an NIH research service award, training grant, career development award, or other NIH grant as a first-time principal investigator or a first-time director of a subproject on a grant or cooperative agreement. In fiscal year 2003, the Director of the Office of Loan Repayment and Scholarship told us that NIH plans to remove the NIH-funding restriction and allow clinical investigators who receive funding from other sources, such as other federal agencies and nonprofit foundations, to apply for the loan repayment program. In addition, NIH expects to almost double the size of the extramural Clinical Research Loan Repayment Program in fiscal year 2003.

Twenty-one of NIH’s ICs Participate in the Loan Repayment Program

Although NIH has a central office that administers all the loan repayment programs, funding for the clinical research loan repayment program was distributed to the ICs, based on reported clinical research expenditures in fiscal year 1999. Thus 21 of NIH’s 27 ICs plan to participate in the program by reviewing applications and awarding loan repayment contracts (see app. II). The ICs sponsoring the highest number of contracts are NCI, NHLBI, and NIMH. NCRR also plans to sponsor a significant number of loan repayment contracts. As with most of the training and career development awards, an NIH official told us that the ICs were in the best position to assess applications and the clinical research career potential of awardees.

Conclusion

In general, NIH has complied with the key provisions in CREA. It has increased its financial support of clinical research, expanded its clinical research activities, made improvements in its review of clinical research proposals, expanded its support of GCRCs, established new clinical research career development and training programs, and begun to implement a new extramural clinical research loan repayment program. Some of NIH’s actions were taken prior to CREA’s passage and some are still being implemented. However, we identified some inconsistencies with the way that NIH counts clinical research expenditures. These inconsistencies limit the precision of NIH’s reports of clinical research expenditures and its ability to monitor the support of clinical research.

Recommendation for Executive Action

To strengthen the tracking and reporting of intramural and extramural expenditures for clinical research, we recommend that the Director of NIH develop and implement a consistent, accurate, and practical way for all ICs to count intramural and extramural clinical research expenditures.

Agency Comments

NIH reviewed a draft of this report and provided comments, which are included as appendix III. NIH concurred with our recommendation and reported that it is taking steps to implement a better, more unified system for tracking and reporting clinical research expenditures across the ICs. According to NIH, this new system will be implemented in fiscal year 2003. NIH also provided technical comments, which we incorporated as appropriate. In particular, NIH clarified its response to our questions about the peer review of clinical research. NIH emphasized that it recognizes the importance of collecting data on the grouping of clinical research applications and reviewers. Toward that end, NIH stated that one of the responsibilities of CSR’s newly appointed Special Advisor on Clinical Research Review will be to investigate new methods to reliably identify and track clinical research applications and clinical research reviewers.

We will send copies to the Secretary of Health and Human Services, the Director of NIH, appropriate congressional committees, and others who are interested. We will also make copies available to others on request. In addition, the report will be available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staffs have any questions, please contact me at (202) 512-7119 or Martin T. Gahart at (202) 512-3596. Key contributors to this assignment were Anne Dievler, Cedric Burton, and Elizabeth Morrison.

Appendix I: NIH’s Estimated Expenditures for Extramural and Intramural Clinical Research, by IC, Fiscal Years 1997 - 2001

Dollars in millions

Appendix II: NIH’s Extramural Clinical Research Loan Repayment Contracts by IC, Fiscal Year 2002

Institutes and centers National Cancer Institute National Heart, Lung, and Blood Institute National Institute of Mental Health National Institute of Allergy and Infectious Diseases National Institute on Aging National Institute on Drug Abuse National Center for Research Resources National Institute of Diabetes and Digestive and Kidney Diseases National Institute of Child Health and Human Development National Institute on Alcohol Abuse and Alcoholism National Institute of Neurological Disorders and Stroke National Institute of Arthritis and Musculoskeletal and Skin Diseases National Eye Institute National Institute on Deafness and Other Communication Disorders National Institute of Dental and Craniofacial Research National Human Genome Research Institute National Institute of Environmental Health Sciences National Institute of General Medical Sciences National Institute of Nursing Research National Center for Complementary and Alternative Medicine John E. Fogarty International Center Total

Appendix III: Comments from the National Institutes of Health