NIH Extramural Clinical Research: Internal Controls Are Key to Safeguarding Phase III Trials Against Misconduct
Released: 1996-07-11T00:00:00

Letter

Background

NIH is a Public Health Service (PHS) agency within HHS. It consists of a director’s office and 14 staff offices that oversee the operations of 24 separate units. These units include 17 institutes, each focused on specific health or medical issues, such as cancer or aging; six research centers; and the National Library of Medicine. Each unit separately awards funds for the research it sponsors. NIH’s Office of Extramural Research is responsible for agencywide activities concerning oversight of Phase III clinical trials, such as developing policy on the review, funding, and management of extramural grants.

NIH’s extramural research units (generally referred to in this report as “institutes”) used various methods to fund the 470 Phase III clinical trials they sponsored in fiscal year 1994. As figure 1 shows, the largest number of trials (180) were funded through cooperative agreements. Regardless of the method used to fund the trials, the institutions that are awarded the funds are referred to as “grantee institutions” or “grantees.”

Most Phase III clinical trials involving multiple sites are funded through contracts and cooperative agreements. Trials funded through contracts are typically planned, initiated, and controlled by the sponsoring NIH institute. The institute details the trial’s objectives, protocols, and controls.

Under cooperative agreements, however, the grantees and principal investigators have more flexibility in planning, managing, and conducting the trial. Although the sponsoring institute is expected to make substantial contributions to the trial, such as providing technical assistance, coordinating the trial’s activities, and helping to manage the trial, operational control of the trial rests with the grantee.

NIH Institutes and Grantees Provide Oversight of Clinical Trials

The institutes and research centers at NIH along with the grantee institutions directly oversee and monitor Phase III clinical trials. These entities are to ensure that controls are in place to prevent or detect the misuse of federal funds and the falsification of data in conducting extramural clinical research. According to NIH, these institutes and grantee institutions know the nature and objectives of the trials and are therefore in the best position to develop monitoring procedures to ensure safety and data integrity. At the sites we visited, controls that safeguard against fiscal misconduct are consistently applied among institutes and trials. Some controls that safeguard against scientific misconduct, however, are not always consistently used for various reasons, including the type of trial and the sponsoring institute’s management philosophy.

Fiscal Oversight Policies and Controls Are Consistent Among Institutes and Grantees

Although each institute independently oversees the clinical trials it sponsors, the controls established to prevent and detect fiscal misconduct were consistent among the institutes in our review. The control procedures must conform with federal requirements and policies on the expenditure of federal funds. Grantee institutions are responsible for ensuring that their research scientists and other employees comply with all applicable federal rules, regulations, and policies on the use of federal funds. Independent auditors review grantee compliance annually in a required financial audit.

Most grantee institutions receive federal funds from several federal agencies. The grantees must adhere to a uniform series of regulations laid out by the Office of Management and Budget (OMB). Chief among these policies are cost principles that grantees must adhere to as specified in OMB Circulars A-21, A-87, or A-122. These principles provide guidance on what expenses a grantee may incur and charge against an NIH grant award. Grantees must also follow a uniform set of administrative requirements in OMB Circulars A-102 or A-110, detailing how grant funds should be managed. Foremost among these requirements are standards for such areas as fiscal reporting, accounting records, internal controls, and cash management. Other administrative requirements cover procurement and property standards.

Independent auditors annually audit grantees’ fiscal management of federal funds as required by OMB Circulars A-128 and A-133. It was such an audit that detected the embezzlement of more than $700,000 of NIH grant funds in the early 1990s. This case of fiscal misconduct by a manager of grants accounting occurred at the New York Medical College—the grantee. Because the grantee institution is responsible for ensuring that federal grant funds are properly used, the college was required to fully refund these funds to NIH.

At the five grantee institutions we visited, we reviewed the annual financial audits. The audits included a review of internal controls established by the grantees to safeguard federal funds. In each case, the audits disclosed that grantees had complied with federal guidelines and no material weaknesses were detected in the internal controls.

When grantee institutions fail to establish and maintain adequate internal controls and proper accounting procedures to safeguard federal funds, NIH can impose requirements that the grantee must comply with to continue receiving and managing grant awards. In 1995, NIH designated the University of Minnesota, a grantee, an “exceptional organization” because of its failed internal controls and poor accounting procedures. This designation enabled NIH more oversight of its funds than would be feasible under the administrative procedures normally associated with its grant programs. NIH increased the conditions and restrictions attached to the University’s grant award. It also required the University to develop and successfully implement a corrective action program to address the deficiencies before NIH would consider removing the exceptional organization designation.

Grants Management Officers Monitor Fiscal Integrity

Each institute assigns grants management officers to clinical trials to oversee the use of federal funds awarded to grantee institutions. One method used by the grants management officers to reduce the agency’s risk is to limit the amount of funds readily available to the grantees. For instance, because a cooperative agreement usually covers more than 1 year, the initial award specifies how much funding will be provided each year for the life of the agreement. However, funding is provided on a year-to-year basis only. The grantee must apply each year for a continuation award for additional funding even though the total grant amount is committed. Institutes release funds on the basis of satisfactory performance as detailed in the annual progress reports that principal investigators must submit. If a grantee’s progress is not satisfactory, a grants management officer may reserve all or some of the funding until grantee progress improves.

Institutes awarding funds for clinical trial research issue award notices that include a section indicating whether any of the funding is restricted and what must be done to lift the restriction. If a grantee’s funds are restricted, the grants management officer might release the funds but restrict their use until the grantee has completed certain tasks. For example, in one trial we reviewed, the officer restricted administrative funds until the sites developed a contractual agreement indicating how they would work together.

In addition to annual progress reports, the grantee must include a summary of annual expenditures in its Financial Status Report to the grants management officer. This allows the officer to compare the reported overall expenditure totals with the original budget and progress reports. If the officer finds any significant differences, the grantee is expected to explain them. For Phase III clinical trials funded through contracts or cooperative agreements, according to NIH, when grantees do not spend funds as budgeted, grants management officers must approve all requests to rebudget funds as well as requests to carry over funds from one year to the next.

Program Officers and Oversight Boards Monitor Scientific Conduct

Institutes’ oversight monitoring of clinical trials has some consistent safeguards against scientific misconduct and protections for the safety of trial participants. Each institute usually requires specific monitoring methods. For example, an NIH program officer is assigned by the sponsoring institute to monitor each Phase III trial. Program officers are research scientists with expertise in the area being studied. Each institute trains and develops its own program officers in monitoring and managing clinical trials. Therefore, program officers’ training can vary by institute.Also, their responsibilities often vary by the institute’s management philosophy, the type of trial, and the funding method—contracts or cooperative agreements. Typically, program officers, at a minimum, rely on basic oversight controls in monitoring clinical trials, including annual progress and budget reports and trial participants’ recruitment and retention statistics.

Oversight boards also monitor trials. For example, most clinical trials that pose a potential hazard to human trial participants must be monitored by a Data and Safety Monitoring Board or an equivalent. This board, composed of scientists not connected with the trial, monitors a trial’s clinical data and progress. The board also focuses on reported adverse events—adverse changes in the health status of a human research subject in a clinical trial. In addition to a Data and Safety Monitoring Board, each grantee institution must establish an Institutional Review Board to approve and monitor all research involving human subjects. An important function of this board is to review and approve informed consent forms, making sure they have been signed. All prospective research subjects must sign consent forms that explain the objectives, risks, and benefits of the proposed research before they can participate in a trial.

In our review of clinical records at the five sites we visited, we did not find any cases in which a consent form had not been signed by a trial participant. However, according to a report on the NCI-sponsored breast cancer trial, only about 71 percent of trial participants gave written informed consent before surgery; consent forms were missing or not available or data were insufficient for 7 percent of the participants.

Controls That Protect Trial Data Integrity Are Not Always Consistently Applied

Clinical trials have controls that safeguard against scientific misconduct, including direct data verification to ensure data integrity. Because institutes and grantees, however, have flexibility in deciding how these controls are used, the application of the controls often varies by institute and type of trial.

One control designed to safeguard trials against scientific misconduct is the use of clinical monitors to review trial data. These monitors visit clinical trial sites to verify that the established protocols are being followed and that the data being reported match the data in the clinical records. In one trial sponsored by the National Eye Institute, clinical monitors found that clinical test results were being entered on data collection sheets and not in the patients’ medical records. Clinical research policy states that medical records are the acceptable source documents for clinical test results so monitors required that the site also enter reported data in the patients’ medical records.

Because clinical monitors add both expense and time to a trial, institutes tend to use them only in the large and more complex trials. For example, clinical monitors are being used in NIA’s largest and most expensive ongoing Phase III clinical trial—alternative therapies for Alzheimer’s disease. This trial is being conducted at 35 research sites and costs $16.9 million in NIH funds. The NIA program officer for the Alzheimer’s trial estimated that using clinical monitors in this trial delayed data entry by 6 months. This delay is acceptable, however, because of the increased quality assurance that clinical monitors bring to the trial, according to the program officer.

Another internal control procedure to protect data integrity is the use of data coordinating centers. Most Phase III clinical trials that have multiple research sites use data coordinating centers to process patient clinical data generated during a trial. These centers inspect the data for inconsistencies among the sites, irregularities, and fraud.

In one NIA-sponsored trial, Continence Program for Women, the data coordinating center detected data inconsistencies between two clinical sites and alerted both the institute and the Data and Safety Monitoring Board. The inconsistency was caused by a different classification of patients by the two sites. However, the center’s detection of the data problem allowed the problem to be corrected. In an NHLBI-sponsored trial, the data coordinating center questioned test results from one laboratory. Further investigation by NIH’s Office of Research Integrity revealed that a lab technician had not conducted the tests as required and had reported false test results. NHLBI took corrective action to minimize the damage to the trial. The institute also recovered funds paid to the laboratory, and the technician was sanctioned.

Benefits of Data Coordinating Centers’ Independence

NHLBI officials believe that the independence of data coordinating centers is an essential part of internal controls. It is a way for the institute to create a direct link to a key data verification point and to help ensure prompt notification of potential scientific misconduct or other data irregularities. For this reason, NHLBI directly funds data coordinating centers and requires that the heads of the centers report directly to the institute’s program officer and the Data and Safety Monitoring Board. This approach places data coordinating centers beyond the direct control of a trial’s principal investigator.

Other institutes that have not provided for data coordinating centers’ independence in trials have experienced problems with researchers’ influence over the centers. For example, for the three NIA trials included in our review, data coordinating centers were funded through a subcontract with research centers. This arrangement allowed a lead researcher, in a dispute with the center, to withhold the center’s operating funds. The institute’s program officer had to intervene to resolve the situation. During our review, NIA’s policy was to independently fund data coordinating centers for most of its multisite clinical trials.

In the breast cancer trial, NCI permitted the trial’s principal investigator to oversee the operations of the data coordinating center. When the center detected suspect data, the principal investigator was notified. The investigator took about 3 months to establish that fraud had occurred and another 5 months before notifying NCI. The investigator’s failure to promptly notify NCI as required delayed corrective action and jeopardized the integrity of the trial. NCI had to spend time and resources to revalidate the trial’s initial results.

Centralized NIH Oversight of and Guidance on Managing Clinical Trials Is Limited

NIH conducts limited centralized monitoring of Phase III clinical trials. No agencywide registry or database exists to track progress and performance of all clinical trials and provide NIH’s management with comprehensive reports for oversight and decisionmaking purposes. Although periodic meetings occur to discuss progress of ongoing trials, no data are systematically collected nor used to provide centralized oversight. Furthermore, NIH has not adopted its internal committee’s recommendations to develop agencywide guidance on quality assurance measures and data monitoring procedures for institutes to use in managing clinical trials. According to NIH, some of its institutes have selectively adopted some of the committee’s recommendations, but the agency believes adopting these policies agencywide is inappropriate because this erroneously assumes that all clinical trials should be monitored in the same manner. Nonetheless, NIH and HHS have implemented some agencywide measures in the past designed to discourage misconduct in federally funded research.

NIH Conducts Limited Central Oversight and Monitoring of Phase III Clinical Trials

Even though NIH’s Office of Extramural Research is responsible for centralized activities concerning oversight of extramural research, such as developing policy on the review, funding, and management of clinical trials, it has limited knowledge of and data on the Phase III clinical trials NIH funds and the performance of individual institutes and grantees. The office does review institutes’ initial requests for Phase III clinical trial research. Once a request is reviewed and ultimately approved, however, the awarding of the grants and most of a trial’s oversight and management are left to individual institutes.

The Office of Extramural Research might learn of a trial’s progress from meetings of the Extramural Program Management Committee, whose members are staff from each institute. The committee meets regularly to discuss, among other issues, those related to Phase III clinical trials and to exchange ideas. However, unless an institute’s representative mentions a problem with a trial or raises concern about fiscal or scientific misconduct, the committee or the Office of Extramural Research would not likely know about it.

NIH has not yet implemented a centralized database or a central trial registry to improve its oversight of the clinical trials it funds. An automated database of all clinical trials could track progress and performance and generate reports that would increase management’s knowledge about the trials and improve its ability to oversee them. Because no active central trial registry exists, NIH would have to survey each institute just to determine the total number of Phase III clinical trials it funds. The NIH Revitalization Act of 1993 required NIH to develop a registry of clinical trials involving women’s health issues. NIH, however, decided not to limit this registry to trials involving women’s health but to include other trials. NIH’s Office of Extramural Research is developing the Streamlined Non-Competing Award Process (SNAP) database as a pilot experiment. According to NIH, this database will allow it to interact with the grantee institutions and monitor research progress. NIH expects that when SNAP is expanded to include all clinical trials, NIH staff will be able to monitor trial progress in areas such as recruitment.

NIH Has Not Adopted a Recommendation to Develop Agencywide Guidance on Managing Clinical Trials

Because the NIH institutes and grantees have more flexibility in deciding how to manage clinical trials funded through cooperative agreements, the scientific controls used in such trials vary. Aware of this variability, NIH’s Office of Extramural Research established the NIH Working Committee on Clinical Trial Monitoring in June 1994 to determine how its institutes manage clinical trials. The committee members were representatives from the institutes and research centers and were selected for their expertise in various aspects of clinical research.

The committee’s task was to specifically review how the institutes manage the Phase III clinical trials they sponsor. On the basis of its review, the committee decided in 1995 that attempting to develop standards to dictate how these trials are managed is inadvisable given the unique characteristics of each Phase III clinical trial and the diverse nature of the institutes. The committee did recommend, however, that NIH consider formulating guiding principles for all the institutes to follow in managing the trials.

The principles the committee recommended covered such areas as quality assurance and site monitoring, patient confidentiality, level of NIH staff involvement, and data access. Specifically, for example, in the area of quality assurance and site monitoring, the committee recommended that the institutes, at a minimum, conduct regular on-site monitoring of all clinical centers and monitor key outcome data. It also recommended that trials involving multiple clinical centers, large study populations, or potentially harmful interventions have the substantial involvement of and close oversight by the sponsoring institute. NIH has decided not to adopt any of the committee’s recommendations agencywide. According to NIH, adopting agencywide policies such as those the committee recommended is inappropriate because it assumes that all clinical trials should be monitored in the same way.

At one of the data coordinating centers we visited, officials expressed frustration because standards and procedures for data collection differ by institute as well as among program officers at the same institute. They believe that minimal data collection standards and procedures should be established for all trials.

HHS and NIH Have Implemented Some Agencywide Policies on Scientific Misconduct

Prompted by legislation and on their own initiative, HHS and NIH have taken steps to discourage misconduct in federally sponsored research, including clinical trials. These efforts have focused mainly on establishing proper scientific conduct and conflict-of-interest reporting requirements for grantee institutions.

In response to the Health Research Extension Act of 1985, HHS required each grantee institution to develop a formal process delineating the steps to be taken to resolve allegations of scientific misconduct. In addition, institutions are required to diligently try to protect the positions and reputations of whistleblowers. ORI monitors compliance with this requirement.

As required by the NIH Revitalization Act of 1993, HHS recently took action to ensure that the design, conduct, or reporting of PHS-funded research is not affected by researchers’ outside financial interests. This applies also to all NIH-sponsored research. Specifically, HHS issued a regulation effective October 1, 1995, requiring that each grantee institution develop a conflict-of-interest policy applicable to all staff benefitting from PHS funding. To comply with this regulation, researchers must file annual financial disclosure forms that allow the institution to determine if a conflict of interest exists. All applications for PHS funding must contain a certification by the institution that no conflict of interest exists. Each of the five grantee institutions we visited had developed and implemented conflict-of-interest policies. Because of the newness of the policies, however, officials said it would take time to see how these policies operated in practice and how effective the policies would be.

Conclusions

A large percentage of NIH-sponsored Phase III clinical trials are funded through cooperative agreements so both the institutes and grantees are involved in managing the trials and developing procedures for conducting them, according to NIH. The trials have controls designed to safeguard against fiscal and scientific misconduct that the institutes, grantee institutions, and research sites can apply in overseeing the trials. However, no practical level of oversight and controls can completely eliminate the potential for misconduct.

Most oversight of these trials is decentralized and performed independently by each of the different institutes that sponsor clinical research and by the grantee institutions. Because of the large number of diverse Phase III clinical trials NIH funds and the independent nature of its institutes, NIH charged a working committee with determining how such trials are managed. The committee recommended that NIH develop some agencywide guidance for all institutes to follow in managing these trials. The guidance was recommended for areas such as quality assurance, site monitoring, and the level of NIH staff involvement. Although some institutes have implemented some of the principles, NIH believes adopting them agencywide is inappropriate.

In the past, NIH has done little centralized oversight and monitoring of the trials it funds and the institutes that sponsor them, except for tracking women’s and minorities’ participation in clinical trials. NIH is, however, developing a database that it expects will allow for monitoring elements of clinical trials’ progress and performance.

Agency Comments

In commenting on a draft of this report, NIH agreed in general with our conclusions and noted that the report provides a balanced discussion of the relevant issues. (See app. II.) NIH also provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to appropriate congressional committees, the Secretary of HHS, the Director of NIH, and other interested parties. We also will make copies available to others on request.

If you or your staff have any questions about this report, please call me at (202) 512-7119. Other major contributors to this report are listed in appendix III.

Scope and Methodology

To determine how NIH provides oversight to protect Phase III clinical trials from fiscal and scientific misconduct, we conducted audit work at NIH; National Institute on Aging (NIA); and National Heart, Lung, and Blood Institute (NHLBI). We selected NIA because it is among the institutes that provide the smallest amount of funding for Phase III clinical trials and NHLBI because it is among the institutes that provide the largest amount of funding. In fiscal year 1995, NIA sponsored 7 clinical trials costing about $9 million, and NHLBI sponsored 42 trials costing about $73 million. Selecting these institutes for review provided some perspective on whether oversight might be influenced by the size of an institute’s clinical trial portfolio. Also, these two institutes offered a variety of trials from which to select for review.

We limited the scope of our review to Phase III clinical trials funded through cooperative agreements. Under cooperative agreements, grantee institutions have more flexibility in planning, conducting, and managing the trials than under contracts, the other major funding method for Phase III clinical trials. NIH institutes that sponsor the trials are expected to provide assistance to and oversight of the trials. Our review included a nonstatistical sample of four multisite clinical trials that varied in nature, size, complexity, and number of sites (see table I.1). We visited five of the clinical research sites that participated in the trials and two data coordinating centers that processed and monitored the clinical data. The clinical sites we visited were either state or private institutions located in Virginia, Connecticut, and Massachusetts. The data coordinating centers we visited differed in how they were funded.

FY 95 funding (000s)

To determine the oversight roles played by NIH, the institutes, and the institutions receiving research funds, we conducted interviews, reviewed NIH rules and regulations, examined NIH studies and reports, and reviewed grant documents on the chosen Phase III clinical trials. We interviewed agency officials from NIH, NIA, and NHLBI. Within NIH, we interviewed officials from the Office of Extramural Research and the Office of Research on Women’s Health. At NIA and NHLBI, we interviewed senior officials, grants management personnel, and program management officers. We also met with staff from HHS’ Office of Research Integrity to discuss their role in investigating allegations of misconduct and the Office of the Inspector General, which was investigating allegations of scientific misconduct.

We also met with the principal research investigators, key research personnel, grants and fiscal management officials, and internal audit staff at the research sites to get their views on oversight responsibilities and controls that protect trials against misconduct. We reviewed grantee institutions’ policies and procedures for preventing, detecting, and resolving scientific misconduct, conflicts of interest, and fiscal mismanagement. Also, we examined research documentation, clinical records, correspondence, and external audits of the institutions.

To determine what controls exist at the central data processing point to help ensure clinical data integrity, we visited two data coordinating centers. One of the centers was funded independently of the clinical sites; the other’s funding was included in the research center’s grant award. At the coordinating centers, we observed their operation, reviewed their policies and procedures, and interviewed key personnel about the centers’ data collection and analysis role and responsibilities. We examined reports generated by the centers and observed the procedures they use to ensure consistency of each clinical site’s data collection and recording methodology. We also established how research data are analyzed to detect data problems and reviewed the follow-up procedures the centers use when potential problem data are discovered.

Our work was performed between September 1995 and May 1996 in accordance with generally accepted government auditing standards.

Comments From the Department of Health and Human Services

Major Contributors to This Report

James O. McClyde, Assistant Director, (202) 512-7152 Frank F. Putallaz, Evaluator-in-Charge, (617) 565-7527 Thomas S. Taydus, Senior Evaluator The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.