Health Research Misconduct: HHS' Handling of Cases Is Appropriate, but Timeliness Remains a Concern
Released: 1995-09-05T00:00:00

Letter

Background

ORI is an independent group within HHS; its Director reports to the Secretary. Created from a merger of two offices within HHS, ORI’s mission is to oversee and direct PHS research integrity activities, which it does primarily through its handling of scientific misconduct investigations. In fiscal year 1994, ORI had a total operating budget of $4 million and maintained a staff of about 50 employees; currently, it has 43 employees.

Although ORI investigates misconduct related to intramural research programs, about three-fourths of its caseload in 1994 related to oversight of extramural integrity reviews conducted by grantee institutions. ORI generally monitors the progress of an extramural investigation and reviews the institution’s final report. ORI also presents the results of misconduct investigations in administrative hearings before the HHS Departmental Appeals Board if ORI’s decisions are challenged.

Besides its investigative function, ORI performs other research integrity activities. These efforts include developing model policies and procedures for handling allegations of scientific misconduct; evaluating institutional policies and processes for conducting investigations; investigating whistleblower retaliation complaints; and promoting scientific integrity through educational initiatives and other collaborations with universities, medical schools, and professional societies.

Most allegations of scientific misconduct are made directly to the institutions conducting the research. Responding to an allegation involves a two-step process: an inquiry and, if necessary, an investigation. Institutions have the primary responsibility for responding to allegations involving extramural research; ORI’s role in these instances is usually that of reviewing the institution’s investigation report. ORI generally does not review institutional inquiries because an institution is not required to inform ORI that an inquiry is under way nor to submit a report at its conclusion.

ORI does, however, review all investigations. Institutions must inform ORI when they begin an investigation and submit a report at its conclusion. ORI reviews the final report, the supporting materials, and the determinations to decide whether the investigation has been performed with sufficient objectivity, thoroughness, and competence.

ORI plays a more direct role in responding to scientific misconduct allegations in PHS intramural research programs. It reviews all misconduct inquiries conducted by PHS agencies and conducts all investigations when they are needed. ORI’s handling of intramural scientific misconduct cases can be a complex undertaking that may involve collaborations among ORI staff, other agencies, and institutions performing research.

In general, for intramural research allegations, the review process begins when an individual making an allegation (referred to as a complainant) alleges to either ORI or a PHS agency that another researcher (a respondent) committed scientific misconduct. If a misconduct allegation is made to ORI, an investigator within ORI’s Division of Research Investigations (DRI) conducts an initial screening primarily to determine if PHS funding is involved and whether the allegation falls within the PHS definition of scientific misconduct. Allegations that do not meet these criteria result in no action or are referred outside of ORI for consideration.

When allegations do fall within PHS’ definition of misconduct, ORI forwards them to the PHS agency that funded the research and directs that agency to conduct a formal inquiry. This involves gathering information—including interviewing the subjects involved—to determine the nature of evidence available to support the allegation. ORI investigators may monitor inquiries and advise PHS agencies on matters such as procedures for sequestering laboratory research notebooks. They often directly assist the agency in sequestering the research data and other evidence.

If the results of an inquiry suggest that misconduct may have occurred, ORI then opens a full investigation to determine the existence and magnitude of misconduct. An investigation could involve an extensive review of experiments and other scientific data as well as interviews with all parties involved with the research. The ORI investigator assigned to the case may seek assistance from a staff biostatistician and other in-house experts. Also, ORI may elicit assistance from outside scientists who have expertise in subject areas that ORI staff lack. Investigators produce a written report with findings. The report is reviewed by ORI management, its legal staff, and the respondent before being issued by the ORI Director. For investigations that result in a finding of misconduct, the ORI Director, in combination with the HHS debarring official, determines possible sanctions against the respondent, which may include debarment from receiving federal grant or contract funds for a specified period.

ORI’s Procedures Conform to Generally Accepted Standards

ORI developed procedures for handling scientific misconduct cases and implemented them in November 1992. These procedures detail ORI’s process for receiving and assessing misconduct allegations, reviewing PHS agency inquiry reports, conducting investigations, and overseeing extramural investigations. The procedures were developed by a task force, consisting mainly of ORI management (in consultation with officials from PHS agencies) and the HHS Office of the General Counsel and the IG.

We compared ORI’s policies and procedures with investigation guidelines established by the President’s Council on Integrity and Efficiency (PCIE). The PCIE guidelines apply to federal government investigations and generally outline issues and procedures for handling matters such as background and security inquiries as well as special investigations requested by any appropriate authority. These standards were established through a collaborative effort of staff from various inspector general offices throughout government.

We found ORI’s procedures for handling scientific misconduct cases to be consistent with PCIE standards. Specifically, ORI procedures meet the PCIE standards by containing explicit statements on the qualifications of staff needed to handle investigations; independence required to conduct investigations; due professional care needed for the work; and other qualitative standards, such as planning, executing, and reporting investigation results.

Investigation Techniques Applied Appropriately

ORI investigators handling misconduct cases are scientists with doctoral degrees who were engaged in scientific research prior to their tenure with ORI. They represent varied scientific disciplines, such as biochemistry, genetics, biomedical engineering, and nutritional science. At the time of our review, each investigator had received the introductory investigation course given to most federal law enforcement agents. Supervisory investigators had taken some of the more advanced courses as well.

Our assessment of case files confirmed that ORI investigators documented the work performed and followed established procedures in screening allegations and handling misconduct investigations.

Allegations Screened and Closed Appropriately

ORI investigators appeared to be making appropriate decisions as to which allegations did not merit further examination beyond their initial screening. We reviewed ORI case files on 30 allegations made to ORI since June 1993 that were closed without a formal inquiry. We sampled these 30 cases from a universe of 113 such closures. In each case, investigators followed established procedures and appropriately followed up on leads, and logically closed out the screening process.

Our interviews with four individuals who had contacted ORI revealed a general satisfaction with ORI’s handling of their allegations or requests for information. For example, a scientist who had asked whether a laboratory chief could take authorship credit for research conducted in his facility told us he accepted ORI’s explanation that his inquiry did not constitute misconduct. The scientist added that the ORI investigator handling the call provided useful information on NIH guidelines for research collaborations.

Investigations Handled Appropriately

ORI investigators also appeared to have followed established procedures for the 10 investigations we reviewed. However, two limitations on our analysis should be noted. First, at the time of our review, ORI had opened and closed only four intramural investigations since its formation in May 1992. Second, these four investigations did not require investigators to apply sophisticated investigative or scientific techniques. (For example, two of them related to alleged falsification of academic credentials.) The remaining six cases involved possible misconduct in extramural research in nonfederal institutions. In these six cases, ORI’s role was that of oversight, reviewing the institutions’ investigations.

We concluded from our review of case files for the four ORI-led investigations that ORI investigators employed appropriate techniques. Specifically, investigators developed investigation plans, interviewed relevant individuals, analyzed scientific data where appropriate, coordinated with other HHS offices, appropriately followed up on leads, and wrote reports with evidence supporting their decisions.

ORI investigators also appeared to have followed proper procedures in reviewing the extramural investigations. Our examination of the six extramural case files revealed that ORI investigators adequately documented their work and included relevant documents, such as copies of the inquiry and investigation reports, in case files. We observed from our review of documentation in the case files that investigators generally followed the steps outlined in the ORI procedures manual. For example, investigators made appropriate contacts with institutions and took steps to ensure that the institution conducting the investigation properly notified the complainant and respondent at various stages of the investigation.

Delays in Closing Cases Contribute to Backlog

ORI’s procedures specify time frames for screening allegations and for conducting inquiries and investigations. These procedures state that screening should be completed within 30 days of receipt of the allegation. Inquiries are generally to be completed within 60 days of their initiation and investigations within 120 days.

We observed delays in ORI’s handling of misconduct cases. ORI’s inability to close current cases in a timely manner has contributed to a backlog, some of which it inherited from its predecessor offices. When ORI was established, it inherited 70 active cases (inquiries and investigations) and about 420 more allegations which had apparently not been reviewed or screened. Although it has made progress in working through these inherited cases, ORI still has a substantial backlog. On April 30, 1995, ORI reported 169 active cases, including 71 inquiries and investigations.

Many Allegations Still Not Screened After 6 Months

Although ORI completed the initial screening on 208 of the 288 misconduct allegations it received between June 1, 1993, and December 6, 1994, ORI investigators had not completed the screening process for the remaining 80 allegations, even though most of them had been unresolved for more than the 30 days allotted. More importantly, a majority of these (45 of 80) had remained open for over 6 months.

Investigators and supervisors we interviewed attributed the backlog to competing work priorities. Our discussions with investigators and analysis of their workload indicated that, generally, investigators are each assigned 6 to 10 allegations to review in addition to their caseload of open investigations, inquiries, and oversight of extramural investigations. Although none of the investigators indicated that the workload was too high, they expressed concern about the backlog of initial allegations.

Closures of Investigation and Oversight Cases Take a Long Time

For the four ORI-led investigations we reviewed, ORI went well beyond the targeted 120 days to complete them. Although we could not determine the actual staff time spent on these cases, the elapsed calendar time ranged from about 6 to 13 months. In two instances, investigators took what appeared to be an inordinate amount of time to complete relatively straightforward cases. For example, ORI took over a year to investigate and adjudicate a case of alleged falsification of academic credentials in several NIH grant applications. In another case, ORI took about 6 months for an investigation in which the respondent submitted a statement partially admitting to the misconduct prior to ORI’s opening an investigation.

ORI investigators indicated that higher priority cases prevented them from closing these cases more expeditiously. The investigators also gave specific reasons for each case. In the first case, investigators wanted to establish a pattern of falsifying credentials to counter the respondent’s claim that the incident was not common. In the other case, ORI initiated an investigation because it wanted to ensure that appropriate procedures were followed and that the full extent of the respondent’s misconduct was identified.

We also observed a lack of timeliness in closing extramural investigations. The six cases we reviewed were open for about 9 to 13 months. The time spent on four of these cases can be partly attributed to additional work ORI did on these cases after the institutions completed their investigations.

ORI Efforts Improve Case Handling

During the course of our review, ORI officials took various steps to reduce the case backlog and improve ORI’s work. These actions ranged from giving greater attention to setting priorities among cases to providing increased guidance to extramural institutions.

Priority Setting—ORI has begun holding frequent management meetings to systematically review all open cases. The point is to decide which cases can be closed and to set priorities among the open cases.

Early Settlement Agreements—ORI has also begun to seek earlier resolutions of cases through advance settlements with respondents (generally referred to as voluntarily exclusions). When respondents voluntarily agree to or accept ORI’s early disposition of a case, further pursuit of an investigation or appeal can be avoided. Significant savings in investigative and litigation resources may result.

Reassigning Program Analysts—ORI has assigned a program analyst to expedite allegation assessments by performing initial tasks, such as securing research articles and grant information. Managers and investigators indicated that this effort has proven useful and support the increased use of program analysts for this purpose.

Guidance to Institutions—In an effort to better educate intramural and extramural institutions on handling scientific misconduct, ORI has instituted formal processes for communicating with these entities. ORI now issues a quarterly newsletter, conducts seminars, and posts notices on an HHS computer bulletin board. Additionally, in November 1994, ORI issued draft model policies and instructions for handling misconduct cases to extramural institutions. In their present form, the guidelines are intended to assist institutions in complying with federal regulations. ORI sent these draft procedures to officials at 40 extramural institutions requesting their review and comment. We interviewed four of these officials, and the consensus was that the draft procedures would have a positive effect by giving institutions improved guidance for investigations.

Although these measures appear to have helped ORI improve its handling of cases, additional efforts are needed to more effectively respond to workload demands.

Further Efforts Needed to Effectively Respond to Workload Demands

Facing a substantial case backlog and lengthy delays in completing its work, ORI needs additional management tools to meet its workload demands. Specifically, ORI still needs strategic planning and resource assessments to decide how to most efficiently and effectively deploy its staff.

For example, 11 of ORI’s staff (within DRI) are directly involved in investigations full time. The remaining 32 staff members (about 75 percent of total staff) are either professional or administrative staff who support DRI or are devoted to other ORI functions, such as policy development and education. Investigative work is not ORI’s only responsibility. Given the case backlog, however, ORI’s current staff allocation to investigations may not be sufficient even with the recent improvements ORI has made.

ORI also needs a system to track the amount of time investigators spend on cases. Generally, each investigator handles 6 to 10 initial allegations of misconduct, 1 to 3 investigations, and 1 to 4 oversight cases. Some investigators we interviewed expressed occasional uncertainty about whether their use of time coincided with management’s priorities.

Planning processes, such as routine staffing assessments, could help ORI’s management team systematically gauge the appropriate balance between ORI’s needs and resources. Staffing assessments might also help identify ways to augment ORI’s skill base—for example, identifying the need for different disciplines and backgrounds among the staff, such as trained criminal investigators. Such assessments might also help management determine ways to better use its administrative staff.

The HHS IG reached a similar conclusion in its November 1994 report on ORI’s staffing and management. The IG recommended that ORI develop a strategic plan to help it “be better prepared to handle fluctuations in its work load and to provide a balance between its roles in stewardship and research integrity education.” The plan, according to the IG, should detail objectives in specific, measurable terms and show how resources and staff should be allocated to accomplish these objectives. The IG’s report made a number of other recommendations designed to improve ORI’s productivity.

Another deficiency noted in the IG’s report was the absence of a structured timekeeping system. The report concluded that implementing such a system would greatly aid in determining whether ORI needs additional investigative staff. The IG recommended that ORI set and enforce performance measures for its staff regarding the quality, quantity, and timeliness of work conducted.

Our work supports the IG’s conclusion that ORI needs a strategic plan and specific performance measures for its staff. Such a plan—particularly if it includes (1) a comprehensive assessment of ORI’s workload and staffing requirements and (2) measures to reduce the case backlog and close cases more quickly—should help ensure an optimum use of resources.

Among its fiscal year 1995 management initiatives, ORI has started work on a strategic plan and will begin setting specific performance measures. Additionally, ORI officials told us they had initiated a two-pronged pilot study for tracking investigators’ time. One part of the pilot requires investigators to track time spent on an investigation. The second part requires investigators to record the time they devote to the specific tasks they perform, such as interviewing and analyzing research experiments, in addition to the total time spent.

Conclusion

Since its inception, ORI has made progress in improving its handling of scientific misconduct cases. By continuing to follow sound investigative procedures and striving to improve its handling of cases, the office will gain increased public trust as a preserver of federal interest in biomedical research. However, persistent delays in case handling and deficiencies in its management systems are barriers that ORI needs to overcome if it is to effectively fulfill its mission in the future. ORI’s management team must confront these challenges and develop strategies to address them.

Agency Comments

HHS provided comments on a draft of this report, which we incorporated where appropriate (see app. II). HHS generally agreed with our findings and representation of its current efforts to improve productivity. HHS also described planned efforts to reduce the “management superstructure of ORI,” which should result in productivity gains. We incorporated technical comments provided by HHS, but did not include them in the appendix.

As arranged with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after its issue date. At that time, we will send copies to interested parties and make copies available to others on request.

Please call me on (202) 512-7119 if you or your staff have any questions about this report. Other major contributors are listed in appendix III.

Scope and Methodology

To assess ORI’s process for handling misconduct cases, we reviewed its written guidance and examined how it screens allegations and conducts investigations and oversight functions. We compared ORI’s written policies and procedures for handling misconduct allegations and investigations with guidelines established for federal agencies that engage in comparable activities.

In examining how ORI handles and screens misconduct allegations, we reviewed case files for 30 of the almost 300 allegations received from June 1993 to December 6, 1994. We selected cases that did not proceed to the inquiry phase. For four of these cases, we interviewed the individuals who made the allegations to obtain their perspectives on how well ORI handled them. We selected these particular individuals primarily because their case files did not contain sufficient information for us to determine whether ORI had completed its work responding to the allegations.

To assess ORI procedures for conducting and monitoring misconduct investigations, we reviewed the 10 investigations that were opened since ORI’s establishment in May 1992 and completed by the time of our review. ORI conducted 4 of the 10 investigations; the remaining 6 were done by institutions and reviewed by ORI. We did not review cases initiated and conducted primarily by ORI’s predecessor offices because ORI had not implemented its current investigation procedures when these cases were opened. In addition, we neither independently verified the information ORI investigators used to reach their conclusions nor conducted our own investigation of cases. We supplemented our reviews of ORI case files with interviews with the seven investigators, two supervisory investigators, and the DRI Acting Director. We primarily sought to further our understanding of the investigative techniques used in handling misconduct cases, particularly the cases that presented greater technical challenges for investigators. As part of our interviews, we discussed procedures being used for cases currently under review.

We interviewed officials at intramural and extramural institutions to gain their perspectives on ORI guidance for handling misconduct and on the quality of ORI investigations. We sought to obtain their views on ways in which ORI could improve its handling of misconduct cases.

We also analyzed ORI’s automated case tracking system, which contains misconduct allegations. Finally, we interviewed ORI’s Deputy Director and the DRI Acting Director to ascertain current strategies to improve misconduct case management.

We did not independently verify the accuracy of the data in ORI case files or automated databases. We did our work between July 1994 and April 1995 in accordance with generally accepted government auditing standards.

Comments From the Department of Health and Human Services

Major Contributors to This Report

Barry Tice, Assistant Director, (202) 512-4552 Glenn Davis, Evaluator-in-Charge, (312) 220-7600 Fred Chasnov Woodrow Hunt Cameo Zola The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (301) 258-4097 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Address Correction Requested