Human Subjects Research: Undercover Tests Show the Institutional Review Board System Is Vulnerable to Unethical Manipulation
Released: 2009-03-26T00:00:00

Letter

Background

The Secretary of HHS has issued regulations that form the “Federal Policy for the Protection of Human Subjects.” This policy is often referred to as the “Common Rule” because 17 other federal agencies that conduct, support, or regulate human subjects testing now follow some form of the policy. The Common Rule lays out the basic policies that should govern any research involving human subjects that is approved, funded, or conducted by the agencies that follow the Common Rule, as well as by all entities that need these agencies’ approval of their human subjects research.

Much of the Common Rule focuses on the role of IRBs in the testing process, as IRBs are the primary oversight mechanism for human testing. For example, the policy specifies that there must be at least five members of an IRB, with varying backgrounds, who are sufficiently qualified through experience, expertise, and diversity. The IRB must include members who have the professional competence to review the specific research activities being considered, as well as members with an understanding of a testing entity’s internal protocols, the applicable law, and standards of professional conduct. Furthermore, among other requirements, the IRB should have members of mixed gender and mixed professions; should include at least one member with a scientific background and one with a nonscientific background; and should not have any members with a conflict of interest with the project being reviewed.

The IRB review process is intended to assure, both in advance and by periodic review, that appropriate steps are taken to protect the rights and welfare of humans participating as subjects in the research. IRBs have the authority to approve, require modifications in, or disapprove proposed research. Figure 1 below provides a simplified illustration of the IRB approval process for human subjects research protocols. By law, clinical trials of experimental medical devices and drugs involving human subjects cannot begin until an IRB has approved the research protocol and any changes requested by the IRB have been made. To approve a research proposal, IRBs must determine that the following requirements are satisfied: risks to research participants are minimized; risks to research participants are reasonable in relation to any anticipated benefits, and to the importance of the knowledge that the research might produce; informed consent will be sought from each prospective study participant or the participant’s authorized representative; and there are adequate provisions in place to protect research participants’ privacy and to maintain the confidentiality of research data.

When seeking to obtain research participants’ informed consent to participate in a study, researchers must make sure they offer the potential participants sufficient opportunity to consider whether or not to participate without undue influence or possibility of coercion. In addition, consent forms must contain language that is easily understood, and cannot contain any language that causes or appears to cause the participants to waive their legal rights, or that minimizes or appears to minimize the liability for negligence of the researcher and the sponsors of the research. In addition to reviewing proposed research protocols, IRBs are responsible for conducting continuing review of research at least once a year, or more frequently if the research represents a higher degree of risk to the human research subjects.

IRBs also play a central role in the process by which entities apply for federal funding for human subjects research. An entity must have an approved assurance in order to receive federal funding for research involving human subjects testing from HHS and other federal agencies. An assurance is basically a declaration submitted by an entity engaged in human subjects research that it will comply with the requirements for the protection of human subjects under 45 C.F.R. Part 46. HHS has jurisdiction over human subjects research that is supported through federal funding, and approves assurances for federalwide use. As such, other federal agencies that have adopted the Common Rule may rely on an assurance from HHS for any human subjects research they sponsor. To obtain an assurance, HHS requires an entity to declare to HHS that its activities related to human subjects are guided by ethical principles and federal regulations—the Common Rule—and to designate one or more IRBs to review the research covered by the assurance. In order for the application for assurance to be approved by HHS, all IRBs listed on the application are required to be registered with HHS. IRB registration involves providing HHS with basic information about the IRB, such as the name and contact information for the organization operating the IRB and for its head official, and the names and qualifications of its board members. In evaluating an application to determine whether or not to approve an assurance, HHS is required to consider, among other things, the adequacy of the proposed IRB in relation to the research activities of the entity that submitted the assurance.

Results of Investigation

Establishing an IRB

We succeeded in getting a real company to send a research protocol and related materials to our bogus IRB for its review. As mentioned above, we created a Web site for our bogus IRB that resembled those of actual IRBs, and then advertised the services of our bogus IRB online and in newspapers to attempt to persuade legitimate medical researchers to send protocols to us. In our advertisements, we stated that we were “HHS approved,” in reference to our bogus IRB’s registration with HHS. We also sought to make our IRB look as attractive as possible by emphasizing the speed of our review process (“Fast Approval!”) and flexibility to customer needs. The company that sent materials to us was seeking our bogus IRB’s approval to add one of the company’s clinics as a new test site for ongoing human trials involving invasive surgery. Our bogus IRB could have authorized human subjects testing to begin at this new test site—even though it was a fictitious IRB, with no medical research expertise whatsoever. Moreover, because this transaction involved a company conducting private (i.e., not federally funded) research, and did not involve any FDA-regulated products, our bogus IRB could have approved the research to begin without needing to register with any federal agency. We also received inquiries from five other real companies, which expressed interest in our bogus IRB’s services. However, none of these five companies submitted any materials for us to review.

All IRBs that review federally funded human subjects research are required to be registered with HHS. After we registered our bogus IRB with HHS, HHS provided us with a registration number and listed our bogus IRB in its online directory of registered IRBs that review federally funded research. Our only communication with HHS as part of registering our IRB was through an online registration form, with no human interaction. The IRB registration process is meant to collect data that HHS uses during the subsequent assurance approval process. As such, HHS is not required to verify the information it receives during the IRB registration process. However, our investigation of the assurance process, as described below, shows the importance of IRB registration data as they relate to HHS’s evaluation of assurance applications. Moreover, if our bogus IRB had been an actual IRB that did not intend to review federally funded human subjects research, it would not have been required to submit any registration information. IRBs that intend to review privately funded human subjects research are not currently required to register with HHS or any other federal agency, although recently implemented regulations will change this as of July 2009.

HHS’s Federalwide Assurance Process

We found that the process for obtaining HHS approval for an assurance lacks effective controls. As mentioned above, we formed a fictitious medical device company with phony company officials and a mailbox for its business location—where human subjects research would supposedly be conducted. We then submitted an application to HHS for its approval of an assurance on behalf of our fictitious medical device company. As part of the application, we named our bogus IRB as the IRB responsible for reviewing the research covered by the assurance. HHS approved our assurance application, provided us with an assurance approval number, and listed our bogus medical device company in its online directory of approved assurances. Our only communication with HHS as part of this application was through an online application form and a faxed signature to complete the application. We did not have any real-time contact with HHS, whether by telephone, in person, or through a site visit.

We do not know what verification HHS performed, if any, in its review of our assurance application. However, if HHS had performed basic screening of the assurance application, HHS would have found discrepancies that would have warranted further investigation, such as the fact that we used only a mailbox as our business location. As mentioned above, in evaluating an application to determine whether or not to approve an assurance, HHS is required to consider the adequacy of any IRB designated on the application, as the IRB will be responsible for overseeing the research activities of the entity that submitted the assurance application. By approving our assurance application, HHS essentially deemed our bogus IRB as adequate to oversee human subjects research, as conducted by our fictitious medical device company. Moreover, by obtaining an approved assurance from HHS, our fictitious medical device company can apply for federal research funding from HHS or other federal agencies. In addition, we used the assurance approval to boost the credibility of our fictitious medical device company by posting our assurance number on the fictitious medical device company’s Web site.

The IRB that approved our fictitious medical device protocol, as discussed below, is listed on HHS’s Web site as being involved in more than 70 assurances on behalf of actual medical researchers. Each of these assurances is a first step for the medical researcher to apply for federal funding for human subjects research, with this IRB formally designated to oversee the research.

IRBs’ Research Protocol Approval Process

We were able to get an actual IRB to approve a fictitious protocol for human subjects research, which raises concerns that other IRBs may conduct protocol reviews without exercising due diligence, thereby exposing research volunteers to significant risk. For this test, we created a research protocol for a fictitious medical device with no proven test history and bogus specifications, and sent the protocol to three actual, independent IRBs under the guise of the medical device company we created for obtaining an assurance from HHS in our second test, as mentioned above. Our protocol offered only vague information about certain aspects of our proposed study and was designed using information publicly available on the Internet. As mentioned above, our fictitious device was a post-surgical healing device for women that matched multiple examples of “significant risk” devices provided in FDA guidance. In addition, we fabricated additional documents we needed to submit along with our protocol, such as a CV detailing the educational and professional experience of a fictitious researcher at our company, and a bogus medical license for the researcher. We succeeded in getting our fictitious protocol approved by an IRB, even though we were a bogus company with falsified credentials and an unproven medical device. If we had been a real medical device company, we could have begun testing our “significant risk” experimental device on actual human subjects. We also could have used our bogus IRB mentioned above to approve our fictitious protocol. This shows the potential for unethical manipulation in the IRB system.

The IRB that approved our bogus research protocol (IRB 1) required only minor edits to our submission materials, and did not verify that the information contained in our protocol and related materials was correct or authentic, or even that our medical device company actually existed. For example, we provided IRB 1 with bogus information that FDA had already cleared our device for marketing because our device was found to be substantially equivalent to an existing, legally marketed device. IRB 1 did not attempt to verify this information even though a quick check of FDA’s online database would have shown no evidence that FDA had ever cleared our device. By taking advantage of this lapse, our investigators—who lacked technical expertise in this subject—bypassed any requirement to develop a risk assessment for a device that, under normal circumstances, would be considered “significant risk” according to FDA guidance. Meeting minutes from IRB 1’s board meeting show that it accepted the bogus information about FDA clearance of our device as evidence that our device did not require any further risk assessment. See figure 2 below.

IRB 1 “conditionally approved” our protocol after a full board review, but requested that we modify our informed consent form for study participation in order to make the language understandable at a fifth-grade reading level. We modified our informed consent form as requested by using medical information found on the Internet, after which the board members of IRB 1 voted unanimously to approve our fictitious medical device protocol (see fig. 2 above). IRB 1 approved our fictitious protocol, thereby authorizing us to begin human testing, after only contacting us by e-mail or fax, and never by telephone or in person. IRB 1’s board meeting minutes indicate that it believed our device was “probably very safe,” as shown in figure 2 above. Although our protocol mentioned fictitious animal studies that we conducted on our device to ensure its safety, IRB 1 approved our protocol without ever seeing proof of these studies or any other evidence that our device was reasonably safe for use in human subjects. On its Web site, IRB 1 advertises the speed of its reviews and states that it performs a “triple check” for quality. IRB 1 has approved research protocols for experimental drugs tested by major pharmaceutical companies.

The remaining two IRBs (IRB 2 and IRB 3) provided feedback on our protocol that was so extensive we determined we did not have the technical expertise or resources to gain approval. The extensive nature of the feedback IRB 2 and IRB 3 provided on our initial submission materials indicated that they follow a much more thorough review process than IRB 1, which approved our protocol. For example, IRB 2 noticed that our fictitious protocol mentioned previous testing of the device performed on animals, and requested that we provide a copy of the results from the fictitious animal testing. In addition, IRB 3 requested that we send it a copy of the diagram that our bogus researcher would use to record incision lines he made as part of the surgery involved in our study, and raised a number of questions about the timing and locations involved in our fictitious testing. The documents and information that IRB 2 and IRB 3 requested would have taken extensive time and research to fabricate, and demanded a level of technical expertise that we did not possess. IRB 1 approved our protocol without obtaining any of the additional information requested by IRB 2 and IRB 3. Our contacts with IRB 2 and IRB 3, during their review of our protocol, were done entirely by e-mail.

We later interviewed representatives from IRB 2 and IRB 3 to obtain additional details about why they did not approve our protocol. Representatives from both IRBs expressed concern that our protocol did not contain adequate information about the safety of our fictitious medical device. For example, the manager of IRB 2 said that she worried that our device could cause infection in patients, or possibly even cause patients to develop sepsis. In addition, a board member from IRB 3, who claimed to have 15 years of experience reviewing research protocols with this IRB, stated that our protocol lacked any evidence that our bogus medical device was actually safe for implantation into a human body. He also said that IRB 3’s board voted unanimously to reject our bogus protocol. Figure 3, below, shows additional examples of IRB 2’s and IRB 3’s comments on our fictitious medical device and protocol.

None of the three IRBs questioned us about the authenticity of our bogus CV and counterfeit medical license. As mentioned above, we fabricated these documents by using information found online and with commercially available hardware, software, and materials. Our bogus CV contained information on our fictitious researcher’s human subjects research background, which we created by using phony drug and device names and with information that we accessed on the Internet. Our counterfeit medical license contained a bogus license number with a similar format to real license numbers used by the state we claimed our license was from.

Briefing with HHS

We briefed HHS officials on the results of our investigation. They stated that HHS receives around 300 IRB registrations and 300 assurance applications every month, and that OHRP currently has three employees who review all registrations and applications. According to HHS officials, the department does not review IRB registrations or assurance applications to assess whether the information submitted is factual. HHS officials said that the department reviews assurance applications to ensure that applicants have submitted all of the necessary information and meet minimum standards. Moreover, although HHS is required by law to consider the adequacy of IRBs listed on assurance applications when reviewing applications, the director of OHRP stated that his office would require more staff to do so. However, HHS officials added that they would not consider additional evaluation of IRB registrations or assurance applications to be worthwhile even if the office had increased resources.

HHS officials stated that the assurance process is not a meaningful protection against unethical manipulation. They stated their belief that anyone submitting false or misleading information as part of the assurance application process would likely be detected during the subsequent process of applying for federal funding for human subjects research. However, our work shows that an unethical company could leverage an HHS assurance for purposes unrelated to the federal funding application process. For example, representatives from one of the IRBs that rejected our protocol stated that the HHS assurance number listed on our bogus medical device company’s Web site gave our company credibility because it meant that HHS had recognized our company. When we discussed this with HHS, the director of OHRP acknowledged that an HHS-approved assurance is meaningful in this regard.

Mr. Chairman, this concludes our statement. We would be pleased to answer any questions that you or other members of the subcommittee may have at this time.

Contacts and Acknowledgments

For further information about this testimony, please contact Gregory D. Kutz at (202) 512-6722 or kutzg@gao.gov. Contacts points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. GAO staff who made major contributions to this testimony include Matthew D. Harris, Assistant Director; Matthew Valenta, Assistant Director; Timothy Persons, Chief Scientist; Christopher W. Backley; Ryan Geach; Ken Hill; Jason Kelly; Barbara Lewis; Andrew McIntosh; Sandra Moore; James Murphy; and Seong B. Park.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.