Human Subjects Research: Undercover Tests Show the Institutional Review Board System Is Vulnerable to Unethical Manipulation
Published: 2009-03-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Millions of Americans enroll in clinical studies of experimental drugs and medical devices each year. Many of these studies are meant to demonstrate that products are safe and effective. The Department of Health and Human Services' (HHS) Office for Human Research Protections (OHRP) and the Food and Drug Administration (FDA) are responsible for overseeing aspects of a system of independent institutional review boards (IRB). IRBs review and monitor human subjects research, with the intended purpose of protecting the rights and welfare of the research subjects. GAO investigated three key aspects of the IRB system: (1) the process for establishing an IRB, (2) the process through which researchers wishing to apply for federal funding assure HHS their human subjects research activities follow ethical principles and federal regulations, and (3) the process that medical research companies follow to get approval for conducting research on human subjects. GAO investigated these three aspects of the IRB system by creating two fictitious companies (one IRB and one medical device company), phony company officials, counterfeit documents, and a fictitious medical device.
section_title: What GAO Found
paragraphs: The IRB system is vulnerable to unethical manipulation, which elevates the risk that experimental products are approved for human subject tests without full and appropriate review. GAO investigators created fictitious companies, used counterfeit documents, and invented a fictitious medical device to investigate three key aspects of the IRB system. These are the results: Establishing an IRB. GAO created a Web site for a bogus IRB and advertised the bogus IRB's services in newspapers and online. A real medical research company contacted the bogus IRB to get approval to join ongoing human trials involving invasive surgery--even though GAO's investigators had no medical expertise whatsoever. Since the transaction involved privately funded human subjects research and did not involve any FDA-regulated drugs or devices, GAO's bogus IRB could have authorized this testing to begin without needing to register with any federal agency. Obtaining an HHS-approved assurance. GAO also registered its bogus IRB with HHS, and used this registration to apply for an HHS-approved assurance for GAO's fictitious medical device company. An assurance is a statement by researchers to HHS that their human subjects research will follow ethical principles and federal regulations, which is required before researchers can receive federal funding for the research. On its assurance application, GAO designated its bogus IRB as the IRB that would review the research covered by the assurance. Even though the entire process was done online or by fax--without any human interaction--HHS approved the assurance for GAO's fictitious device company. With an HHS-approved assurance, GAO's device company could have applied for federal funding for human subjects research. Obtaining IRB approval for human testing. GAO succeeded in getting approval from an actual IRB to test a fictitious medical device on human subjects. GAO's fictitious device had fake specifications and matched several examples of "significant risk" devices from FDA guidance. The IRB did not verify the information submitted by GAO, which included false information that FDA had already cleared GAO's device for marketing. Although records from this IRB indicated that it believed GAO's bogus device was "probably very safe," two other IRBs that rejected GAO's protocol cited safety concerns with GAO's device. No human interaction with these IRBs was necessary as the entire process was done through e-mail or fax. GAO's bogus IRB mentioned above also could have approved the fictitious protocol, which shows the potential for unethical manipulation in the IRB system. GAO briefed HHS officials on the results of its investigation. The director of OHRP stated that, when reviewing assurance applications, HHS does not consider whether IRBs listed on the applications are adequate--even though HHS is required to do so by law. In addition, HHS officials stated that the department does not review assurance applications to determine whether the information submitted by applicants is factual.

section_title: Letter
paragraphs: Thank you for the opportunity to discuss our investigation of  vulnerabilities in the institutional review board (IRB) system. An IRB is an  entity formally designated to review and monitor biomedical and  behavioral research in clinical trials involving human subjects, with the  intended purpose of protecting the rights and welfare of the research  subjects. Each year, millions of Americans enroll in clinical trials of  experimental drugs and medical devices conducted in over 350,000  locations throughout the United States. Many of these clinical trials are  meant to demonstrate that products are safe and effective, and are  sometimes conducted or sponsored by private pharmaceutical and  medical device manufacturers. Although research subjects are required to  give consent prior to their participation in these studies, a patient has the  expectation that the product being tested presents a risk that is reasonable  in relation to any anticipated benefits, and that all risks are fully disclosed.  The Department of Health and Human Services’ (HHS) Office for Human  Research Protections (OHRP) and the Food and Drug Administration  (FDA) are responsible for overseeing aspects of the system of IRBs.
Unfortunately the IRB system sometimes fails to protect research subjects.  For example, in 2002, a 47-year-old man died after his heart stopped  beating while participating in an experimental trial of antipsychotic  medication at a Texas research center. Before his death, the man spent 22  days suffering from fever, severe diarrhea, a rapid heartbeat, and kidney  failure while under the care of researchers. The warning label for the  experimental medication listed some of these serious side-effects and  other signs of heart failure, but the IRB failed to ensure the risks were  communicated to participants at the outset of the trial. During the clinical  trial, the lead researcher continually delegated control of the clinical trial  to a man who was unlicensed to practice medicine in the United States. In  its follow-up investigation after the death, the FDA noted that the IRB  repeatedly violated regulations governing the proper conduct of clinical  trials and did not adequately supervise the clinical trial.
Most IRBs were historically located at academic institutions. However,  independent IRBs are playing an increasingly prominent role in the  protection of human research subjects. Questions have been raised as to  whether all of these independent IRBs exercise effective due diligence in  reviewing research protocols. Given the importance of IRBs in protecting  human health and safety, you asked us to perform undercover tests to find  out whether the IRB system is vulnerable to unethical manipulation.  Specifically, we investigated three key aspects of the IRB system: (1) the  process for establishing an IRB, (2) the process through which researchers  who wish to apply for federal funding assure HHS that their activities  related to human subjects are guided by ethical principles and federal  regulations, and (3) the process that medical research companies follow to  get approval for conducting research on human subjects.
To investigate the process for establishing an IRB, we created a fictitious  IRB with phony company officials and only a mailbox for a business  location. We then registered our fictitious IRB with HHS using its online  registration form. We created a Web site that resembled those of other  actual IRBs. We also advertised the services of our bogus IRB in various  media, such as Web sites dedicated to the clinical trials industry and  newspapers, in an attempt to persuade legitimate medical researchers to  send protocols to our bogus IRB. In our advertisements, we stated that we  were “HHS approved,” in reference to our bogus IRB’s registration with  HHS. In addition, we emphasized the speed of our review process (“Fast  Approval!”), customer service, and flexibility to customer needs in order  to make our IRB look as attractive as possible.
To investigate the process through which human subjects researchers who  wish to apply for federal funding assure HHS that their activities related to  human subjects are guided by ethical principles and federal regulations,  we attempted to file a Federalwide Assurance for the Protection of Human  Subjects for Institutions Within the United States (assurance) application  using HHS’s online application form, under the guise of a fictitious medical  device company. We created a fictitious medical device company with  phony company officials and only a mailbox for a business location,  claiming that this mailbox was the facility where we intended to conduct  our human subjects testing. As part of filing for an assurance, we were  required to submit information about the IRB that would be reviewing our  research protocol, for which we listed our fictitious IRB.
To investigate the process that medical research companies follow to get  approval for conducting research on human subjects, we created a  research protocol for a fictitious medical device with no proven test  history and bogus specifications, using information publicly available on  the Internet. We designed our protocol so that it would contain vague  information about certain aspects of our proposed study. Our fictitious  device was a post-surgical healing device for women that matched  multiple examples of “significant risk” devices provided in publicly  available FDA guidance. Our bogus medical device company then  approached three actual, independent IRBs with information about our  device and indicated that we wanted to submit our protocol for review and  approval to conduct human testing. We selected these three IRBs by  conducting a search online to identify independent IRBs, and then  choosing three that we determined had less burdensome initial paperwork  requirements than other IRBs for protocol submission. We fabricated  additional documents requested by the IRBs for their initial review of our  protocol, such as a curriculum vitae (CV) detailing our fictitious  researcher’s educational and professional experience, and a medical  license for our fictitious researcher. We created these counterfeit  documents by using information found online and with commercially  available hardware, software, and materials. After concluding the  undercover portion of our investigation, we contacted two of the three  IRBs to obtain information about their review process.
We performed this investigation from January 2008 to March 2009 in  accordance with quality standards for investigations prescribed by the  President’s Council for Integrity and Efficiency.
subsections: section_title: Background
paragraphs: The Secretary of HHS has issued regulations that form the “Federal Policy  for the Protection of Human Subjects.” This policy is often referred to as  the “Common Rule” because 17 other federal agencies that conduct,  support, or regulate human subjects testing now follow some form of the  policy. The Common Rule lays out the basic policies that should govern  any research involving human subjects that is approved, funded, or  conducted by the agencies that follow the Common Rule, as well as by all  entities that need these agencies’ approval of their human subjects  research.
Much of the Common Rule focuses on the role of IRBs in the testing  process, as IRBs are the primary oversight mechanism for human testing.  For example, the policy specifies that there must be at least five members  of an IRB, with varying backgrounds, who are sufficiently qualified  through experience, expertise, and diversity. The IRB must include  members who have the professional competence to review the specific  research activities being considered, as well as members with an  understanding of a testing entity’s internal protocols, the applicable law,  and standards of professional conduct. Furthermore, among other  requirements, the IRB should have members of mixed gender and mixed  professions; should include at least one member with a scientific  background and one with a nonscientific background; and should not have  any members with a conflict of interest with the project being reviewed.
The IRB review process is intended to assure, both in advance and by  periodic review, that appropriate steps are taken to protect the rights and  welfare of humans participating as subjects in the research. IRBs have the  authority to approve, require modifications in, or disapprove proposed  research. Figure 1 below provides a simplified illustration of the IRB  approval process for human subjects research protocols. By law, clinical  trials of experimental medical devices and drugs involving human subjects  cannot begin until an IRB has approved the research protocol and any  changes requested by the IRB have been made. To approve a research  proposal, IRBs must determine that the following requirements are  satisfied:  risks to research participants are minimized;  risks to research participants are reasonable in relation to any  anticipated benefits, and to the importance of the knowledge that the  research might produce;  informed consent will be sought from each prospective study  participant or the participant’s authorized representative; and  there are adequate provisions in place to protect research participants’  privacy and to maintain the confidentiality of research data.
When seeking to obtain research participants’ informed consent to  participate in a study, researchers must make sure they offer the potential  participants sufficient opportunity to consider whether or not to  participate without undue influence or possibility of coercion. In addition,  consent forms must contain language that is easily understood, and cannot  contain any language that causes or appears to cause the participants to  waive their legal rights, or that minimizes or appears to minimize the  liability for negligence of the researcher and the sponsors of the research.  In addition to reviewing proposed research protocols, IRBs are  responsible for conducting continuing review of research at least once a  year, or more frequently if the research represents a higher degree of risk  to the human research subjects.
IRBs also play a central role in the process by which entities apply for  federal funding for human subjects research. An entity must have an  approved assurance in order to receive federal funding for research  involving human subjects testing from HHS and other federal agencies. An  assurance is basically a declaration submitted by an entity engaged in  human subjects research that it will comply with the requirements for the  protection of human subjects under 45 C.F.R. Part 46. HHS has jurisdiction  over human subjects research that is supported through federal funding,  and approves assurances for federalwide use. As such, other federal  agencies that have adopted the Common Rule may rely on an assurance  from HHS for any human subjects research they sponsor. To obtain an  assurance, HHS requires an entity to declare to HHS that its activities  related to human subjects are guided by ethical principles and federal  regulations—the Common Rule—and to designate one or more IRBs to  review the research covered by the assurance. In order for the application  for assurance to be approved by HHS, all IRBs listed on the application are  required to be registered with HHS. IRB registration involves providing  HHS with basic information about the IRB, such as the name and contact  information for the organization operating the IRB and for its head official,  and the names and qualifications of its board members. In evaluating an  application to determine whether or not to approve an assurance, HHS is  required to consider, among other things, the adequacy of the proposed  IRB in relation to the research activities of the entity that submitted the  assurance.
section_title: Results of Investigation
subsections: section_title: Establishing an IRB
paragraphs: We succeeded in getting a real company to send a research protocol and  related materials to our bogus IRB for its review. As mentioned above, we  created a Web site for our bogus IRB that resembled those of actual IRBs,  and then advertised the services of our bogus IRB online and in  newspapers to attempt to persuade legitimate medical researchers to send  protocols to us. In our advertisements, we stated that we were “HHS  approved,” in reference to our bogus IRB’s registration with HHS. We also  sought to make our IRB look as attractive as possible by emphasizing the  speed of our review process (“Fast Approval!”) and flexibility to customer  needs. The company that sent materials to us was seeking our bogus IRB’s  approval to add one of the company’s clinics as a new test site for ongoing  human trials involving invasive surgery. Our bogus IRB could have  authorized human subjects testing to begin at this new test site—even  though it was a fictitious IRB, with no medical research expertise  whatsoever. Moreover, because this transaction involved a company  conducting private (i.e., not federally funded) research, and did not  involve any FDA-regulated products, our bogus IRB could have approved  the research to begin without needing to register with any federal agency.  We also received inquiries from five other real companies, which  expressed interest in our bogus IRB’s services. However, none of these  five companies submitted any materials for us to review.
All IRBs that review federally funded human subjects research are  required to be registered with HHS. After we registered our bogus IRB  with HHS, HHS provided us with a registration number and listed our  bogus IRB in its online directory of registered IRBs that review federally  funded research. Our only communication with HHS as part of registering  our IRB was through an online registration form, with no human  interaction. The IRB registration process is meant to collect data that HHS  uses during the subsequent assurance approval process. As such, HHS is  not required to verify the information it receives during the IRB  registration process. However, our investigation of the assurance process,  as described below, shows the importance of IRB registration data as they  relate to HHS’s evaluation of assurance applications. Moreover, if our  bogus IRB had been an actual IRB that did not intend to review federally  funded human subjects research, it would not have been required to  submit any registration information. IRBs that intend to review privately  funded human subjects research are not currently required to register with  HHS or any other federal agency, although recently implemented  regulations will change this as of July 2009.
section_title: HHS’s Federalwide Assurance Process
paragraphs: We found that the process for obtaining HHS approval for an assurance  lacks effective controls. As mentioned above, we formed a fictitious  medical device company with phony company officials and a mailbox for  its business location—where human subjects research would supposedly  be conducted. We then submitted an application to HHS for its approval of  an assurance on behalf of our fictitious medical device company. As part  of the application, we named our bogus IRB as the IRB responsible for  reviewing the research covered by the assurance. HHS approved our  assurance application, provided us with an assurance approval number,  and listed our bogus medical device company in its online directory of  approved assurances. Our only communication with HHS as part of this  application was through an online application form and a faxed signature  to complete the application. We did not have any real-time contact with  HHS, whether by telephone, in person, or through a site visit.
We do not know what verification HHS performed, if any, in its review of  our assurance application. However, if HHS had performed basic  screening of the assurance application, HHS would have found  discrepancies that would have warranted further investigation, such as the  fact that we used only a mailbox as our business location. As mentioned  above, in evaluating an application to determine whether or not to approve  an assurance, HHS is required to consider the adequacy of any IRB  designated on the application, as the IRB will be responsible for  overseeing the research activities of the entity that submitted the  assurance application. By approving our assurance application, HHS  essentially deemed our bogus IRB as adequate to oversee human subjects  research, as conducted by our fictitious medical device company.  Moreover, by obtaining an approved assurance from HHS, our fictitious  medical device company can apply for federal research funding from HHS  or other federal agencies. In addition, we used the assurance approval to  boost the credibility of our fictitious medical device company by posting  our assurance number on the fictitious medical device company’s Web  site.
The IRB that approved our fictitious medical device protocol, as discussed  below, is listed on HHS’s Web site as being involved in more than 70  assurances on behalf of actual medical researchers. Each of these  assurances is a first step for the medical researcher to apply for federal  funding for human subjects research, with this IRB formally designated to  oversee the research.
section_title: IRBs’ Research Protocol Approval Process
paragraphs: We were able to get an actual IRB to approve a fictitious protocol for  human subjects research, which raises concerns that other IRBs may  conduct protocol reviews without exercising due diligence, thereby  exposing research volunteers to significant risk. For this test, we created a  research protocol for a fictitious medical device with no proven test  history and bogus specifications, and sent the protocol to three actual,  independent IRBs under the guise of the medical device company we  created for obtaining an assurance from HHS in our second test, as  mentioned above. Our protocol offered only vague information about  certain aspects of our proposed study and was designed using information  publicly available on the Internet. As mentioned above, our fictitious  device was a post-surgical healing device for women that matched  multiple examples of “significant risk” devices provided in FDA guidance.  In addition, we fabricated additional documents we needed to submit  along with our protocol, such as a CV detailing the educational and  professional experience of a fictitious researcher at our company, and a  bogus medical license for the researcher. We succeeded in getting our  fictitious protocol approved by an IRB, even though we were a bogus  company with falsified credentials and an unproven medical device. If we  had been a real medical device company, we could have begun testing our  “significant risk” experimental device on actual human subjects. We also  could have used our bogus IRB mentioned above to approve our fictitious  protocol. This shows the potential for unethical manipulation in the IRB  system.
The IRB that approved our bogus research protocol (IRB 1) required only  minor edits to our submission materials, and did not verify that the  information contained in our protocol and related materials was correct or  authentic, or even that our medical device company actually existed. For  example, we provided IRB 1 with bogus information that FDA had already  cleared our device for marketing because our device was found to be  substantially equivalent to an existing, legally marketed device. IRB 1 did  not attempt to verify this information even though a quick check of FDA’s  online database would have shown no evidence that FDA had ever cleared  our device. By taking advantage of this lapse, our investigators—who  lacked technical expertise in this subject—bypassed any requirement to  develop a risk assessment for a device that, under normal circumstances,  would be considered “significant risk” according to FDA guidance.  Meeting minutes from IRB 1’s board meeting show that it accepted the  bogus information about FDA clearance of our device as evidence that our  device did not require any further risk assessment. See figure 2 below.
IRB 1 “conditionally approved” our protocol after a full board review, but  requested that we modify our informed consent form for study  participation in order to make the language understandable at a fifth-grade  reading level. We modified our informed consent form as requested by  using medical information found on the Internet, after which the board  members of IRB 1 voted unanimously to approve our fictitious medical  device protocol (see fig. 2 above). IRB 1 approved our fictitious protocol,  thereby authorizing us to begin human testing, after only contacting us by  e-mail or fax, and never by telephone or in person. IRB 1’s board meeting  minutes indicate that it believed our device was “probably very safe,” as  shown in figure 2 above. Although our protocol mentioned fictitious  animal studies that we conducted on our device to ensure its safety, IRB 1  approved our protocol without ever seeing proof of these studies or any  other evidence that our device was reasonably safe for use in human  subjects. On its Web site, IRB 1 advertises the speed of its reviews and  states that it performs a “triple check” for quality. IRB 1 has approved  research protocols for experimental drugs tested by major pharmaceutical  companies.
The remaining two IRBs (IRB 2 and IRB 3) provided feedback on our  protocol that was so extensive we determined we did not have the  technical expertise or resources to gain approval. The extensive nature of  the feedback IRB 2 and IRB 3 provided on our initial submission materials  indicated that they follow a much more thorough review process than   IRB 1, which approved our protocol. For example, IRB 2 noticed that our  fictitious protocol mentioned previous testing of the device performed on  animals, and requested that we provide a copy of the results from the  fictitious animal testing. In addition, IRB 3 requested that we send it a  copy of the diagram that our bogus researcher would use to record  incision lines he made as part of the surgery involved in our study, and  raised a number of questions about the timing and locations involved in  our fictitious testing. The documents and information that IRB 2 and IRB 3  requested would have taken extensive time and research to fabricate, and  demanded a level of technical expertise that we did not possess. IRB 1  approved our protocol without obtaining any of the additional information  requested by IRB 2 and IRB 3. Our contacts with IRB 2 and IRB 3, during  their review of our protocol, were done entirely by e-mail.
We later interviewed representatives from IRB 2 and IRB 3 to obtain  additional details about why they did not approve our protocol.  Representatives from both IRBs expressed concern that our protocol did  not contain adequate information about the safety of our fictitious medical  device. For example, the manager of IRB 2 said that she worried that our  device could cause infection in patients, or possibly even cause patients to  develop sepsis. In addition, a board member from IRB 3, who claimed to  have 15 years of experience reviewing research protocols with this IRB,  stated that our protocol lacked any evidence that our bogus medical  device was actually safe for implantation into a human body. He also said  that IRB 3’s board voted unanimously to reject our bogus protocol. Figure  3, below, shows additional examples of IRB 2’s and IRB 3’s comments on  our fictitious medical device and protocol.
None of the three IRBs questioned us about the authenticity of our bogus  CV and counterfeit medical license. As mentioned above, we fabricated  these documents by using information found online and with  commercially available hardware, software, and materials. Our bogus CV  contained information on our fictitious researcher’s human subjects  research background, which we created by using phony drug and device  names and with information that we accessed on the Internet. Our  counterfeit medical license contained a bogus license number with a  similar format to real license numbers used by the state we claimed our  license was from.
section_title: Briefing with HHS
paragraphs: We briefed HHS officials on the results of our investigation. They stated  that HHS receives around 300 IRB registrations and 300 assurance  applications every month, and that OHRP currently has three employees  who review all registrations and applications. According to HHS officials,  the department does not review IRB registrations or assurance  applications to assess whether the information submitted is factual. HHS  officials said that the department reviews assurance applications to ensure  that applicants have submitted all of the necessary information and meet  minimum standards. Moreover, although HHS is required by law to  consider the adequacy of IRBs listed on assurance applications when  reviewing applications, the director of OHRP stated that his office would  require more staff to do so. However, HHS officials added that they would  not consider additional evaluation of IRB registrations or assurance  applications to be worthwhile even if the office had increased resources.
HHS officials stated that the assurance process is not a meaningful  protection against unethical manipulation. They stated their belief that  anyone submitting false or misleading information as part of the assurance  application process would likely be detected during the subsequent  process of applying for federal funding for human subjects research.  However, our work shows that an unethical company could leverage an  HHS assurance for purposes unrelated to the federal funding application  process. For example, representatives from one of the IRBs that rejected  our protocol stated that the HHS assurance number listed on our bogus  medical device company’s Web site gave our company credibility because  it meant that HHS had recognized our company. When we discussed this  with HHS, the director of OHRP acknowledged that an HHS-approved  assurance is meaningful in this regard.
Mr. Chairman, this concludes our statement. We would be pleased to  answer any questions that you or other members of the subcommittee may  have at this time.
section_title: Contacts and Acknowledgments
paragraphs: For further information about this testimony, please contact Gregory D.  Kutz at (202) 512-6722 or kutzg@gao.gov. Contacts points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this testimony. GAO staff who made major contributions to this  testimony include Matthew D. Harris, Assistant Director; Matthew Valenta,  Assistant Director; Timothy Persons, Chief Scientist; Christopher W.  Backley; Ryan Geach; Ken Hill; Jason Kelly; Barbara Lewis; Andrew  McIntosh; Sandra Moore; James Murphy; and Seong B. Park.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.