VA Research: Protections for Human Subjects Need to Be Strengthened
Published: 2000-09-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed rights and welfare of veterans who volunteer to participate in research at Department of Veterans Affairs (VA) and the effectiveness of its human subject protection system, focusing on: (1) VA's implementation of human subject protections; (2) whether weaknesses exist in VA's system for protecting human subjects; and (3) VA's actions to improve human subject protections at those sites affected by sanctions applied by regulatory agencies and throughout VA's health care system.
section_title: What GAO Found
paragraphs: GAO noted that: (1) VA has adopted a system of protections for human research subjects, but GAO found substantial problems with its implementation of these protections; (2) medical centers GAO visited did not comply with all regulations to protect the rights and welfare of research participants; (3) among problems GAO observed were failures to provide adequate information to subjects before they participated in research, inadequate reviews of proposed and ongoing research, insufficient staff and space for review boards, and incomplete documentation of review board activities; (4) GAO found relatively few problems at some sites that had stronger systems to protect human subjects, but GAO observed multiple problems at other sites; (5) although the results of GAO's visits to medical centers cannot be projected to VA as a whole, the extent of the problems GAO found strongly indicates that human subject protections at VA need to be strengthened; (6) three specific weaknesses have compromised VA's ability to protect human subjects in research; (7) VA headquarters has not provided medical center research staff with adequate guidance about human subject protections and thus has not ensured that research staff have all the information they need to protect the rights and welfare of human subjects; (8) insufficient monitoring and oversight of local human subject protections have permitted noncompliance with regulations to go undetected and uncorrected; (9) VA has not ensured that funds needed for human subject protections are allocated for that purpose at the medical centers, with officials at some medical centers reporting that they did not have sufficient resources to accomplish their mandated responsibilities; (10) to VA's credit, substantial corrective actions have been implemented at three medical centers in response to sanctions by regulatory agencies taken against their human research programs, but VA's systemwide efforts at improving protections have been slow to develop; (11) medical centers affected by sanctions have taken numerous steps to improve human subject protections; and (12) VA has, however, been slow to take action to identify any systemwide deficiencies and obtain necessary information about the human subject protection systems at its medical centers.

section_title: Letter
paragraphs: Every year thousands of veterans volunteer to participate in biomedical or behavioral research projects under the auspices of the Department of Veterans Affairs (VA). They do so for different reasons, including to improve the medical conditions of themselves and others, advance science, and serve their country. But research is not without risks. VA studies, like other federally funded research programs, are governed by regulations designed to minimize risks and protect the rights and welfare of research participants. VA must ensure that veterans who agree to become subjects in VA research are given accurate and understandable information about procedures, risks, and benefits so that they can make informed decisions about volunteering. However, veterans who rely on VA for their health care often do so because they are unable to afford private health care and may fear jeopardizing their care if they do not agree to participate in research. VA has a special responsibility, therefore, to ensure that veterans are not unduly influenced in their decision to participate in research projects and that their rights and welfare are protected if they volunteer to participate.
Concerns about VA’s protection of its human research subjects came to national attention in March 1999. At that time, all human research was suspended at the West Los Angeles VA Medical Center after officials there failed to correct long-standing problems with its system for protecting human subjects. Concerns about human subject protections have not been limited to VA. Public attention has recently been focused on concerns about the safety of research subjects in other public and private research institutions. Department of Health and Human Services (HHS) investigations of violations of human subject protections have led to the suspension of research activities at several universities and hospitals. Furthermore, the HHS Office of Inspector General has raised concerns about the adequacy of current human subject protections nationwide in a series of reports. These reports note that protections for research subjects are threatened by recent changes in the research environment that have included a heightened industry role in sponsoring research, proliferation of multicenter trials, increased number and complexity of research proposals, increased demand by patients to participate in clinical trials, and changes in health care delivery systems affecting research programs.
Since the suspension of human research at the West Los Angeles VA Medical Center in March 1999, four additional VA medical centers have been affected by sanctions applied by regulatory agencies against their affiliated universities. Concerned about the rights and welfare of veterans who volunteer to participate in research at VA and the effectiveness of its human subject protection system, you asked us to (1) assess VA’s implementation of human subject protections, (2) identify whether weaknesses exist in VA’s system for protecting human subjects, and (3) assess VA’s actions to improve human subject protections at those sites affected by sanctions applied by regulatory agencies and throughout VA’s health care system.
To address these objectives, we reviewed VA regulations and guidance and interviewed headquarters officials responsible for human subject research activities. We visited eight VA medical centers to review their policies, procedures, and practices for protecting human subjects. We selected these medical centers to reflect major differences in VA research programs, including the number of studies they conduct with human subjects and the institutions responsible for operating the local institutional review board (IRB), the committee tasked with reviewing a research study’s protections for human subjects. At these medical centers, we examined IRB activities to determine whether appropriate protections for human subjects were in place. We also visited three other VA medical centers affected by recent restrictions of their human research programs and reviewed the changes implemented there in response to HHS regulators. We talked with senior research officials at two additional VA medical centers similarly affected by sanctions applied by regulatory agencies. Because research programs at five of the VA facilities we visited involved university affiliates, we also reviewed documents and interviewed officials responsible for human subject protection issues at those institutions. We performed our work from June 1999 through August 2000 in accordance with generally accepted government auditing standards. See app. I for a detailed discussion of our scope and methodology.
subsections: section_title: Background
paragraphs: Conducting research is one of VA’s core missions. VA researchers have been involved in a variety of important advances in medical research, including development of the cardiac pacemaker, kidney transplant technology, prosthetic devices, and drug treatments for high blood pressure and schizophrenia. For fiscal year 2000, Congress appropriated $321 million for VA’s research programs, which support a wide range of human, animal, and basic science studies. VA uses a competitive funding process in which its Office of Research and Development (ORD) allocates about $296 million of these funds to VA researchers, with awards based on scientific merit and potential contribution to knowledge of issues of particular concern to VA. VA allocates most of the remainder to indirect costs of research, which includes support for the human subjects protection system.
Besides the appropriation for research, VA allocates funds from its medical care appropriation to support the research infrastructure at medical centers such as laboratory facilities and investigator salaries. In fiscal year 2000, this allocation amounted to $343 million. VA researchers receive additional grants and contracts from other federal agencies such as the National Institutes of Health (NIH), research foundations, and private industry sponsors, including pharmaceutical companies. In fiscal year 1999, these additional funds amounted to approximately $481 million. Nonprofit research foundations linked to VA medical centers control some of these non-VA research funds.In fiscal year 2000, biomedical or behavioral research involving human subjects is being conducted at about 70 percent of VA medical centers.
VA is responsible for ensuring that all human research it conducts or supports meets the requirements of VA regulations, regardless of whether that research is funded by VA, the subjects are veterans, or the studies are conducted on VA grounds. Responsibility for administration and oversight of the research program has rested primarily with ORD. Recently, VA created the Office of Research Compliance and Assurance (ORCA), which has been charged with advising the Under Secretary for Health on all matters affecting the integrity of research protections for humans and animals, promoting the ethical conduct of research, and investigating allegations of research improprieties.
Some VA research is also subject to oversight by two HHS components. The Food and Drug Administration (FDA) is responsible for protecting the rights of human subjects enrolled in research with products it regulates— drugs, medical devices, biologics, foods, and cosmetics. Research that involves human subjects and is funded by HHS is subject to oversight by its Office for Human Research Protections (OHRP).HHS requires institutions conducting human research with HHS funds to file a document with OHRP that indicates a commitment to comply with federal regulations. This document, called an assurance, may cover a single study (a single project assurance), or it may allow the institution to conduct multiple studies (a multiple project assurance).When an institution files a multiple project assurance with OHRP, all federally funded research involving human subjects at that institution must comply with HHS regulations. Both FDA and OHRP have the authority to monitor those studies conducted under their jurisdiction, and each can take action against investigators, IRBs, or institutions that fail to comply with applicable regulations.
subsections: section_title: Regulations Establish a System to Protect Human Subjects
paragraphs: Research with human subjects conducted at VA facilities is governed by regulations designed to protect their rights and welfare. These regulations establish minimum standards for the conduct and review of research to ensure that research involving human subjects is conducted in accordance with the three ethical principles outlined by the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research.First, the principle of respect for persons requires acknowledgement of individual autonomy, and conversely, the need to protect those with diminished autonomy. In practice, this principle requires that subjects give informed consent to participate in research; that is, they must be given sufficient information about a study including its purpose, procedures, to decide whether to participate. They must also understand this information, and their consent must be voluntary. Second, the principle of beneficence requires that the expected benefits of research to the individual or to society outweigh its anticipated risks. Third, the principle of justice requires fair subject selection procedures, so that both the benefits and the burdens of research are distributed across a number of individuals in a just manner.
In 1981, in response to the National Commission, both HHS and FDA promulgated revised regulations for the protection of human subjects. Seventeen federal departments and agencies, including HHS and VA, have adopted the core of HHS regulations.FDA’s regulations are slightly different from those adopted by HHS and VA.
To safeguard the rights of subjects and promote ethical research, these federal regulations create a system in which the responsibility for the protection of human subjects is assigned to three groups.
Investigators are responsible for conducting their research in accordance with applicable federal regulations and for ensuring that legally effective consent is obtained from each subject or his or her legally authorized representative.
Institutions are responsible for establishing oversight mechanisms for research including establishing local committees known as institutional review boards (IRB), which are responsible for reviewing research proposals before studies are initiated and after they are under way to help ensure that research is conducted in accordance with the three principles described above.
Agencies, including VA, are responsible for ensuring that their IRBs comply with applicable federal regulations and that they have sufficient space and staff to accomplish their obligations.
section_title: VA Human Subject Protection System
paragraphs: VA requires each of its medical centers that engages in research with human subjects to establish its own IRBor secure the services of an IRB at an affiliated university. As of August 2000, approximately 40 percent of the medical centers conducting research with human subjects relied on an IRB at an affiliated university. The IRB sends its recommendations to the VA medical center’s research and development committee, which is responsible for maintaining standards of scientific quality, laboratory safety, and the safety of human and animal subjects. The research and development committee is charged with reviewing each study’s budget; assessing the availability of needed space, personnel, equipment, and supplies; and determining the effect of the planned research on the investigator’s other responsibilities, including the provision of clinical services. The committee can disapprove a study; however, VA regulations prevent the research and development committee (or any other institutional official or body) from overturning an IRB decision to disapprove a study.
A VA investigator who wants to conduct research with human subjects must develop a research plan (called a protocol), supporting documents, and a consent form. The consent form is designed to provide potential subjects with sufficient information about the study, including its procedures, risks, and benefits, to allow the subject to make an informed decision about whether to participate in the study (see fig. 1). The investigator then submits these materials for review. The study is not to be initiated until both the IRB and the research and development committee have approved it, and these committees may insist on changes to the protocol or consent form. Once approval has been given, VA regulations prohibit any unapproved changes to the study’s procedures, unless doing so is absolutely necessary to ensure the safety of a subject. If an investigator wants to alter some aspect of the study, then the IRB must review and approve an amendment or modification to the protocol. In a process known as continuing review, each study is to be re-reviewed at least once per year, and more frequently if the degree of risk warrants it.
section_title: Implementation Of Human Subject Protections Uneven
paragraphs: We found variation across medical centers and their affiliated universities in the implementation of VA regulations and policies involving protections for human subjects. At the eight sites we visited, we found noncompliance with VA regulations in four areas: (1) informed consent; (2) IRB review; (3) IRB membership, staff, and space; and (4) IRB documentation. The problems we identified are similar to problems that OHRP noted in letters to universities and hospitals it has found to be out of compliance with federal regulations. As shown in fig. 2, some sites we visited had more problems than did others.
Of the sites we visited, those with the most extensive violations of VA regulations relied on VA-run IRBs. We identified fewer problems at the IRBs in our sample that were run by universities. In particular, we observed fewer problems with IRB membership, staff, space, and documentation at university-run IRBs than at VA-run IRBs. University-run IRBs were also more likely to conduct thorough and timely continuing reviews than VA-run IRBs. University-run IRBs we visited were not without problems, however. We found that some IRB-approved consent forms at each site omitted required information and some investigators used nonapproved consent forms.
subsections: section_title: Informed Consent
paragraphs: We found problems with the content or use of informed consent forms at all of the medical centers we visited. We found that some informed consent documents that had been approved for use by IRBs provided incomplete or unclear information. In addition, we found some studies in which the investigators used nonapproved consent forms when enrolling subjects. We also found one instance in which research was conducted without consent. Informed consent is a primary ethical requirement of research with human subjects and reflects the principle of respect for persons. The ability of competent subjects to make informed decisions about whether to participate in research and the ability of legally authorized representatives to protect those who are unable to provide consent because they are incapacitated are undermined when IRBs fail to ensure that all required information is included in consent forms or when investigators fail to obtain consent using approved procedures.
subsections: section_title: Information in IRB-Approved Consent Forms
paragraphs: We found that 60 percent of the 138 IRB-approved consent forms that we randomly sampled from lists of active projects provided incomplete or unclear information about required elements of informed consent. (Fig. 3 lists the elements of informed consent required by VA regulations.) Each IRB we visited approved some consent forms that contained incomplete information. For example, IRB-approved consent forms did not indicate that blood would be drawn in a study on the effects of exposure mention possible risks of a biopsy in a study designed to test a treatment describe alternative treatment options in a study comparing two drug treatments for schizophrenia, and indicate who would have access to data obtained during a study on treatment for cirrhosis of the liver.
Of the 84 IRB-approved consent forms we identified that omitted required elements or provided incomplete information, almost half did so for two or more required elements. For example, the consent form for a study of treatments to reduce the recurrence of melanoma did not provide clear information about the duration of the study, nor did it state whom to contact for information about research subjects’ rights. Participants were also told that data would continue to be obtained from their medical records even if they withdrew from the study. Thus, the consent document for this study provided incomplete information about two required elements and appeared to negate the subject’s right to withdraw from the study at any time. Moreover, this consent form might have created undue influence because it inappropriately suggested that the subjects’ own physician endorsed the potential benefits to the subject of participating in this study. Because the participants in this study are randomly assigned to receive either an unproven treatment or no treatment, the physician would have no way of knowing whether participation would benefit the subject.
VA regulations allow an IRB to approve a consent procedure that alters or omits one or more of the required elements of consent if it finds and documents certain conditions. We were unable to find such documentation in the cases we reviewed. Moreover, 37 of the IRB-approved consent forms that omitted or provided incomplete information about a required element were for studies that involved investigational drugs or devices. Thus, both VA and FDA regulations had to be met, and when informed consent is required, FDA regulations do not permit IRBs to alter or omit any required elements of informed consent.
The information that was omitted most frequently—in about 15 percent of forms was the person to whom subjects should direct questions about their rights as research subjects. This information, which is required by regulations, is not included in the standard template for informed consent that VA policy requires investigators to use.
Sites varied in the number of IRB-approved forms that provided incomplete information and the number of incomplete or absent elements in approved forms. The percent of approved consent forms with incomplete information ranged from 78 to 100 percent of our sample at the four sites with the greatest number of these problems. Moreover, forms from these four sites often provided incomplete descriptions of two or more required elements of informed consent. As many as four elements of informed consent were missing or incomplete in IRB-approved forms at these sites. At the two sites where we found the fewest problems, about three-fourths of our sample of approved consent forms were problem-free, and multiple problems in the same form were rare.
In addition to information required by VA regulations, VA policy also requires that informed consent forms indicate that VA will provide free medical treatment for research-related injuries.We found that about 30 percent of the IRB-approved consent documents we reviewed did not include this statement. The absence of this statement varied by site. (These data are not included in fig. 2, which presents noncompliance with VA regulations.) The majority of forms we sampled at two university-run IRBs did not include this information, and one VA-run IRB included it only about half the time. In contrast, the forms at the other university-run IRBs and at the four other VA-run IRBs almost always included it.
The requirement for informed consent was waived for eight of the projects we reviewed, and in each case, our review indicated that the study qualified for the waiver. According to VA regulations, certain categories of research directly or indirectly, to specific individuals do not require informed for example, studies of existing data that cannot be linked, consent or IRB approval. VA regulations also allow for a waiver of informed consent in some research that is not eligible for an exemption from IRB review, provided that the IRB determines that certain conditions apply.
section_title: Investigator Noncompliance With Consent Requirements
paragraphs: Although all the consent forms we obtained from investigators indicated that consent to participate in research had been obtained, we found that investigators did not always obtain consent appropriately. In this review of consent forms, we found 18 studies in which the investigators used nonapproved consent forms when enrolling subjects. We also separately identified one instance in which research was conducted without consent.
We asked investigators at each site to show us signed consent forms from a randomly selected sample of their subjects. We examined 540 such consent forms, all of which had the signature of a subject or a surrogate. In addition to determining that investigators were able to produce these signed consent forms, at four sites we also compared these signed forms with consent forms that IRBs had approved for use in these studies. We found that investigators had used nonapproved consent forms with one or more subjects in 18 of the 73 studies we examined. A total of 33 of 292 subjects had signed nonapproved consent forms. The extent of this problem varied by site. We found that one or more subjects had signed a nonapproved form in 12 to 33 percent of the studies we examined at these four sites. Some of the nonapproved forms that were signed by subjects omitted key information that had been included in the IRB-approved version of the consent form. For example, the nonapproved form that had been signed by all four subjects enrolled in a study on treatments for lymphoma did not mention that the study would involve multiple bone marrow biopsies, the possible risks of those biopsies, or possible side effects of two drugs used information that was included in the IRB-approved consent form.
We identified one instance in which research procedures were performed without consent in the projects in our sample. In this instance, a patient who had not given consent was subjected to an esophageal biopsy for research purposes. This biopsy, which was not reported to the IRB, occurred in conjunction with a biopsy performed for diagnostic purposes in November 1997.
We also found that investigators or their staff had not fully complied with requirements for obtaining consent in three other studies in our sample. In each of these, subjects had consented and steps were implemented to address the problem.
In October 1998, an investigator learned that a subject with schizophrenia did not understand his right to withdraw from research at any time. Upon discovering this, the investigator fired the person who had obtained the subject’s consent, withdrew the subject from the study, and reported the incident to the IRB.
In May 1997, FDA discovered that the consent form signed by subjects in a study of an investigational device to facilitate walking among paraplegics had not included all the necessary information about their participation. The problem was reported to the IRB, the consent form was rewritten, and three previously enrolled subjects were given a revised form and a chance to withdraw from participation.
In July 1997, an investigator realized that he did not have IRB approval for the protocol and consent form that had been used for 73 subjects care providers who had completed a questionnaire to assess decision- including schizophrenics, their family members, and health making. The investigator reported the situation to the IRB, which required that subjects be given a revised approved consent form.
We found one other problem in subject enrollment procedures used by an investigator, although in this case VA regulations were not involved. One subject who was incapacitated as a result of dementia was in a noninvasive study of abdominal aneurysms. Although the subject’s surrogate had provided consent,VA policy establishes protections for incapacitated subjects by prohibiting their enrollment in research that can be conducted with competent subjects. We encountered eight other cases in which surrogates enrolled incapacitated subjects in research, but we were unable to determine whether these cases were in accordance with VA’s policy.
section_title: IRB Review of Research
paragraphs: We found that five of the sites we visited did not implement certain required procedures for IRB review of research. For example, we found that studies at two sites were not reviewed by all necessary IRB members and four IRBs did not ensure timely or thorough continuing review of ongoing research.
subsections: section_title: Initial IRB Review— Required Approval During Convened IRB Meetings
paragraphs: We found that two IRBs did not comply with VA regulations that research must be approved during properly convened meetings, either because meetings were held without a quorum or because the IRB chair improperly approved a high-risk study outside an IRB meeting. With the exception of certain categories of research involving minimal risk to subjects, VA regulations require IRBs to review research at convened meetings attended by a quorum, defined as a majority of members that includes at least one member whose primary concerns are in nonscientific areas. These regulations establish criteria for IRB meeting quorums to ensure that decisions about the protection of human participants in research reflect the consideration of diverse perspectives on research, including the views of scientists and nonscientists with a range of experience and expertise. These protections are undermined when initial review is not conducted in accordance with these requirements.
Four of seven meetings held by one VA-run IRB between January 1998 and August 1999 were held without a quorum. As a result, 17 studies were initiated without legitimate IRB approval, including studies on new drug treatments for unstable coronary symptoms and pneumonia. We examined four to six sets of minutes from IRB meetings held at the other seven IRBs we visited and found that a quorum was present at each.
We found one other instance in which requirements for approval of research at convened IRB meetings were violated. A university-run IRB considered a high-risk drug study for cardiac patients and determined that re-review would be necessary after the investigator addressed several concerns. IRB minutes stated that because the drug company sponsoring the research would have rejected their site if a time deadline were not met, the IRB chair approved the study before the IRB reconsidered it. Although there are circumstances under which an IRB chair can approve a study, in all such cases the research must have been found to pose only minimal risk to subjects. In this instance, the IRB had determined that the study posed a high degree of risk.
On the other hand, our sample also included 16 other studies that met criteria for approval outside a convened IRB meeting. VA regulations allow such a procedure (called expedited review) for studies that pose only minimal risk to subjects and that fall into one of several categories of research. Under expedited review procedures, the IRB chair, or one or more experienced IRB members designated by the chair, are authorized to approve research. For example, IRB approval was expedited for a study on the effects of a weight loss program in which subjects would attend informational sessions about diet and weight loss and have their weight and health monitored using routine, minimal-risk procedures.
section_title: Written Information From Investigators
paragraphs: We found that the IRBs we visited differed in the sufficiency of the written information they asked investigators to provide about human subject protections prior to review. VA regulations identify eight criteria that IRBs must assess before approving research (see fig. 4). Although VA regulations do not specify the information IRBs must review to assess these criteria upon initial review, much of the information can only be provided by investigators. Because offsite study sponsors often prepare the consent forms and protocols used in multisite studies, IRBs must have sufficient information to assess whether the local investigator can properly implement human subject protections.
We found that information in IRB files did not always address all the criteria that must be satisfied for an IRB to approve a study. Of the sites we visited, only one university-run IRB routinely requested detailed information from local investigators about each criterion in its application forms. For example, two IRBs did not routinely ask local investigators any questions about risks or about plans for monitoring the safety of subjects.
Similarly, IRBs differed in the information they had from investigators about special protections for subjects who are likely to be vulnerable to coercion or undue influence. VA regulations require that IRBs ensure that additional safeguards are in place to protect the rights and welfare of such subjects; however, the regulations do not specify the nature of such safeguards. We analyzed project files for 27 studies designed to address issues involving psychiatric conditions that can be associated with a diminished capacity for decision-making psychoses, mood disorders, and organic mental disorders such as dementia. We found that the investigator had included information about additional safeguards in applications for IRB approval in only about half of these studies. For example, we reviewed from two to six files for projects involving potentially vulnerable subjects at each site and found references to additional protections in most of the relevant project files we sampled at four sites. In contrast, at two other locations no such documentation was evident in any of the IRB files we reviewed for projects involving subjects with psychiatric disorders that could affect decision-making.
Some sites have implemented procedures that afford special protections for some such subjects. Examples follow.
Subjects at one medical center who are recruited for psychiatric research and whose mental illness can affect decision-making are typically tested for their comprehension of central consent issues before enrollment in a study.
At another medical center, seriously mentally ill subjects who participate in studies involving a risk that their symptoms might worsen are monitored by a physician who is independent of the research and who is assigned responsibility for deciding whether the subject should remain in the study or be withdrawn.
Alzheimer’s researchers at a third site have established research registries for potential subjects, who were still able to give consent, and their caregivers. By enrolling, subjects agree to allow medical information to be entered into a data bank and to be contacted about future studies. By agreeing to be contacted, however, potential subjects have not consented to participate in future studies. Because these potential subjects are recruited for future studies through registries, the risk of undue influence that occurs when physicians recruit their patients is minimized. Moreover, rules for these registries limit the number of researchers who may contact each person, ensure that potential subjects are recruited only for studies for which they are in fact eligible, and allow registry managers to conduct follow-up surveys to ensure that members of the registries are satisfied with the way researchers treat them.
section_title: Continuing Review
paragraphs: We found that three VA-run IRBs did not meet VA’s regulatory requirement that each study must be re-reviewed at intervals not to exceed 1 year.Regular re-review of a project and associated reports of problems allows an IRB to assess the ratio of risks to benefits on the basis of data obtained since the study began and to ensure that subjects are appropriately informed of those risks and benefits. We examined the dates of continuing review for 73 projects at 6 sites that had received initial approval more than 1 year before our visit. Of these projects, 54 (74 percent) had been reviewed on time within the past year. The median delay for the 19 projects that were not re-reviewed on time was about 1 month. At one VA-run site, only one of the nine projects we reviewed that were more than 1 year old had been re- reviewed on time. At another VA-run site, about half of the necessary continuing reviews from our sample were conducted within 1 year, but delays of up to 14 months occurred in the other half. The three university- run IRBs we visited achieved high rates of timely continuing review.
Four VA-run IRBs we visited reviewed insufficient information when conducting continuing review. OHRP has stated that compliance with regulatory requirements for continuing review entails, at a minimum, IRB review of the study protocol and any amendments; the current consent form; the number of subjects who have been enrolled; and information relevant to risks, including adverse events, unanticipated problems involving risks to the subject or others, withdrawal of subjects from the study, complaints about the study, and a summary of any recent information relevant to risk assessment.
Only half of the IRBs we visited required the investigator to submit the most recent version of the consent document or asked about subjects who have withdrawn (or been withdrawn) from the study. All eight IRBs required reports of the number of subjects who had participated and adverse events.
IRB staff told us that reports of adverse events are difficult for IRBs to handle. Regulations require investigators to report to the IRB unanticipated problems involving risks to subjects, and IRBs must review adverse events reported by all sites where the study is being conducted. The concerns we heard on our site visits were similar to those described in several recent reports on difficulties that IRBs nationwide face when handling large numbers of adverse event reports in the absence of key information necessary for their interpretation.For example, reports of adverse events from drug studies do not indicate whether the subject who experienced the adverse event had received an experimental drug or a different treatment, such as a placebo. Regulatory bodies such as FDA and OHRP and research sponsors such as the National Cancer Institute have recently argued that adverse event reports from studies involving many subjects are often best handled by special committees called data and safety monitoring boards. These boards are typically established by research sponsors and include statisticians and other scientists who analyze data collected during the course of a clinical trial to detect risks to subjects. A few of the IRBs we visited were attempting to develop systems to track adverse events.
Even when a data and safety monitoring board has been established to analyze adverse event reports associated with a study, it is not required to report its findings to IRBs. In VA these boards, referred to as data monitoring boards, analyze only those adverse events reported in multicenter studies funded by VA through a program called Cooperative Studies. If results indicate that a study protocol or consent form must be modified, reports are released by the coordinating center for that cooperative study. It sends such reports to investigators and to the associate chiefs of staff for research and development at participating medical centers, with instructions to share the information with IRBs. Reports are not submitted to IRBs directly. Similarly, VA’s policy manual does not require that reports from data and safety monitoring boards associated with non-VA-funded research be submitted to its IRBs or medical centers. VA’s policy manual also does not require investigators or IRBs to ascertain whether a data and safety monitoring board has been established for studies in which its investigators participate.
section_title: IRB Membership, Staff, and Space
paragraphs: IRBs at the eight facilities we visited met certain membership requirements, but two did not ensure that their members had no potential conflicts of interest. We also found problems involving the number of IRB staff or IRB space at five facilities. VA regulations require that IRBs have sufficient administrative staff and space to review research and preserve the confidentiality of files.
VA regulations for IRB membership include requirements that IRBs have at least five members and must include a scientist, a nonscientist, and at least one person who is not otherwise affiliated with the institution. (Individual members may fulfill more than one criterion.) We checked IRB membership rosters from the eight facilities we visited and found that all met these requirements. In addition, VA regulations state that if the IRB regularly reviews research involving a vulnerable category of subjects, then consideration should be given to including at least one member who has experience working with that group. Each of the eight IRBs we visited included someone from the institution’s psychiatry, psychology, or other mental health department, allowing access to specialized expertise with regard to the potential vulnerabilities of mentally ill subjects.
We also found that each of the university-run IRBs we visited had members who were on staff at the affiliated VA medical center. Inclusion of VA staff helps fulfill VA’s regulatory requirement that IRBs have knowledge of the local research institution, including the scope of research activities, types of subjects likely to be involved, and the size and complexity of the institution. Officials at the medical centers we visited that relied on the IRBs of university affiliates reported that the larger academic community of the university offered advantages for IRB membership, including a broader range of expertise and reduced potential for conflicts of interest because IRB members would be less likely to be research colleagues of investigators. In addition, because all VA investigators at these three medical centers also held faculty appointments at the university, investigators did not need to apply for IRB approval from both the university and VA. Officials at some of the medical centers that operated their own IRBs reported that the advantages of doing so included maintaining greater control over the research review process and the increased likelihood that the IRB would know particular investigators and veteran subjects.
We found that two VA-run IRBs did not ensure that their members had no potential conflicts of interest. VA regulations state that no IRB may have a member participate in an IRB initial or continuing review of any project in which that member has a conflict of interest. Although we found that investigators who were IRB members appropriately abstained or recused themselves from voting on their projects, two IRBs had, as a voting member, the associate chief of staff for research and development for their medical centers. The duties of a VA medical center’s associate chief of staff for research and development include helping local investigators obtain intramural or extramural research funds. As noted by OHRP, such institutional officials thus have a potential conflict of interest in conducting IRB reviews. These two officials told us, however, that they believed their objectivity as IRB members was not compromised by their other responsibilities.
Officials at four of the VA-run IRBs told us that they did not have adequate staff to support IRB operations, as required by VA regulations. IRB administrative staff provide crucial services such as reviewing applications for completeness, corresponding with investigators, and maintaining IRB records. In addition, some administrative staff serve on IRBs as experts on regulatory issues. The VA-run IRBs we visited typically had one or two IRB staff members who often had other responsibilities. For example, at one of these sites, where a single staff person worked part-time for an IRB that reviews 200 to 300 projects annually, the IRB chair reported that IRB activities, such as suggesting revisions to consent forms, were curtailed due to insufficient staff support. In May 2000, VA headquarters distributed preliminary estimates for the number of administrative IRB staff that a medical center should have. This guidance noted that staffing levels would vary with the breadth and complexity of the research program. ORD officials acknowledged that these benchmarks are a first approximation in an effort to identify appropriate staffing levels.
In addition to staff, IRBs must have secure, private areas for the review and discussion of confidential materials. IRBs also need office space for the IRB chair and administrative staff, secure file storage, and computer support. We found that IRB administrative staff at three sites—two of them lacked sufficient space to conduct their work or store all IRB documents. For example, we observed IRB file folders stacked loosely on top of file cabinets and on floors at one of these sites.
section_title: Documentation of IRB Activities, Findings, and Procedures
paragraphs: Six of the eight IRBs we visited did not maintain all the records required by VA regulations. Inadequate documentation does not, in itself, place subjects at risk. However, records of actions, deliberations, and procedures can help identify problems and corrective actions. Thus, documentary failures prevent appropriate monitoring and oversight activities.
We found inadequate documentation in IRB files for about 9 percent of the ongoing projects we reviewed. For example, some files failed to include copies of all correspondence regarding IRB actions between the IRB and investigators, or copies of all approved consent forms. VA regulations require IRBs to retain these documents for at least 3 years after a study is terminated. Required documents were missing from one or more IRB files at five of the eight sites we visited.
VA regulations require each facility to maintain written procedures that it will follow for conducting initial and continuing review, reporting IRB findings and actions to investigators and appropriate officials, and determining when special steps are necessary to monitor ongoing projects. Our review indicated wide differences between facilities in the adequacy of these documents. One VA-run facility has written procedures regarding criteria for exemption from IRB review and for use of expedited review procedures that are not in accordance with VA regulations. In addition, one medical center had been cited by the FDA for failure to have adequate written procedures in June 1999. The center agreed to have them in place by August 1999 but did not do so until December 1999. The written procedures available from three other VA-run IRBs did not include required descriptions of procedures for conducting project review, determining when additional monitoring of projects is necessary, or responding to investigator noncompliance. In contrast, the written procedures of the three university-run IRBs included all required procedures.
We found one instance in which failure to have required written policies resulted in a further violation of VA regulations. Specifically, the previously discussed esophageal biopsy, which was conducted without consent, was not reported to the IRB or OHRP as required. VA regulations require institutions to ensure that “serious or continuing noncompliance”by investigators is reported to the IRB. A similar report must be filed with OHRP if the institution has an HHS-approved assurance, as did the medical center involved. The Associate Chief of Staff for Research and Development told us that he did not report the event to the IRB or OHRP because he followed the procedures for handling scientific misconduct outlined in VA’s policy manual. Nothing in the IRB’s project files for that investigator indicated a finding or report of noncompliance, imposition of any special restrictions or conditions for future research, or suspension or termination of research.
We found that some IRB minutes did not comply with VA regulations, which require the minutes to include a record of actions, the basis for requiring changes in or disapproving research, and a written summary of discussions of controverted issues and their resolution. At each site, we reviewed from four to seven sets of minutes from IRB meetings held from December 1997 through October 1999. IRB actions were almost always clearly recorded in the minutes we examined at each site. Minutes from six facilities routinely included written summaries of discussions and reasons for actions. Two VA-run IRBs, however, rarely included substantive discussions of these matters in their minutes.
Facilities also varied in their compliance with VA regulations about recording votes by IRB members during project review. The regulations state that minutes of IRB meetings must indicate the number of members voting for and against and the number of those abstaining. Two VA-run IRBs typically recorded votes as unanimous, and minutes from one other VA-run IRB recorded some votes as “approved,” without specifying vote totals. Without exact numbers, the presence of a majority of IRB members required during each vote cannot be confirmed. The voting records in minutes from the remaining IRBs we visited were generally in compliance with regulations. However, in one set of minutes from one site, we found that the total number of votes cast for each decision consistently exceeded the number of members listed in attendance.
section_title: Specific Weaknesses Compromise VA’s Protection of Human Subjects
paragraphs: We identified three specific weaknesses in VA’s system for protecting human subjects: not ensuring that research staff have appropriate guidance, insufficient monitoring and oversight activity, and not ensuring that the necessary funds for human subject protections are provided. These weaknesses indicate that human subject protection issues have not historically received adequate attention from VA headquarters.
subsections: section_title: Headquarters Has Not Provided Adequate Guidance
paragraphs: VA headquarters has not provided the guidance necessary to ensure that its medical center staff are adequately informed about requirements for the protection of human research subjects. We found that VA did not develop a systemwide educational program, ensure that each of its facilities had an appropriate training program in place, or provide guidance about training to its facilities. We also found problems with the guidance VA provides about procedures for handling informed consent records. Efforts to protect the rights and welfare of human subjects are undermined when research staff have not been given clear, comprehensive guidance about human subject protections.
VA headquarters officials told us that VA did not have a systemwide educational program devoted to human subject protection issues and that more training is needed. We found that three of the medical centers we visited had no educational program for IRB members, IRB staff, or investigators. From its October 1999 survey of VA field management, VA headquarters research officials learned that 12 of 22 Veterans Integrated Service Networksdid not have an adequate plan for the ongoing education of IRB members, IRB administrative staff, or investigators about the regulatory requirements for protecting human subjects. In particular, medical centers with small research programs identified difficulties in establishing educational programs. Those facilities that had programs often reported that their university affiliates ran the training programs. A need for increased educational guidance from headquarters was one of the most commonly identified issues regarding human subject protections in the survey. OHRP and HHS’s Office of Inspector General have stressed that educational programs are critical to ensuring that IRBs comply with regulations and are able to assess the acceptability of research proposals in light of those regulations and to ensuring that investigators understand their responsibilities to protect human subjects.
On the other hand, two VA-run IRBs and the three university-run IRBs we visited have implemented their own educational programs for both investigators and IRB members and staff, generally without guidance from headquarters. These programs included training new IRB members, devoting a portion of IRB meetings to discussion of issues involving the protection of human subjects, having some IRB members and staff attend national conferences about IRB operations, and instituting a certification program for investigators. Although we did not evaluate the adequacy of these programs, one of these sites, a university affiliate, developed an educational program that has been cited by HHS’s Office of Inspector General as a best practice for training in human subject protection issues.
In addition to finding that VA did not have a systemwide educational program, we found problems with VA guidance for documenting consent to participate in research. VA’s policy manual includes two requirements that go beyond its regulations for the protection of human subjects: (1) the original signed consent form is to be placed in the subject’s medical record and (2) investigators are to use a standard template developed by VA to obtain consent.
A VA official in ORD told us that the purpose of requiring the placement of signed research consent forms in medical records is to ensure that treating professionals are aware of relevant medical information. He acknowledged, however, that consent forms in medical records are not always readily accessible to treatment staff because they may be housed in old volumes of medical records maintained in storage areas. He also noted that medical records personnel at some VA medical centers have discarded consent forms rather than filing them. Our findings confirmed this. We were unable to locate consent forms in 20 percent of 187 medical records we reviewed at 7 of the 8 medical centers we visited. The remaining medical center we visited recently developed a system for scanning signed consent documents into its electronic medical records. However, these consent forms were not located in a part of the electronic record that would be routinely accessed by treating personnel.
Some medical center research staff suggested that placing a synopsis of each study in a prominent place within subjects’ medical records would ensure that treating professionals know about relevant research participation, thus minimizing risks to subjects. We observed such a strategy at the Denver VA Medical Center, where a special flag in each subject’s electronic medical record links the reader to a brief summary of the study and to any investigational drugs involved. VA has not implemented a systemwide procedure for indicating research involvement in electronic medical records.
Another area of concern is VA’s standard template for informed consent. This template includes space for investigators to enter study-specific information and exact language for requirements common to all consent forms. VA’s policy manual requires all VA investigators to use this form. We identified several problems with this template.
The template does not reflect the regulatory requirement that a contact be provided for subjects to call with questions about their rights as research participants.
For studies conducted at both VA and non-VA locations, use of the VA template created problems. In these cases, adherence to VA’s policy requires development and IRB approval of two consent forms—one based on VA’s template and one for the other location. Failure to use an appropriate IRB-approved consent form in these dual-form studies was the reason subjects signed nonapproved forms in 10 of the 33 cases previously discussed.
VA has not provided clear guidance about the role of a witness to the consent process. Under VA regulations, a witness signature is needed only when the elements of informed consent have been presented orally. We found only 1 study in our sample of 146 in which consent was obtained orally. However, we found that 405 of the 540 signed consent forms we examined had been signed by a witness. OHRP guidance indicates that a witness to a subject’s consent to participate in research may be appropriate when aspects of the study create concerns about the enrollment process. In such cases, an independent witness can provide a valuable check on the consent process to certify, for example, that key information was properly conveyed and that subjects were not unduly coerced into participation. On the other hand, such a witness can represent an unnecessary intrusion into a potential subject’s privacy. VA’s consent template includes a line for the signature of a witness, without specifying who may serve as a witness, what the witness is attesting to, or the circumstances under which the witness is needed. Similarly, VA’s policy manual lacks guidance about who should serve as a witness or what that person’s role is.
section_title: VA Has Not Exercised Sufficient Oversight
paragraphs: We found that VA did not have an effective system for monitoring protections of human subjects. Several instances follow.
VA headquarters and affected medical centers were generally unaware of regulatory investigations and impending actions by OHRP or FDA against university-run IRBs until after the regulatory sanctions were applied. VA was unable to ensure that FDA could notify VA of planned inspections and provide copies of post-inspection correspondence because VA was unable to provide FDA with a list of its university-run IRBs until July 2000. VA did not have a complete list of those medical centers that used their own IRBs, relied on a university-run IRB, or were covered by an OHRP assurance until July 2000.
Until OHRP’s regulatory action against the West Los Angeles VA Medical Center, VA was unaware that each of its facilities was required to provide a written assurance that it will comply with all federal regulations regarding the protection of human subjects. Written assurances facilitate proper oversight by ensuring documentation of core agreements between VA headquarters and IRBs. They also can provide evidence of knowledge of the regulations governing human subject protections and demonstrate an institution’s commitment to those protections. When VA subsequently obtained these assurances, it did not require medical centers to submit local written procedures for implementing human subject protections, as the regulations required. Review of written procedures can indicate gaps or errors in required local policies and procedures.
VA headquarters has not provided medical centers with guidance in ensuring access to minutes or other key information when they arrange for the services of a university-run IRB. As a result, one medical center we visited did not have access to the minutes of its university-run IRB, and two medical centers affected by regulatory sanctions against their affiliated universities had not monitored IRB minutes to assess compliance with regulations.
Furthermore, we found that VA headquarters and medical centers we visited did not effectively monitor investigators and their studies. Specifically, only one of the eight medical centers we visited checked whether investigators provided subjects with the correct IRB-approved consent form. That medical center recently began checking one signed consent form from each study as part of its continuing review. In addition, the files of one university-run IRB we visited did not correctly identify which researchers at the VA medical center were responsible for the studies the IRB had approved because the medical center required that department chairs rather than researchers be listed as principal investigators.
section_title: VA Has Not Ensured Necessary Funding for Human Subject Protections
paragraphs: Responsibility for funding human subject protections at medical centers is diffused across several decisionmakers, each of whom may also have competing priorities for the same funds. As a result, no one official is responsible for ensuring that medical center research programs have the resources necessary to support IRB operations and provide training in human subject protections. Although VA has not determined the funding amounts needed for human subject protection activities at the medical centers, research officials at five of the eight medical centers we visited told us that they had insufficient funds to ensure adequate operation of their human subject protection systems.
We found that medical centers typically relied on several sources of funds to support the indirect costs of research, which include human subject protection activities. These sources included VA’s research appropriation, VA’s medical care appropriation, and non-VA research sponsors such as NIH or pharmaceutical companies. Different decisionmakers control the funds potentially available to a medical center from these sources.
The medical center’s associate chief of staff for research and development controls the portion of the research appropriation targeted for the indirect costs of research.
The medical center’s director controls the portion of the medical care appropriation allocated for indirect costs of research.
Funds from non-VA research sponsors are generally held by a medical center’s nonprofit research foundation and are controlled by its board of directors, which has discretion over their use.
As a result, responsibility for ensuring that human subject protections are adequately funded at each medical center is diffused across several decisionmakers. In addition, the decisionmakers at some of the medical centers we visited told us that they did not allocate additional funds for human subject protection activities because they had to consider those needs against the competing priorities of research support and medical care delivery. Headquarters research officials confirmed that these organizational tensions have created a situation in which there is no clear focus of responsibility for funding human subject protection activities at medical centers.
One of the indirect costs of operating an IRB is the time spent by IRB chairs and members meeting their IRB responsibilities. Headquarters research officials told us that providing release time for IRB chairs and members has been a long-standing problem. VA staff at the medical centers we visited conduct their IRB activity as a collateral duty. We were told that the time commitment for members, and particularly for IRB chairs, is significant. Chairs and members spend time reviewing protocols before meetings, corresponding with investigators, attending IRB meetings, and preparing and reviewing documentation. We were told that the lack of release time made it difficult to recruit and retain IRB chairs and members. We found one instance in which a university paid VA to subsidize the costs of covering the emergency room duties of a VA physician who chaired an IRB that VA used. In another instance, a research official at one medical center told us that IRB meetings are held in the evening and that the nonprofit foundation pays IRB members. This arrangement allows members to fulfill their primary VA obligations during the day without the collateral responsibility of serving on the IRB.
Research officials at five of the eight medical centers we visited reported that they had insufficient funds to ensure adequate operation of their human subject protection systems. Of particular concern, officials told us, was that lack of funds prevented hiring and training staff. Officials from some medical centers also told us that their nonprofit research foundations recognized that the level of VA funding for IRB operations was inadequate, and therefore contributed varying amounts of funds for specific local needs, such as training investigators in human subject protections or hiring IRB staff. For example, one nonprofit contributed $25,000 in fiscal year 2000 to support investigator training in human subject protections. Some VA nonprofit foundations and universities are charging private industry sponsors a fee for IRB review of their projects to help support IRB operations. However, headquarters research officials told us that VA has not determined the funding amounts needed for human subject protection activities at the medical centers. They said that such a determination is necessary for planning funding levels and ensuring that human subject protection activities are appropriately funded.
section_title: Local Actions Address Problems Identified by Regulators But Systemwide Focus Slow to Develop
paragraphs: Substantial corrective actions have been implemented at three medical centers in response to sanctions by regulatory agencies against their human research programs. These steps represent progress in meeting the requirements imposed by regulators and VA management, and each of the facilities, despite some difficulties, has resumed human research activities. VA has, however, been slow to identify systemwide deficiencies and to obtain information needed to step up oversight of human subject protection systems at its medical centers. Nonetheless, VA’s recent responses, such as establishment of the Office of Research Compliance and Assurance (ORCA) to monitor human subject protections at individual medical centers and across the system, are promising.
subsections: section_title: Substantial Local Actions Taken to Correct Noncompliance
paragraphs: The three medical centers and their affiliated universities we visited that had actions taken against them by regulators—West Los Angeles, Chicago Westside, and Denver—have made progress in implementing substantial changes to their human subject protection systems. Their written procedures appear to be in compliance with regulations, and their staffing levels seem reasonable for the workload.
These medical centers and their affiliated universities, along with two others, had been affected by serious regulatory sanctions. Regulators found numerous problems at these institutions, including failure to obtain informed consent, failure to conduct adequate and timely continuing review of research, and failure to have adequate written IRB policies and procedures. OHRP deactivated West Los Angeles VA Medical Center’s multiple project assurance with HHS on March 22, 1999. It restricted the assurance held by the University of Illinois at Chicago, which served as the IRB of record for Chicago Westside VA Medical Center on August 27, 1999. On September 13, 1999, FDA suspended certain research projects at a consortium of six Colorado research institutions, including the Denver VA Medical Center. The University of Colorado, the location of the consortium’s IRB, suspended research with human subjects at all six sites in response to a letter from OHRP dated September 22, 1999, which raised concerns about IRB noncompliance with regulations. On December 17, 1999, OHRP restricted the multiple project assurance with Virginia Commonwealth University, which had been the IRB of record for the Richmond VA Medical Center. FDA had issued a warning letter to the university several months earlier about the IRB operations. On January 19, 2000, OHRP restricted the multiple project assurance with the University of Alabama at Birmingham, which was the IRB of record for the Birmingham VA Medical Center.
There were three immediate responses in West Los Angeles, Chicago, and Denver to the sanctions imposed by regulatory agencies: a suspension of enrollment of new subjects in almost all research projects; an assessment of the appropriateness of the continued participation of previously enrolled subjects; and a determination by VA headquarters and affiliated universities of actions needed to improve human subject protection programs at each site. Each medical center or affiliated university that we visited then made extensive changes to its human subject protection system. These changes involved reconstituting IRBs; increasing the number of IRB administrative staff; training IRB members, staff, and investigators in the principles and procedures of human subject protection; creating or extensively revising IRB procedures; increasing working space for IRB operations; creating new databases for tracking protocols through the review process; re-reviewing projects; and resuming research activities. As of February 2000, all projects at the West Los Angeles VA Medical Center had been re-reviewed by an IRB. As of June 2000, all projects for the Chicago Westside VA Medical Center had been submitted to university-run IRBs for re-review, and as of July 2000, all projects had been re-reviewed for the Denver VA Medical Center. The Denver VA Medical Center’s IRB has been informed by OHRP and FDA that as of June 2000, its corrective actions are appropriate. On July 18, 2000, OHRP removed the restriction on the University of Illinois at Chicago stating that the university has developed and implemented an improved system for the protection of human subjects in research and has adequately completed all required actions.
Responses varied across sites, however, because of differing responsibilities for IRB operations and site-specific problems that needed to be addressed. For example, at the West Los Angeles VA Medical Center, which operated its own IRB, VA headquarters and medical center officials made extensive changes in research personnel responsible for human subject protections. From April 1999 to the time of our visit in March 2000, about 50 employees had been rotated through the program with a few assigned full-time to support research and development and IRB operations. The university affiliated with the Chicago Westside VA Medical Center hired a nationally known expert in human subject protections to lead a comprehensive restructuring of its IRB operations.
We identified two issues of concern at the West Los Angeles VA Medical Center. First, VA’s authorization of a resumption of IRB operations at West Los Angeles on April 19, 1999—less than 1 month after OHRP’s deactivation of its multiple project assurance—was premature. At that time, the medical center still lacked approved, written procedures for operation. Such procedures are required by regulations. It also was relying on untrained administrative staff to assist the newly formed IRBs. Furthermore, VA’s investigators had not been trained in human subject protection issues.
Our second issue of concern is that officials at the West Los Angeles VA Medical Center were particularly slow to respond to OHRP’s requirements. In its 1999 letter deactivating the medical center’s multiple project assurance, OHRP noted the medical center’s continued lack of responsiveness to issues raised by OHRP over a 5-year period. For example, in 1994, OHRP required that the medical center establish a data and safety monitoring board to oversee studies involving subjects with severe psychiatric disorders. It took until February 2000 for medical center officials to approve standard operating procedures for the data and safety monitoring board and to hire its staff.In another instance, OHRP cited the medical center in 1995 for a lack of adequate written procedures for human subject protections. However, it took the medical center until February 2000 to develop and approve these procedures. Similarly, in 1995, OHRP strongly recommended that medical center officials develop an ongoing training program for investigators. Medical center officials told us they plan to begin such training in September 2000.
At the Chicago Westside VA Medical Center, we found that, in permitting the continued participation of previously enrolled subjects in some projects, VA and the university-run IRBs did not ensure that continuing review requirements were met for these projects. When we raised this issue with officials during our February 2000 visit, they acknowledged this lack of oversight. They have since required investigators for these projects to submit materials for continuing review.
We found that the Chicago Westside VA Medical Center did not play an active role in assisting its university-run IRBs to improve its human subject protection system. The medical center organizational chart for research and development did not show any linkage with the three university IRBs. The medical center had only one representative among the 18 members of the biomedical IRB and one on the 17-member combined biomedical- behavioral IRB. There were no VA representatives on the third IRB, an IRB that reviewed behavioral studies because, as officials told us, VA conducted few such studies. At the time of our visit to the medical center—over 5 months after the OHRP action—the medical center had done little to improve its communication with the IRBs despite the recommendation to do so made by the VA headquarters site visit team in September 1999. Although one local VA research official participated on a university committee charged with prioritizing studies for re-review and made suggestions to modify the IRB form used by investigators to submit protocols for review, the medical center had not established a mechanism for routine contact with and monitoring of the IRBs. In addition, the medical center was unaware of VA protocols being submitted for IRB review, IRB actions to approve or disapprove continuation of studies, and serious adverse events that could affect veterans who were subjects of research. At the time of our visit, the medical center was unable to provide us with reliable data on which investigators had been trained by the university in human subject protection regulations and issues. Furthermore, as of July 2000, the medical center had not responded to a May 2000 request from the university for comments on their new IRB procedures manual.
In contrast, the Denver VA Medical Center established mechanisms to enhance communication between the research and development program and its three university-run IRBs by having regular meetings and increasing the number of VA personnel on the IRBs. As of June 2000, the chair of one of the university-run IRBs and the co-chair of another were VA employees. Five other VA employees served as members of the IRBs. Medical center personnel were working closely with their counterparts in the university to design a database that would allow VA research officials access to VA project information at the university-run IRBs.
When the IRBs at their affiliated universities faced sanctions by regulatory agencies, officials at the Richmond and Birmingham medical centers chose to establish their own IRBs. They told us they did so to increase their control over the research review process. These officials told us they each created an IRB, developed written procedures, trained IRB members, and resumed their research programs after re-reviewing their projects. In addition, the Birmingham VA Medical Center has trained investigators and IRB staff, and the Richmond VA Medical Center has trained research staff.
section_title: Systemwide Approach Slow to Develop
paragraphs: VA has been slow to recognize and address systemwide deficiencies in its human subject protection activities. Although OHRP identified problems with human subject protections at the West Los Angeles VA Medical Center in 1994, VA did not have a plan to address systemwide concerns involving research until July 1998. VA did not begin to implement systemwide changes until after OHRP took regulatory action against the medical center in March 1999. VA’s initial responses to regulators’ actions affecting the West Los Angeles VA Medical Center and other medical centers were crisis- driven and site-specific. Specifically, headquarters formed teams that conducted site visits to determine actions needed at the affected medical centers. Headquarters monitored corrective actions at the medical centers primarily through an exchange of reports and correspondence.
In July 1998, VA developed a plan to reorganize its field research operations. This plan addressed a variety of research concerns including the involvement of human subjects and the ethical conduct of studies. Only recently, however, has VA headquarters begun to implement systemwide changes to improve its human subject protections. Its steps have included providing information to investigators and research staff, obtaining information about medical centers’ research programs, and making organizational changes to enhance monitoring and oversight of research involving human subjects. These steps have been slowly implemented, but they provide a promising foundation for improvements to protections for human subjects in VA research.
subsections: section_title: Providing Information to Investigators and Local Research Staff
paragraphs: VA headquarters officials have taken several steps to provide information to VA investigators and local research staff about human subject protections. The initial information provided by ORD described issues at affected medical centers. It was not until October 1999 that ORD provided medical centers with specific actions that could be helpful in strengthening their human subject research programs.
Starting with its May 1999 bimonthly conference call with associate chiefs of staff for research and development, ORD began discussing human subject protection issues in light of the March 1999 OHRP action against the West Los Angeles VA Medical Center. Also in May 1999, they began to plan a series of educational programs for investigators, IRB members, research administrators, and medical center directors focused on human subject protection issues. In October 1999, ORD held a nationwide videoconference in which OHRP and VA research officials discussed human subject protection issues and answered questions from VA staff.
Also in October 1999, ORD began to list on its Web site human subject protection information available through OHRP and other organizations and distributed a summary of lessons learned from institutions that had been affected by recent sanctions by regulatory agencies. ORD officials told us they expect to complete a draft of a revised policy manual for VA research by September 2000.
ORCA officials have also implemented initiatives. For example, it began bimonthly teleconference calls in February 2000 with IRB and research officials at medical centers to share information and obtain input on human protection issues. In March 2000, ORCA issued its first newsletter to local research officials. This educational newsletter, planned as a twice a month series, will address informed consent and human subject protection issues. In April 2000, ORCA convened a group of VA research staff and outside experts in human subject protections to identify training courses developed elsewhere that VA could use. The group also plans to develop guidance and strategies for VA to use to train IRB staff, members, and investigators. Beginning in May 2000, ORCA sent the first of three notices to local research programs alerting them of current human subject protection concerns. In June 2000, it began issuing a monthly set of news clippings on human subject protection issues.
In 1999, VA’s National Center for Ethics sponsored a conference on ethics in research and issued related reports including a discussion of principles for researchers’ consideration on the principles guiding the ethical conduct of research involving participants with impaired capacity to consent. VA is participating in national efforts to develop policies and procedures for protecting these participants.
section_title: Obtaining Information About Local Research Programs
paragraphs: VA headquarters officials have acknowledged that they lacked key information about research programs at medical centers. To obtain more accurate and complete information, they have taken several steps. Examples follow.
In October 1998, VA research officials began to develop a new computerized data system to improve the comprehensiveness and accuracy of data about studies involving human subjects at VA medical centers. As of June 2000, development was still under way.
In April 1999, VA asked its medical centers whether they operated their own IRB or relied on the IRB of an affiliated university. VA also asked whether assurances with OHRP were involved. ORCA finished verifying this information in July 2000.
In October 1999, ORD sent a questionnaire to the director of each Veterans Integrated Service Network to assess the adequacy of staffing and support for human subject protections at the medical centers in each network. A lack of adequate resources was one of the three most common problems identified. Sixteen of the 22 networks reported inadequate IRB support, including staff, space, and equipment. Fourteen networks identified education as a priority issue and cited the need for educational opportunities and guidance documents. In May 2000, headquarters sent information to the networks on educational opportunities and made suggestions for the level of administrative staffing of IRBs.
By February 2000, VA had accepted an assurance from each medical center conducting human research that it would comply with regulations for the protection of human subjects.
In April 2000, VA’s Chief Financial Officer reported that VA would implement a system to allow for the explicit accounting of funds from the medical care appropriation that are used by medical centers to support the indirect costs of research.
These steps are necessary to obtain key information about human subject research programs at medical centers. This information will allow headquarters officials to determine the additional steps that may be needed locally or systemwide to ensure compliance with regulations and the protection of human subjects.
section_title: Implementing Organizational Changes to Enhance Monitoring and Oversight
paragraphs: VA is implementing two organizational changes to enhance its monitoring and oversight of human research programs. The Under Secretary for Health announced these changes in April 1999, but as of August 2000, they had not been fully implemented. They are designed to allow routine onsite monitoring of research programs, thereby helping medical centers identify weaknesses and develop strategies to improve compliance with regulations and the protection of human subjects. Although promising in concept, it is too soon to determine whether the initiatives described below will fulfill their objectives.
In April 1999, VA announced the creation of ORCA. VA did not begin staffing this office until it appointed the chief officer in December 1999. VA plans that ORCA will have eight headquarters staff by September 30, 2000, and four regional offices with four staff each by December 31, 2000. As of July 2000, VA had not completed its staffing of the headquarters component and had not filled any regional office positions. Although ORCA’s specific plans for monitoring medical center research activities were still under development in summer 2000, officials told us that they planned to conduct a site visit on a rotating basis to each medical center conducting human research. As of July 2000, ORCA officials told us they had not developed a specific schedule for conducting these visits, but they expect to do so when the regional offices are staffed. ORCA’s headquarters has a budget of $600,000 for fiscal year 2000 and $1.5 million for fiscal year 2001. The regional offices have a budget of $1.9 million for fiscal year 2000 and $2.3 million for fiscal year 2001.
In August 2000, VA awarded a $5.8 million, 5-year contract for external accreditation of its IRBs. This contract requires the contractor to conduct a site visit every 3 years to each medical center conducting human research. The contractor is expected to review IRB performance and to assess its compliance with regulations. VA officials told us that VA expects that the university-run IRBs it uses will grant access to the accreditation team. VA is the first research organization to have an external accreditation of its human research programs.
section_title: Conclusions
paragraphs: VA has not ensured that its medical centers have fully implemented required procedures for the protection of human subjects. Primary responsibility for implementation of these protections lies with local institutions medical centers and their IRBs. Although we cannot generalize from our sample to the universe of VA research institutions, we found sufficient evidence of noncompliance with applicable federal regulations to be concerned. We also found that incomplete access to information about adverse events experienced by research participants made it difficult for IRBs to fulfill their mandate. We found widespread weaknesses in the management of human subject protections that VA had not identified because of its low level of monitoring. VA’s past failure to ensure that its research facilities had the resources, including staff, training, and guidance, needed to accomplish their obligations suggests that headquarters has not given attention or sufficient priority to the protection of human subjects.
Despite a 5-year record of problems at the West Los Angeles VA Medical Center, VA did not begin to implement systemwide improvements until OHRP took regulatory action against the medical center. VA’s initial actions were primarily crisis-driven and site-specific. Generally, appropriate corrective actions have now been implemented at each of the three medical centers we visited that were affected by regulatory sanctions. However, VA’s progress on systemwide improvements to its human subject protection system has been slow. VA only recently began to obtain the information it needs—such as identifying which medical centers use their own IRBs and which rely on university-run IRBs improvements.
Some facilities we visited and projects we reviewed appeared to have reasonably strong protections for the rights and welfare of participants. VA’s recent efforts to improve its human subject protections systemwide and its commitment to developing an effective oversight and monitoring system are important steps toward ensuring that all VA facilities meet requirements, but it is too soon to determine how well these initiatives will fulfill their objectives.
VA has a long history of important contributions to medical research, and it could set important precedents in improving human research protections. For example, VA is the first federal agency to take action to externally accredit its IRBs. Whether VA medical centers establish their own IRBs or work with university-run IRBs, VA needs to ensure that the IRBs have adequate resources, and VA must exercise its oversight authority if it is to know what guidance, preventive efforts, or corrective actions are needed.
section_title: Recommendations
paragraphs: To strengthen VA’s protections for human subjects, we recommend that the Acting Secretary of Veterans Affairs direct the Under Secretary for Health to take immediate steps to ensure that VA medical centers, their IRBs— whether operated by VA or not—and VA investigators comply with all applicable regulations for the protection of human subjects by providing research staff with current, comprehensive, and clear guidance regarding protections for the rights and welfare of human research subjects; providing periodic training to investigators, IRB members, and IRB staff about research ethics and standards for protecting human subjects; developing a mechanism for handling adverse event reports to ensure that IRBs have the information they need to safeguard the rights and welfare of human research participants; expediting development of information needed to monitor local protection systems, investigators, and studies and to ensure that oversight activities are implemented; and determining the funding levels needed to support human subject protection activities at medical centers and ensuring an appropriate allocation of funds to support these activities.
section_title: Agency Comments
paragraphs: In written comments (see app. II) on a draft of this report, VA agreed with our findings and recommendations. VA said that initiatives it has already planned and implemented will provide a foundation for a national prototype in effective human subject protections.
Although VA agreed that its implementation of a systematic approach to human subject protections has been slow to develop, it provided clarification regarding statements in the draft report that VA had not focused attention on systemwide weaknesses until after the March 1999 regulatory action at the West Los Angeles VA Medical Center. VA stated that planning for the establishment of regional offices for risk management and research compliance had begun almost 1 year earlier. We have modified the report accordingly.
In concurring with our recommendations to provide research staff with current, comprehensive, and clear guidance and training about human subject protections, VA identified initiatives planned or under way to improve its guidance, disseminate the guidance, and train research staff in its use. These initiatives represent promising efforts. Whether VA’s plans for guidance and training are effective will depend upon implementation details. VA must ensure that its research staff have access to and receive current guidance and training to enable them to meet their obligations to protect the rights and welfare of human research subjects.
VA agreed with our recommendation to improve adverse event reporting and said it has expanded the distribution of reports from its data monitoring boards to include all appropriate IRBs. VA has also indicated its intention to participate in governmentwide efforts to address this matter. These are important first steps in ensuring that IRBs have the information they need to safeguard the rights and welfare of human subjects. However, because the VA monitoring boards analyze only those adverse events reported in VA’s multicenter Cooperative Studies program, further efforts to address reports of adverse events from other studies are necessary.
VA also concurred with our recommendation to improve monitoring and oversight of human subject protection activities and identified several activities it has planned or implemented, such as external accreditation of IRBs and establishment of performance measures related to human subject protections for medical center research officials. Oversight and monitoring are essential if VA is to know whether the procedures at its medical centers and affiliated universities comply with human subject protection regulations. Whether the actions VA plans to take in this area will be sufficient depends on how effectively they are implemented.
Finally, VA concurred with our recommendation to determine the funding levels needed to support human subject protection activities at medical centers and then ensure an appropriate allocation of funds to support these activities. VA’s response notes that it has begun to account for medical center expenditures associated with research support—an important first step toward determining necessary funding levels. However, VA did not discuss how it would ensure that funds are appropriately allocated to human subject protection activities. As we noted, organizational tensions within VA have created a situation in which there is no clear focus of responsibility for funding such activities at medical centers. Until this is addressed, we are concerned that VA cannot ensure that human subject protections will be appropriately funded.
VA officials also provided technical comments, which we incorporated where appropriate.
We are sending this report to the Honorable Hershel W. Gober, Acting Secretary of Veterans Affairs, appropriate congressional committees, and other interested parties. We will also make copies available to others upon request.
Please contact me at (202) 512-7101 if you or your staff have any questions. An additional GAO contact and the names of other staff who made major contributions to this report are listed in app. III.
section_title: Scope and Methodology
paragraphs: Our objectives were to (1) assess the Department of Veterans Affairs’ (VA) implementation of human subject protections, (2) identify whether weaknesses exist in VA’s system for protecting human subjects, and (3) assess VA’s actions to improve human subject protections at those sites affected by sanctions imposed by regulatory agencies and throughout VA’s health care system.
To achieve these objectives, we reviewed VA, Food and Drug Administration (FDA), and Department of Health and Human Services (HHS) regulations and VA policies for the protection of human subjects; interviewed VA research officials; visited selected VA medical centers to assess local implementation of these standards; and visited VA medical centers affected by research restrictions. We also interviewed officials from the Office for Human Research Protections (OHRP) and reviewed HHS guidance. We reviewed records of congressional hearings; reports on human subject protections, including those issued by the HHS Office of Inspector General, the Institute of Medicine, and the National Bioethics Advisory Commission; and the literature on the history of human subject protections.
To assess VA’s implementation of human subject protections, we conducted site visits at eight VA medical centers: Atlanta, Ga.; Baltimore, Md.; Cleveland, Ohio; Dallas, Tex.; Louisville, Ky.; Providence, R.I.; Seattle, Wash.; and Washington, D.C. We selected sites to reflect major differences in VA research programs (see table 1). First, we selected medical centers that differed in the number of studies they conduct with human subjects. Second, we selected medical centers that differed in the institutions responsible for operating the committee tasked with reviewing each study to assess its protections for human subjects board (IRB). Third, we selected facilities that differed in the assurance arrangements they had with OHRP. Some institutions had filed a legally binding commitment to comply with federal regulations called a multiple project assurance with OHRP; other institutions had not. Our results from these eight medical centers cannot be generalized to other sites.
At each site, we interviewed local research personnel, including the associate chief of staff for research and development, the IRB chair, and staff responsible for providing administrative support to the IRB. We attended an IRB meeting at six sites (Atlanta, Cleveland, Dallas, Providence, Seattle, and Washington, D.C). We also reviewed written procedures describing how the IRB and institution implement human subject protections and a sample of four to seven sets of IRB minutes from the last 2 years (December 1997 through October 1999) at each site.
We randomly selected a sample of 15 to 22 projects at each site for detailed analysis. To ensure that our selection included research on potentially vulnerable participants, we oversampled studies designed to provide information about psychiatric conditions that can affect decision-making capacity, such as dementia, schizophrenia, and depression. Up to one- fourth of the studies we sampled at any one site were in this category. We examined IRB records for each project in our sample (146 in all, including 27 psychiatric studies). For the subset of 138 studies that required written consent, we reviewed the most recently approved consent form.To determine whether subjects had signed appropriate consent forms indicating willingness to participate in research and whether those forms were available as required, we examined about 5 signed consent forms maintained in investigators’ files from each of 125 studies. We also tried to obtain about two signed consent forms from each project in paper medical records. This sample included 98 projects. Some medical records could not be made readily available to us. For example, some medical records were at a different location during our visit.
To assess corrective actions at VA medical centers in response to restrictions on their human research programs, we conducted 2-day visits to three facilities where human research was suspended Chicago Westside, Ill.; Denver, Co.; and West Los Angeles, Ca.Our site visit team included an expert in human subject protections under contract to us. For each of these sites, we examined the OHRP and FDA reports associated with the restriction of human research, action plans for resolving identified problems, documents regarding current human subject operations, and the status of the research program and human subject protections at the time of our visits (February 2000 and March 2000). We discussed these matters with medical center officials and officials from IRBs at affiliated universities when they were involved. In addition, we reviewed documents and interviewed officials from two other medical centers Birmingham, Ala., and Richmond, Va. These facilities were also affected when the IRBs of their affiliated universities were cited for noncompliance with federal regulations. Both have now established their own IRBs.
We conducted our work between June 1999 and August 2000 in accordance with generally accepted government auditing standards.
section_title: Comments From the Department of Veterans Affairs
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: Cheryl Brand, Kristen Joan Anderson, Jacquelyn Clinton, Patricia Jones, and Janice Raynor also made key contributions to this report. In addition, Barry Bedrick and Julian Klazkin provided advice on legal issues, and Deborah Edwards provided advice on methodological issues.
section_title: Related GAO Products
paragraphs: MedicalRecordsPrivacy:AccessNeededforHealthResearch,But OversightofPrivacyProtectionsIsLimited(GAO/HEHS-99-55).
MedicalRecordsPrivacy:UsesandOversightofPatientInformationin Research(GAO/T-HEHS-99-70).
ScientificResearch:ContinuedVigilanceCriticaltoProtectingHuman Subjects(GAO/T-HEHS-96-102).
ScientificResearch:ContinuedVigilanceCriticaltoProtectingHuman Subjects(GAO/HEHS-96-72).
subsections: section_title: Ordering Information
paragraphs: The first copy of each GAO report is free. Additional copies of reports are $2 each. A check or money order should be made out to the Superintendent of Documents. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
Ordersbymail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Ordersbyvisiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Ordersbyphone: (202) 512-6000 fax: (202) 512-6061 TDD (202) 512-2537 Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.
section_title: To Report Fraud, Waste, or Abuse in Federal Programs
paragraphs: Web site: http://www.gao.gov/fraudnet/fraudnet.htm e-mail: fraudnet@gao.gov
1-800-424-5454 (automated answering system)