Department of Health and Human Services 

OFFICE OF 

INSPECTOR GENERAL 

Institutional Review Boards: 

Promising Approaches 

JUNE GIBBS BROWN 

Inspector General 

JUNE 1998 

OEI-01-91-00191 
OFFICE OF INSPECTOR GENERAL 

The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, is to 

protect the integrity of the Department of Health and Human Services programs as well as the 

health and welfare of beneficiaries served by them. This statutory mission is carried out through 

a nationwide program of audits, investigations, inspections, sanctions, and fraud alerts. The 

Inspector General informs the Secretary of program and management problems and recommends 

legislative, regulatory, and operational approaches to correct them. 

Office of Evaluation and Inspections 

The Office of Evaluation and Inspections (OEI) is one of several components of the Office of 

Inspector General. It conducts short-term management and program evaluations (called 

inspections) that focus on issues of concern to the Department, the Congress, and the public. The 

inspection reports provide findings and recommendations on the efficiency, vulnerability, and 

effectiveness of departmental programs. 

OEI’s Boston regional office prepared this report under the direction of Mark R. Yessian, Ph.D., 

Regional Inspector General. Principal OEI staff included: 

REGION HEADQUARTERS 

Elizabeth A. Robboy Lisa Foley 

Nathan P. Rouillard 

Laura C. McBride 

Dana M. Miller 

To obtain copies of this report, please call the Boston Regional Office at (617)565- 1050. 

Reports are also available on the World Wide Web at our home page address: 

http://www.dhhs.gov/progorg/oei 
EXECUTIVE SUMMARY 

PURPOSE 

To identify promising approaches that institutional review boards use to enhance their 

effectiveness and efficiency in protecting human subjects. 

BACKGROUND 

Role of Institutional Review Boards 

Institutional review boards (IRBs) play vital roles in protecting human research subjects. 

They review initial research plans to make certain that the plans provide subjects with 

adequate opportunity to provide informed consent and do not expose subjects to 

unreasonable risks. They also conduct continuing review of approved research to ensure 

that human-subject protections remain in force. They carry out their initial and 

continuing review functions in accord with Federal regulations first established in the 

1970s and applicable to all research funded by the U.S. Department of Health and Human 

Services or carried out on products regulated by the Food and Drug Administration. 

In carrying out their responsibilities, IRBs face challenges posed by a changing research 

environment. Advances in genetics research, a proliferation of large-scale trials, and the 

blurring definitions of research and therapy make research increasingly complex to 

review and oversee. Across the country IRBs are inundated with research plans and are 

under enormous pressure to review them quickly. At the same time, many are finding it 

difficult to recruit and maintain members. Despite these challenges, many IRBs have 

developed innovative strategies for reviewing research plans and for providing 

educational outreach. Yet, in this environment there is little time for the sharing of 

promising approaches among the thousands of IRBs in existence. 

Promising Approaches: The Focus of this Report 

This report focuses on promising approaches of IRBs. One of 4 reports we are issuing on 

IRBs, it draws on interviews and group discussions with representatives of about 

75 IRBs; site visits to IRBs in 6 academic health centers where extensive clinical research 

is taking place; reviews of Federal records and pertinent literature; and attendance at IRB 

meetings. 

The promising approaches included here address six key areas of responsibility for IRBs. 

Within each area, we have chosen promising approaches that seem to have potential for 

multiple IRBs. We recognize, however, that what works well in one IRB may not 

necessarily work well in another. 

i OEI-01-97-00191 IRBs: Promising Approaches 
PROMISING APPROACHES 

Managing the Workload 

To reduce workload pressures, many institutions are creating multiple IRBs and/or 

increasing the number of IRB meetings. Some institutions are creating specialty IRBs, 

such as dental, medical or social science IRBs to ensure that board members have 

sufficient expertise to assess research plans. Other institutions maintain general-purpose 

IRBs but stagger their meetings. One IRB staggers its four board meetings so that there is 

an IRB meeting every week. A research investigator submitting a research plan to this 

IRB will have his or her plan reviewed by the first available board. The waiting period is 

generally no longer than one week. 

Providing Educational Outreach to Research Investigators 

To ensure that all research investigators receive minimal training about human-subject 

protections, some institutions require their research investigators to attend mandatory 

training programs. A dean at one institution will not sign off on approved research until 

investigators have attended a 2-day workshop. According to IRB staff, this method 

provides a powerful political message to all investigators about the importance of humansubject education. Another institution requires all of its investigators to complete an online tutorial about human-subject protections. This tutorial, which takes an hour to 

complete, provides a base level of education for a nominal fee to the institution. 

Reviewing Approved Research 

Using a third party to oversee research is an effective technique for IRBs that do not have 

the time to observe the research process themselves. One IRB requires a research 

intermediary to interact with all psychiatric patients involved in research projects. 

Among other things, the research intermediary discusses the consent form with patients 

after the form has been signed to ensure that upon reflection, patients continue to want to 

participate in the research. Every few months, the research intermediary reports to the 

IRB about patients’ concerns and the ways in which the consent process and/or research 

could be made more efficient and less obtrusive for patients. 

Providing Educational Outreach to IRB Members 

Educating IRB members is an important strategy for ensuring that they have adequate 

expertise to assess research plans. One IRB sends its members literature relevant to the 

research plan being discussed. This approach has been particularly beneficial to help 

orient new members to the ethical issues they should consider when reviewing protocols. 

Another IRB devotes a portion of each meeting to education. A benefit of this type of 

education, according to the IRB director, is the ripple effect it can have with other 

researchers in the institution. For example, after a recent meeting where AIDS trials were 

discussed, two of the physician members shared the discussion with their residents. 

ii OEI-01-97-00191 IRBs: Promising Approaches 
Broadening Perspectives in the IRB Review Process 

A strategy for providing thoughtful, fair review is to continually maintain contact with the 

community in which the research takes place. One IRB established an Office of Ethnic 

Diversity in Research to help it identify barriers the community faces when participating 

in research and potential strategies to overcome these barriers. For example, many in the 

community are unaware of what being a research subject entails and they associate the 

hospital with treatment rather than research. Through community outreach, the IRB has 

become sensitive to the need to further simplify consent forms in order to impress upon 

potential subjects that they are participating in research. 

Evaluating IRB Effectiveness and Efficiency 

To improve its operations, one IRB performed a two-pronged self-evaluation. First, it 

convened a group of faculty members to perform a month-long evaluation of its 

operations. Second, it hired an outside monitor to assessresearch protocols for regulatory 

compliance and to interview investigators about their interactions with the IRB and their 

experiences with the review process. Both evaluations were helpful in highlighting areas 

that needed improvement, such as increased resources for the IRB, changes in the 

reporting structure, and more support from the University. The resultant changes have 

also helped to create better relations between researchers and the IRB. 

CONCLUSION 

Despite the challenges posed by the new research environment, IRBs have employed 

many promising approaches to enhance their effectiveness and efficiency. A key 

challenge to the Federal government is to find ways of giving IRBs the flexibility to 

develop innovative approaches while at the same time holding them accountable for 

performance. This is a matter we address in our summary report entitled, Institutional 

Review Boards: A Time For Reform. 

COMMENTS ON THE DRAFT REPORTS 

Within the Department of Health and Human Services, we received comments on our 

four draft reports from the National Institutes of Health, the Food and Drug 

Administration and, jointly, from the Assistant Secretary for Planning and Evaluation and 

the Assistant Secretary for Health. We also solicited and received comments from the 

following external parties: the Applied Research Ethics National Association, the 

American Association of Medical Colleges, the Consortium of Independent Review 

Boards, and Public Citizen’s Health Research Group. We include the detailed text of all 

of these comments and our responses to them in appendix D of our overview report, 

Institutional Review Boards; A Time for Reform (OEI-01-97-00193). In the executive 

summary of that report, we summarize the thrust of these comments and our responses. 

111 OEI-01-97-00191 IRBs: Promising Approaches 
TABLE OF CONTENTS 

PAGE 

EXECUTIVE SUMMARY 

INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..l 

IRBs: THE BASICS . . . . . _. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 

PROMISING APPROACHES 

ManagementoftheWorkload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...4 

Educational Outreach to Research Investigators ................................ 8 

Continuing Review of Approved Research ................................... 12 

Educational Outreach to IRB Members ...................................... 15 

Broadening Perspectives in the IRB Review Process ........................... 17 

Evaluation of IRB Effectiveness and Efficiency .............................. .20 

CONCLUSION ............................................................ ..2 2 

APPENDICES 

A:Endnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-l 

OEI-01-97-00191 IRBs: Promising Approaches 
INTRODUCTION 

PURPOSE 

To identify promising approaches that institutional review boards use to enhance their 

effectiveness and efficiency in protecting human subjects. 

BACKGROUND 

On page three we offer a primer on IRBs: why they were established, what roles they 

perform, how they are organized, where they are located, and how they are overseen by 

the Department of Health and Human Services. 

Establishment of Institutional Review Boards 

In the 1970s in the aftermath of some widely publicized abuses involving human subjects 

in clinical research, the Federal government began establishing a set of requirements 

calling for institutional reviews boards (IRBs) to protect human subjects. The IRB 

system was originally envisioned as a decentralized system in which research was to be 

overseen at the local level. Most of the early IRBs were established in the large teaching 

hospitals and medical centers, where nearly all clinical research was conducted.’ At the 

time that the regulations were established, research studies were typically conducted by a 

single research investigate? with a few subjects. 

Challenges Facing Institutional Review Boards 

The environment in which IRBs operate has changed significantly in the past two 

decades. Managed care, with its emphasis on cost control, is squeezing research support 

at academic health centers and limiting providers’ time for administrative duties such as 

IRB participation. The proportion of research that is commercially sponsored has grown? 

The locus of clinical research has shifted from single-site trials to large multi-site trials 

conducted across the country, even the world.4 Biomedical research is constantly 

expanding into new frontiers such as gene testing and gene therapy. Many of these new 

research directions raise complicated questions about human-subject protections and 

challenge our notions of informed consent. When IRBs were first established, research 

was viewed as burdensome rather than beneficial to human subjects and something to be 

protected from. Now, many people with lethal diseases are demanding access to the most 

advanced treatments, even when these treatments are experimental. 

These changes have had a significant effect on IRBs across the country. Many are 

inundated with research proposals’ and are overwhelmed by adverse-event reports from 

sponsors. Many IRBs are not sufficiently supported by their institutions. A number of 

IRBs are having difficulty recruiting and maintaining members, particularly physician 

1 031-01-97-00191 IRBs: Promising Approaches 
members. At the same time, they are under enormous pressure to review protocols 

quickly. Despite these challenges, many IRBs have developed innovative approaches to 

improve their effectiveness and efficiency. Yet, in this environment there is little time for 

the sharing of promising approaches among the thousands of IRBs in existence! 

This Inquiry and this Report 

In the wake of many changes transforming the world of clinical research, there is renewed 

interest in human-subject protections and the role of the IRB. A 1995 report issued by the 

Institute of Medicine (IOM) recommended a study on lessons learned about effective IRB 

functioning.7 The National Bioethics Advisory Commission (NBAC) is currently 

examining the state of these protections and is pursuing information about IRBs.’ 

This report, which is one of four we are issuing as part of a broad inquiry of IRBs, 

focuses on promising approaches of IRBs? In determining what to characterize as a 

promising approach, we depended on the judgements of IRB members and 

administrators, consultations with IRB experts, and our own judgement of whether the 

approach was significantly different and important to warrant attention. The promising 

approaches included here appear to have potential in numerous IRBs, although we 

recognize that what works well in one IRB may not necessarily work well in another. It is 

important to acknowledge that our highlighting a promising approach at one institution 

does not necessarily mean that other institutions have not developed similar approaches. 

Many of the issues raised in this report are addressed more fully in our parallel reports. 

One report, Institutional Review Boards: Their Role in Reviewing Approved Research, 

finds that IRBs are devoting minimal attention to their continuing review responsibilities 

and identifies key obstacles responsible for that situation. Another, Institutional Review 

Boards: The Emergence of Independent Boards, finds that independent IRBs are 

becoming a significant force and addresses the advantages and disadvantages they 

present. A third, Institutional Review Boards: A Time For Reform, provides an overview 

of IRB functioning and presents recommendations emerging from our inquiry. 

Our overall inquiry draws on a rich variety of sources. These include interviews and 

group discussions with representatives of about 75 IRBs of varying sizes and auspices;” 

government documents and national commission reports produced over the past 25 years; 

articles and books addressing human-subject protections; Federal records on IRBs; 

attendance at IRB meetings; site visits with FDA inspectors; site visits to 6 IRBs based in 

academic health centers” where extensive research is taking place;12 issues raised on an 

e-mail forum by those associated with IRBs. We focused on the IRB system as a whole 

and on the environment in which IRBs function. We did not conduct an audit of their 

operations, nor did we carry out an investigation of specific IRBs or of specific research 

plans reviewed by IRBs. 

We conducted this study in accordance with the Quality Standards for Inspections issued 

by the President’s Council on Integrity and Efficiency. 

2 OEI-01-97-00191 IRBs: Promising Approaches 
INSTITUTIONAL REVIEW BOARDS: THE BASICS 

What Do They Do? 

The responsibilities of IRBs fall into two main categories: initial review and continuing review of research 

involving human subjects. 

Initial Review: IRBs review and approve a research plan before the research is carried out. This review 

encompasses the research protocol, the informed consent document to be signed by subjects, any 

advertisements to be used in recruiting subjects, and other relevant documents. In carrying out this review, 

the boards seek to ensure that any risks subjects may incur are warranted in relation to the anticipated 

benefits, that informed consent documents clearly convey the risks and the true nature of research, that 

advertisements are not misleading, and that the selection of subjects is equitable and justified. IRBs focus 

much attention on the informed consent document because it is the vehicle for providing information to 

potential research subjects. 

Continuing Review: The continuing review process is multifaceted and includes required reviews “at an 

interval appropriate to the degree of risk but not less than once per year.” In addition to this continuing 

review, study amendments and reports of unexpected adverse experiences by subjects are received 

periodically and reviewed to ensure that the risk-benefit ratio of the research has not changed and remains 

acceptable. 

Why Were They Established? 

As public awareness and concern about the treatment of human subjects in research increased, the need for 

additional review mechanisms was evident. These concerns grew from stories of the abuse of subjects during 

the World War II trials at Nuremberg, the promotional distribution of thalidomide resulting in numerous 

children born with birth defects, the administration of cancer cells to chronically ill and senile patients at a 

hospital in New York, and others. A 1966 article by Henry Beecher brought prominent attention to human 

research abuses in medical schools and hospitals citing 22 cases involving highly questionable ethics. The 

formal requirements for the establishment of IRBs were outlined in regulations stemming from the National 

Research Act of 1974 and in FDA regulations issued in 1981. 

Where Are They Located? 

An estimated 3,000-5,000 IRBs can be found across the country. They are most commonly associated with 

hospitals and academic centers. Boards also exist in managed care organizations, government agencies (such 

as the National Institutes of Health, the Centers for Disease Control, and State governments), or as for-profit 

entities that are independent of the institutions in which the research takes place. 

How Are They Organized? 

Federal regulations require that boards have at least five members with varying backgrounds. At least one 

member must have primarily scientific interests, one must have primarily nonscientific interests, and one must 

be otherwise unaffiliated with the institution in which the IRB resides. A quorum, with at least one member 

whose interests are primarily nonscientific present, is needed for voting. 

How Does the Department of Health and Human Services (HHS) Oversee Them? 

Two agencies within HHS share responsibility for IRB oversight: the Office for Protection from Research 

Risks (OPRR) in NIH and the FDA. The OPRR’s main tool for oversight is the assurance document. Any 

institution that intends to conduct HHS-funded research must have an assurance on file with OPRR. The 

assurance is a written statement of an institution’s requirements for its IRB and human-subject protections. 

Institutions consistently conducting multiple HHS-supported studies are eligible for a multiple project 

assurance (MPA) which can be renewed every five years. Institutions with smaller HHS-funded workloads, 

however, use a single project assurance (SPA) for each such project it conducts. The OPRR also conducts a 

small number of site visits. The FDA’s main mechanism for IRB oversight is the inspection process. The 

FDA also inspects research sponsors and scientists (known as research investigators). 

3 OEI-01-97-00191 IRBs: Promising Approaches 
Management of the Workload 

Issue 

IRBs across the country are inundated with research proposals and related documentation. 

The number of proposals being submitted to the IRB has grown significantly. With the 

emergence of large, multi-center trials, the 

number of adverse-event reports submitted to 

the IRB has also risen dramatically. The 

escalating volume of work, combined with the 

increase in adverse-event submissions has 

greatly added to the paperwork burden that 

IRBs face. As the field of clinical trials 

becomes increasingly competitive, IRBs are 

One IRB we visited reported that between 

1986 and 1996 the annual number of fullboard reviews increased from 1450 to 

2659, expedited reviews from 301 to 58 1, 

amendments from 435 to 1011, and 

adverse-event reports from 50 to 200. 

under enormous pressure to review protocols quickly. The challenge for these boards is 

to find mechanisms to review protocols thoroughly and efficiently in a way that does not 

place the thrust of the increased burden on volunteer members’ time. 

Federal Reference Points 

The HHS regulations, which are overseen by the Office of Protection from Research 

Risks (OPRR) in the National Institutes of Health (NIH), require that institutions make 

provisions for meeting space and sufficient staff to support the IRB’s review and record 

keeping duties. I3 The Food and Drug Administration (FDA) regulations have no such 

counterpart. Both sets of Federal regulations require that an IRB be composed of a 

minimum of five members. There is no maximum size.14 However, OPRR cautions that 

while an IRB can have as many members as is necessary to perform its duties effectively, 

care should be taken that the board does not become so large that its management 

becomes cumbersome.‘5 In their written guidance to IRBs, neither FDA nor OPRR go 

beyond staffing and membership requirements to address measures of efficiency, such as 

turnaround time or the ratio of protocols to reviewers. 

Promising Approaches 

Develop an Institutional Infrastructure to Assist Research Investigators 

Support for clinical investigators, which can range from minimal assistance to formal 

protocol management, occurs at the departmental level at many institutions. The 

Massachusetts General Hospital Cancer Center in Boston has developed a comprehensive 

centralized protocol management office to assist researchers with all aspects of the 

clinical trial. Sixteen research associates assist investigators with protocol development, 

submission to appropriate departments, and management. The office also provides 

4 OEI-01-97-00191 IRBs: Promising Approaches 
education for investigators, and includes a quality-control function to ensure regulatory 

compliance. The IRB told us that protocol management by the cancer center has 

dramatically improved IRB efficiency. Officials in the cancer center say it has been vital 

to improving investigator awareness of how to run a good clinical trial. 

In an effort to strengthen the role of clinical research, many institutions are developing 

clinical research programs intended to increase coordination among offices responsible 

for clinical research, provide training for investigators, and strengthen biostatistical 

support and scientific review. It is too early to analyze the returns of these programs. It is 

expected, however, that these programs should help the mission of the IRB through 

ensuring that protocols submitted to the IRB are more complete and have gone through 

appropriate review channels. 

Pre-review Research Protocols 

Many institutions are beginning to initiate methods of pre-review in an effort to ensure 

that protocols reviewed at full-board meetings are as complete as possible. A recent 

survey found that IRBs most commonly reject proposals because of improperly designed 

consent forms (54 percent), poor study design (44 percent) and scientific merit (14 

percent).16 New protocols at Roswell Park Cancer Institute in Buffalo are assigned to two 

members of the Scientific Review Committee who check for missing information, 

scientific originality and significance, and for overall scientific merit.17 If the reviewers 

find problems with a protocol, they contact the investigator for clarification. These 

problems must be resolved before the protocol can be presented to the full board. 

Protocols have improved, as has efficiency, since the Scientific Review Committee was 

established a decade ago. Staff at Roswell Park Cancer Institute believe that such a 

committee would be an asset at many institutions, yet they warn that it would require 

resources, committee member time, and an another layer of review for investigators. 

Pre-review can also be done by IRB members or outside experts. At Albany Medical 

Center in New York, an IRB subcommittee composed of three persons (two primary 

reviewers and a staff member) resolves issues with investigators prior to a full-board 

meeting. This process greatly reduces the meeting length by eliminating fact-finding and 

one-on-one discussions between the investigators and IRB members. Using this system, 

they have reduced the length of the IRB meeting by half, increased IRB member 

attendance, and increased the time spent on human-protection issues. The Toledo 

Hospital in Ohio relies on a statistician to pre-review protocols for methodological rigor 

and to help with the consent form. 

Develop Well-focused Applications 

Many IRBs are revising their protocol application forms in an effort to elicit clearer and 

more complete explanations. Protocols that contain complete information regarding the 

conduct of a clinical trial are most meaningful to IRB reviewers and allow the board to 

function with maximum efficiency. Massachusetts General Hospital in Boston has 

OEI-01-97-00191 IRBs: Promising Approaches 
revised its form so that II 

investigators are required to 

check off information 

detailing when informed 

consent will be obtained, by 

whom, and from whom. 

Children’s Hospital of Boston 

has also successfully revised 

their application form (see 

box). Other institutions use 

techniques such as including 

sample consent forms in the 

application form, providing 

checklists of information that 

must be included with the 

protocol application, and 

creating a single form that can 

accommodate a variety of 

types of research. 

Increase the Number of IRE3s and/or 

Outline of Points to be Covered in the Research Protocol 

Item #2: Specific Aims/Objectives 

Old protocol application: ” In outline form, state clearly the 

objectives of the research.” 

Revised protocol application: ” The objectives should give a 

concise statement of the hypotheses. The study design should 

be capable of answering these hypotheses. In short, the 

question: “What will be learned from the proposed study?” 

should be answered. Objectives should appropriately 

correspond to the phase of the study. For each objective 

listed, the protocol should adequately address how the 

answers are to be obtained. Often questions arise because 

protocols are submitted with objectives that are not 

answerable by the research proposed.” 

From Children’s Hospital of Boston Protocol Application, 

Committee on Clinical Investigation 

the Number of Meetings 

The mounting workload can be too much for one IRB to handle. For example, the 

University of Rochester in New York used to have one IRB to review 1,700 active 

protocols. The institution recently created four boards (two medical, one behavioral, and 

one dental) from the original IRB to divide up the workload and to ensure greater 

reviewer expertise. Another option is to have multiple, general-purpose IRBs and stagger 

the meetings. For example, Washington University in Saint Louis staggers the meetings 

of its four IRBs so there is an IRB meeting every week. Yet another approach is to rely 

on alternative members. At the University of Kentucky in Lexington, each IRB member 

on the medical IRB has an alternate who has a similar medical background. Between the 

members and the alternates, the University of Kentucky is able to hold an IRB meeting 

four times a month. 

There are, however, some concerns with using these approaches. Many of these 

institutions are struggling with issues of how to ensure consistency among the various 

IRBs, how to attract a sufficient number of qualified IRB members and alternates, and 

how to ensure that specialty IRBs do not focus solely on the behavioral or medical aspects 

of a research proposal at the exclusion of other concerns. 

Require Research Investigators to Assess the Importance of Adverse-Event Reports 

IRBs are deluged with adverse-event reports from the multi-center clinical trials they 

oversee. Many of these reports are difficult to interpret because they are sent to IRBs as 

individual action reports and convey no trend information. The University of Nebraska 

6 OEI-01-97-00191 IRBs: Promising Approaches 
Medical Center IRB at Omaha is one of a number of IRBs that require investigators to fill 

out an accompanying adverse-event report form to assist the IRB in performing a risk 

assessment. These forms seek investigators’ advice concerning the relationship of the 

adverse event to the intervention, whether or not a change in a protocol is necessary, and 

whether or not information about the adverse event is germane to consent and/or reconsent. The thinking behind these forms is that investigators are in the best position to 

determine the significance of adverse-event reports. Furthermore, it forces the 

investigators to stay abreast of all adverse-events that they submit to the IRB. 

Employ Technology to Relieve Burden on IRB Staff 

Using an electronic database can minimize clerical work, assist the IRB in keeping track 

of studies, and allow for coordination among multiple IRBs. Although we found no IRBs 

that have the capability to allow investigators to submit protocols electronically, many 

IRBs told us they are moving in that direction. New database technology will allow for 

efficiencies such as on-line adverse-event reporting, electronic interaction with other 

clinical departments, and on-line updates for research investigators on the status of their 

protocols. The hope is that by giving investigators better tools, IRBs will receive better 

protocols thus allowing for more time to be spent at full-board meetings on humansubject protection issues. While many institutions embrace the idea of employing 

technology to relieve the burden on IRB staff, many cite the necessary financial 

investment as a barrier. 

7 OEI-01-97-00191 IRBs: Promising Approaches 
Educational Outreach to Research Investigators 

Issue 

Research investigators and their stafr8 must be well-informed of the IRB’s role and 

significance for the system of human-subject protections to work effectively. However, 

education about human-subject protections is not necessarily part of an investigator’s 

formal education. Investigators often have incomplete knowledge of when and how 

research protocols must be submitted. In some cases, they may not be aware of what 

constitutes adequate informed consent or they may have difficulty in distinguishing 

between clinical care and research. Since the late 1970s a number of national 

commissions and groups have called for a greater emphasis on investigator education.” 

Federal Reference Points 

The OPRR Multiple Project Assurance requires the institution to make a copy of the 

assurance available to investigators and staff. In a “Dear Colleague” letter, the OPRR 

suggests that IRBs make investigators aware of policies regarding continuing review and 

women and minorities in research, but does not specify how this educational outreach 

should occur.2o The FDA Information Sheets for IRBs and Clinical Investigators provide 

no guidance to IRBs on educational outreach to investigators and their staff. 

Promising Approaches 

Lectures, Conferences and Workshops 

At the University of Washington in Seattle, the IRB director conducts 25-30 presentations 

annually to persons involved with human-subjects research. Compared with many of the 

IRBs we interviewed, this represents a high volume of educational outreach. 

Approximately one-third of these presentations are oriented toward faculty members and 

research staff, the remaining two-thirds toward graduate and post-doctoral students. Each 

seminar, which lasts up to one and a half hours, addresses the history of the IRBs, major 

principles of the Federal regulations, and specifics the IRBs look for when reviewing 

protocols. The sheer number of presentations conducted means that most research 

investigators interested in attending a presentation will, most likely, find a convenient 

time to do so. Ensuring this convenience demands a significant commitment from the 

director, because each presentation entails five to six hours of work between the 

preparation, and the presentation itself. 

At the University of Minnesota Academic Health Center, research investigators attended 

a mandatory 2-day course on the responsible conduct of research, a component of which 

8 OEI-01-97-00191 IRBs: Promising Approaches 
is human-subjects protections. The Dean of Medicine will not approve an investigator’s 

protocol unless he or she has fulfilled this requirement. One IRB administrator told us 

that this type of education sends a powerful political message to all investigators. 

The University of Minnesota also employs a full-time education coordinator. Working 

with the various departments and deans to determine the education needs of investigators, 

she assesses,plans, and evaluates appropriate educational programs. A large number of 

investigators attend these educational sessions. As a result of these requirements, the IRB 

has seen a heightened awareness among researchers about the need to submit research 

proposals to the board.2’ The IRB has also noted an improvement in important aspects of 

the submitted protocols, for example, the consent document. 

At many institutions, clinical coordinators handle the daily tasks of research, (e.g., 

obtaining informed consent, maintaining records, etc.) As such, their education is crucial 

to human subjects’ protections. However, in many cases, coordinators have little 

understanding of what human-subject regulations are or why they exist. To address this 

problem, Massachusetts General Hospital (MGH) has recently begun offering a 2-day 

course for coordinators through its Clinical Research Program. Topics include: roles and 

responsibilities of the investigator, sponsor, research staff, and coordinator; U.S. 

regulations and guidelines, research documentation, recruiting subjects, and obtaining 

informed consent. Currently, data does not exist to show the impact of the program, but 

people at MGH are enthusiastic about it and are planning to offer more courses for 

clinical coordinators. 

On-line Tutorials 

The NIH requires completion of an on-line tutorial for research investigators and all other 

individuals involved with human subjects research. The tutorial explains the purpose and 

history of IRBs and practical matters such as when research must be submitted to the 

IRB. This tutorial, which can be completed at an investigator’s convenience, provides a 

base level of knowledge at low cost; the only significant cost associated with the tutorial 

is the initial development fee. Because of this practice, one official states, “ignorance of 

the regulations will never be an excuse [at NIH].” The disk is a “nuts and bolts” approach 

that any IRB could use. It is available on the internet at the NIH home page?2 

Computer-assisted Literature Monitoring 

An IRB’s ability to protect human subjects is reliant on informed and compliant 

researcher investigators who are aware they must submit their protocols to the IRB. The 

University of California at San Diego (UCSD) recently initiated a computer-assisted 

literature monitoring program to detect maverick researchers. Every week the UCSD IRB 

runs a search of new publications containing the heading ‘human’ and an author address 

containing the words ‘Diego’ or ‘Jolla’. Word matching algorithms are used to rank the 

similarity of each citation to the research projects in the IRB database. If a tentative 

9 031-01-97-00191 IRBs: Promising Approaches 
match of a publication to a project is 

made, a letter requesting confirmation of 

the match is sent to the investigator. 

Where no match can be made, a query 

letter is sent to the investigator 

requesting he/she contact the IRB office. 

The program has become so successful 

that investigators are now preempting 

the IRB by sending their abstracts before 

the IRB runs its search. To perform 

computer-assisted literature monitoring, 

an IRB does not necessarily need 

advanced computer capability. Private 

vendors offer low-cost search services; 

alternatively, IRB staff can run manual 

searches. 

Websites and Manuals 

An Efficient Means of Improving 

Awareness and Compliance 

Using computer-assisted literature 

monitoring, the University of California at 

San Diego was able to identify 132 recent 

publications by UCSD researchers. Thirteen 

of these had no record of being submitted to 

the IRB and resulted in counseling of the 

research faculty involved. According to the 

IRB, “this has been a valuable, low cost 

method of improving awareness of and 

compliance with human-subjects regulations 

among faculty members.” 

An IRB’s staff may not have the time to conduct conferences, seminars or other 

interpersonal education. Websites and manuals are an alternative. Once developed, they 

are effective and low cost educational tools. Websites have become a popular method for 

sharing information regarding policy, IRB memberships, meeting dates, deadlines, and 

events. At Baylor University in Texas, all investigators receive a manual from the 

institution that includes topics such as the definition of research, how to write protocols, 

and requirements of the IRB. According to the IRB administrator, the manual has been 

extremely helpful. For example, one anesthesiology resident called her to find out how to 

write a research protocol. She sent him the manual and four days later received back a 

“beautifully-written” research protocol. Responses from other users have also been 

positive. Making websites and manuals available for investigators and their staff also 

frees up time for the IRB staff as investigators are less dependent on them for 

information. 

Use of Standardized Education Language 

The University of Nebraska has developed standardized educational language for the 

review letters it sends out to investigators. They have categorized standardized 

statements into three broad areas: problems with protocols, problems with informed 

consent documents, and general-reminder language. A typical problem might be that an 

investigator has not clearly indicated how he/she intends to assess the subject’s 

understanding of the research. In cases such as these, the IRB will send the investigator a 

letter with suggestions. These suggestions are all kept coded on a computer. At the 

stroke of a key, the administrator can insert the language into a customized letter. The 

use of standardized language can eliminate a page and a half of typing for the 

administrator. The University of Nebraska has been using this method for 15 years. Over 

10 OEI-01-97-00191 IRBs: Promising Approaches 
this time, the chair has been using fewer standardized statements as investigators are 

submitting more complete and well thought out protocols. 

Use of Explicit Investigator Assurances 

A research investigator’s desire to be in compliance with Federal regulations can easily be 

frustrated by unclear explanations of what these requirements are. One institution, for 

example, had difficulties with investigators making modifications to approved research 

without IRE3consent. Part of the problem was a vague statement investigators were 

required to sign verifying they met Federal regulations. The statement failed to explain 

explicitly what was entailed in compliance. To solve this problem, this institution created 

a form that clearly states the investigator will: 

“obtain voluntary and informed consent of subjects, submit to the IRB any 

adverse events, give progress reports, obtain prior approval from the IRE3before 

amending or altering the project or before implementing changes in the approved 

consent form, maintain documentation of IRE3 approval for at least three years 

after the project has been completed, and treat subjects in a manner specified in 

this form.” 

According to the IRE3administrator, this practice has been extremely beneficial in 

educating investigators. A greater number of investigators are now aware that changes to 

an approved protocol must also be reviewed by the IRB. 

11 OEI-01-97-00191 IRBs: Promising Approaches 
Continuing Review of Approved Research 

Issue 

Continuing review of approved research is vital to the protection of human subjects. The 

mounting workloads, new advances in biomedical research, and the growing public 

interest in experimental therapy as a way to access medical care make the continuing 

review function of IRBs increasingly important. Our inquiry and other studies indicate 

that due to heavy workloads and lack of institutional support many IRBs devote only 

minimal time to continuing review and to active oversight of the informed consent 

process.23 The current system of adverse-event reporting contributes to the difficulty 

many IRBs face in adequately overseeing multi-center trials. 

Federal Reference Points 

The Federal regulations outline the minimum requirements for continuing review of 

approved protocols: IRBs must conduct periodic review of approved protocols at intervals 

appropriate to the degree of risk, but not less than once a year; and they have the authority 

to observe, or to have a third party observe, the consent process and the research.24 As 

such, the regulations allow for great flexibility in the quantity, quality, and depth of 

continuing review. In an effort to strengthen the continuing review function, the FDA 

and OPRR have issued a number of guidelines in recent years. In a 1995 guidance, the 

OPRR called for continuing review to be “substantive and meaningful.” 

Promising Approaches 

Involve a Third Party in Overseeing the Consent Process 

Ensuring that subjects are adequately informed about participating in research is central to 

the mission of the IRB. Yet, our inquiry and other studies found that although IRBs have 

the authority to observe, or to have a third party observe, the consent process they rarely 

invoke this authority. IRB reviews of informed consent are limited almost entirely to 

paperwork reviews.25 The following are examples of ways in which several IRBs, 

through the use of a third party, actively oversee the consent process. Many of these are 

in accord with the recommendations of the Advisory Committee for Human Radiation 

Experiments.26 

A research intermediary interacts with all psychiatric research subjects at the University 

of Texas Health Science Center in Houston.27 Because it might be considered obtrusive 

in the subject-researcher relationship, the research intermediary does not directly observe 

the consent process. Instead, she discusses the consent form with a subject after the form 

has been signed to ensure that the subject understood its terms and that upon reflection 

12 OEI-01-97-00191 IRBs: Promising Approaches 
the patient continues to want to 

participate in research. According 

to the IRB, patients who go through 

this process make better research 

subjects because they are sure of 

why they are participating and they 

are clear about their relationship to 

investigators. 

Patient advocates provide additional 

oversight for subjects involved in 

Benefits of Using a Research Intermediary 

The research intermediary at the University of Texas 

Health Science Center in Houston acts as a liaison between 

the subjects and the researchers. Every few months she 

reports to the IRB about patients’ concerns and the ways in 

which the consent process and/or research could be made 

more efficient and less obtrusive for patients. The research 

intermediary also shares with the IRB how the consent 

process really works, which is something that many IRBs 

areunawareof. 

risky and/or complex protocols at the Massachusetts General Hospital (MGH) in Boston. 

Typically they are involved in gene therapy protocols, Phase 1 studies;’ and protocols in 

which decisions about participating in research must be made quickly. Patient advocates 

observe the consent process to ensure that subjects understand that they are voluntarily 

participating in research. Currently, most of the patient advocates are physicians. 

However, MGH is in the process of establishing an Office of Patient Advocates. The 

advocates, most of whom will be trained social workers, will be available 24 hours a day 

to oversee or participate in the consent process. 

Some IRBs have begun to require or recommend counseling of research subjects to 

ensure that they understand the consequences of research. Roswell Park Cancer Institute 

in Buffalo requires genetic counselors for cases in which the implications of genetic 

testing are not fully addressed in the consent form. The University of Washington 

recommends financial counseling for Phase 1 oncology protocols, where study expenses 

are often not covered by the research sponsor or the subject’s insurer. 

Review Recent Copies of Signed Consent Forms 

At the reapproval stage, investigators at the University of Kentucky in Lexington are 

required to submit to the IRB copies of signed consent forms for the two most recent 

subjects. This allows the IRB to check if the consent forms being used are the most 

updated and if they have been properly signed. The best indicator of the efficacy of this 

approach, according to the IRB director, is that the IRB has found problems with several 

investigators’ consent forms. A downside, however, to this approach is that subject 

confidentiality is broken. The IRB members reviewing consent forms for an AIDS trial 

know that those consenting are HIV positive.29 The IRB director believes that this 

practice is a particularly useful tool for large IRBs which are unable to review all consent 

forms for each approved protocol. 

Develop a Comprehensive Continuing Review Form 

A well-focused application can assist an IRB in providing a thorough and efficient initial 

review. Likewise, a well thought-out continuing review form can assist an IRB in 

identifying potential problems or concerns in approved protocols. The University of 

13 OEI-01-97-00191 IRBs: Promising Approaches 
Nebraska Medical Center IRB at Omaha recently revised its continuing review form so 

that it asks more probing questions (see box).30 The revised form requires that the 

investigator, who is most 

familiar with the protocol, 

assist the IRB in assessing 

the recruitment of subjects: 

problems and 

complications, the process 

of consent, and the riskbenefit relationship. As a 

result of using this form, 

the IRB has been able to 

detect, and avert, problems 

at the reapproval stage. 

Establish a “Re-review” 

Mechanism 

The Federal regulations 

require that IRBs review 

approved protocols at 

intervals appropriate to the 

degree of risk but not less 

IRB Application for Continuing Review of Therapeutic Research 

Old continuing review application: 

. Did any subject suffer an unanticipated adverse event or 

injury during the study? Yes No 

New continuing review application: 

. Did any subject suffer an unanticipated adverse event which 

was reported to the IRB since the last IRE3review? If the 

answer is yes, specify the number of reported events and 

describe briefly their nature and significance. 

. Did any subject suffer an unanticipated adverse event which 

has not vet been reported to the IRE% If the answer is yes... 

. Was there any increase in the frequency of serious but 

expected adverse events? If the answer is yes... 

. Since the last IRB review were any serious external adverse 

events reports submitted to the IRB? If the answer is yes... 

From the University of Nebraska Medical Center at Omaha 

than once a year.” Yet, our inquiry and other sources found that IRBs typically carry out 

the continuing review requirement in a hurried manner, generally relying on 

administrative staff and on investigators’ self-assessments.‘* Unless there is cause for a 

thorough review, the IRB generally spends minimal time discussing approved protocols. 

To enhance the continuing review function, several institutions have established a “rereview” mechanism. Children’s Hospital of Boston requires that on the third anniversary 

of a protocol’s initial approval investigators submit the protocol to the IRB as an initial 

protocol. This system forces investigators to revise the protocol, where necessary, and to 

compile all amendments and adverse events into one document. This approach can help 

IRB members stay abreast of rapidly changing protocols. The re-review system is not a 

substitute for meaningful continuing review, but it helps to ensure that long-term 

protocols receive scrutiny at regular intervals. 

14 OEI-01-97-00191 IRBs: Promising Approaches 
Educational Outreach to IRB Members 

Issue 

With the mounting workload pressure, few IRBs have the time or resources for 

educational outreach to IRB members. Most IRBs provide minimal orientation for new 

members, usually covering just the Federal regulations and IRB procedures. Yet given 

the growing complexity of protocols, ongoing training is increasingly critical for both the 

physician and lay member. New areas of research, such as xenotransplantation and gene 

therapy, raise complicated ethical questions that may not be immediately apparent to 

many reviewers. Several IRBs told us that when they do offer ongoing member training, 

few physicians have the time to attend. 

Federal Reference Points 

The OPRR guidebook states that IRB members and others charged with the responsibility 

of reviewing and approving research should receive detailed training in the regulations, 

guidelines, and policies applicable to human-subject research.33 The OPRR and FDA 

sponsor a number of conferences for IRB members and administrators.34 The OPRR 

guidebook explicitly encourages IRBs to support their members in attending workshops 

focused on IRB functioning.” The FDA Information Sheets for IRBs and Clinical 

Investigators provide no guidance on educational outreach to IRB members. 

Promising Approaches 

Provide Protocol Specific Education 

Staff at the University of Kentucky in Lexington believe that an IRB gets the best results 

by tying educational outreach to the specific protocols being reviewed. When complex 

protocols come up for review--for example, those involving genetic testing, AIDS, or 

children--staff members copy relevant pages of the OPRR guidebook and send them to 

members. According to the IRB director, this approach is particularly beneficial to help 

orient new members to the ethical issues they should consider when reviewing protocols. 

A downside, however, of this type of education is the requisite resources and staff. For 

this approach to work, staff need to pre-screen protocols, determine relevant literature, 

photocopy and distribute the necessary materials. 

Independent Review Consulting in California routinely includes several articles in each 

members’ agenda packet. Designed to stimulate thought and discussion on a broad range 

of topics, the articles are culled from both the lay and medical press. Topics have 

included health care management, medical devices, and the inclusion of women and 

minorities in clinical trials, as well as consent and protocol issues. According to the IRB 

15 OEI-01-97-00191 IRBs: Promising Approaches 
director, this approach has helped facilitate openness among members and raise 

awareness. 

Use Portion of IRB Meeting to Educate Members 

At the University of Texas Health Science Center in Houston, the first ten minutes of 

each IRB meeting is devoted to the continuing education of IRB members. Recent topics 

have included problems in the designs of AIDS studies, statistical issues in study design, 

and ways of improving the IRB’s efficiency. One benefit of this type of education is the 

ripple effect it can have with other researchers in the institution. For example, after a 

recent meeting where AIDS trials were discussed, two of the physician members shared 

the discussion with their residents. According to the IRB coordinator, the ongoing 

education has been helpful in increasing members’ awareness of human-subject 

protections and in maintaining their tenure on and interest in the IRB. 

Host Annual Trainings for IRB Members 

Western Institutional Review Board of Olympia has hosted annual trainings for 14 years. 

Originally the trainings were intended for Western board members only. Now they have 

been opened it up to other IRB members. Last year, about 200 people attended the all day 

training seminar. Speakers at the seminars vary, but they generally address the following 

topics: someone from FDA provides an update on the regulations, someone speaks on the 

ethical aspects of research, an investigator speaks about the impact of IRB decisions on 

research, and someone from a pharmaceutical or device company speaks about how IRB 

rules and regulations affect their business practices. 

According to the IRB director, the annual training sessions have been helpful in making 

members feel secure in their decisions. The sessions give members a chance to ask their 

questions of someone knowledgeable and then allows them to go back to their IRB with 

confidence. The training sessions are also helpful for giving members a well-rounded 

picture of the role of the IRB and how its decisions effect the parties involved. 

16 OEI-01-97-00191 IRBs: Promising Approaches 
Broadening; Perspectives in the IRB Review Process 

Issue 

Scientific knowledge is necessary but not sufficient for the IRB review process. The 

large number of scientists who serve on IRBs ensure ready access to medical and 

methodological knowledge that is of great value to the IRB in assessing the risks and 

benefits of research protocols submitted to it. This specialized knowledge, however, can 

have a downside. Investigators may inadvertently express a tendency to stress the 

benefits rather than the risks of research, overlook just how well subjects understand the 

informed consent document, or give insufficient attention to the diversity of subjects 

recruited. The inclusion of nonscientific members, noninstitutional members, and 

bioethicists on IRBs can be an important counterbalance to these tendencies. These 

members can help ensure the IRB remains sensitive to the needs and concerns of subjects 

and thereby assists the IRB in its goal of protecting human subjects. 

Federal Reference Points 

Of the five IRB members required by Federal regulations, there must be at least one 

whose primary concerns are in nonscientific areas and one who is not otherwise affiliated 

with the institution. In many IRBs, one member fills both roles. The OPRR guidebook 

suggests that the nonaffiliated member should represent the local community, “should not 

be vulnerable to intimidation by the professionals on the IRE3,and their services should be 

fully utilized by the IRB.‘“6 The importance of the nonscientific and nonaffiliated 

members has been heightened by recent guidance concerning the inclusion of women and 

minorities in research.37 

Promising Approaches 

Enable Nonscientific and Noninstitutional Members to Play a Significant Role 

Given the importance of achieving a balance as noted above, we searched widely for 

examples of IRBs with innovative approaches to recruiting, training and supporting their 

nonscientific and noninstitutional members.38 Few came to our attention. 

We found that many IRBs achieved some broadening of perspectives by reaching out 

within their hospital communities to social workers, attorneys, hospital executives, and 

even IRB administrative staff. These individuals, however, are affiliated with their 

research institutions and strictly speaking, are not unbiased “outsiders”. Some of these 

same IRBs indicated they sought more outside members but with little success, noting the 

difficulties of finding persons who were willing to work long uncompensated hours.” 

17 OEI-01-97-00191 IRBs: Promising Approaches 
Through our research, we found several examples of nonscientific, noninstitutional 

members (that is, a single member who satisfies the requirements for having a 

nonaffiliated and a nonscientific perspective on the board) who had come to play an 

important continuing role in helping the IRB broaden its perspective. In one case, the 

IRB contributed significantly to this development by funding the member’s participation 

at an IRB conference. More typically, the members drew on the knowledge and skills 

they brought with them to the IRB and lessons they learned during the course of their IRB 

service. These nonscientific, noninstitutional members told us they brought a very 

different voice to the IRB’s deliberations, particularly when they raised nettlesome issues 

such as confidentiality or the adequacy of researcher’s efforts to reach out to diverse 

populations. As non-researchers, they also tend to assess the risk/benefit ratio more 

critically. They may view research more from the vantage point of a subject. 

Consequently, they are able to understand the vulnerability subjects often feel. 

The other IIU3 members and staff we spoke with also stressed the importance of the 

balancing role played by these pro-active community members. Yet, along with 

expressions of appreciation, they voiced some apprehension that these members can slow 

the deliberation process--not an insignificant consideration given the substantial number 

of research protocols that IRBs must review. 

Include a Bioethicist on the IRB 

Bioethicists, although not a substitute for nonscientific/nonaffiliated members, also help 

to broaden an IRB’s perspective. For many years, bioethicists have participated on the 

National Institutes of Health IRBs. Through their training:’ they have become sensitized 

to subtle yet significant ethical issues pertaining to human-subjects research. Because of 

this, they can assist other RI3 members in identifying and deliberating on ethical issues. 

Indeed, a recent survey highlighted the usefulness of this program!’ 

As one bioethicist we spoke with noted, “It’s not everyone’s job to be up on research 

ethics, but bioethicists are trained to do that.” For example, bioethicists at NIH have 

raised issues about the clarity of the informed consent document and its treatment of the 

risk/benefit ratio, about the distinction of clinical care from research, and about balancing 

attempts to bring more women and minorities into research without undue inducements. 

Some pointed out, however, that the specialized education of bioethicists is both a benefit 

and a drawback. While on the one hand it makes them more efficient in discovering 

subtle ethical issues, it also makes them more likely to focus on obscure ethical concerns 

which are not necessarily a consideration in human-subject protections. One person we 

spoke with at NIH also mentioned the possibility that other IRB members will feel less 

compelled to bring up ethical concerns when the bioethicist is present. 

Provide Community Outreach 

In response to the 1993 NIH guidelines on the inclusion of women and minorities in 

research, the University of Texas Health Science Center IRB in Houston developed and 

18 OEI-01-97-00191 IRBs: Promising Approaches 
established an Office of Ethnic 

Diversity in Research (OEDR). 

The OEDR’s mission is to 

broaden the community’s 

understanding of health-care 

research, to help eliminate 

cultural and practical 

roadblocks to participation in 

research studies, to establish 

and maintain rapport with 

community leaders, and to 

ascertain the areas in which the 

community would like to see 

additional research. About 

twice a month, a member of the 

OEDR makes presentations to 

the community. She is usually 

accompanied by a health 

Sample Questions Asked of the Community 

1. If you did participate in a research activity, what kinds of 

problems did you encounter? 

2. What could the University of Texas Health Science Center 

do to address the problems you have identified? 

3. What type of healthcare research would you like us to 

study that would be beneficial to you, your community, 

and/or your organization? 

4. Please list three things that you could think of that would 

improve how people in your community feel about 

participating in research. 

Questionnaire by the Office of Ethnic Diversity in Research 

at the University of Texas Health Science Center in Houston 

educator, an IRB member, and/or an investigator from the University. Feedback from the 

OEDR’s field visits is reported to the IRB, via the IRB coordinator. The OEDR has 

helped the IRB to identify barriers to participating in research and potential strategies to 

overcome these barriers. According to the IRB coordinator, many in the community are 

unaware of what being a research subject entails and they associate the hospital with 

treatment rather than research. This has made the IRB sensitive to the need to further 

simplify consent forms in order to impress upon potential subjects that they are 

participating in research. 

19 OEI-01-97-00191 IRBs: Promising Approaches 
Evaluation of IRB Efficiency and Effectiveness 

Issue 

The IRB process is too important not to undergo periodic evaluation. Evaluations can 

help an IRB to determine whether it is effectively protecting human subjects, whether it is 

operating efficiently, and whether it has adequate authority.42 The answers to these 

questions are important not only for protecting human subjects, but also for protecting 

institutions and investigators against liability. Many of the IRBs we spoke with expressed 

an interest in improving their effectiveness and efficiency. However, given the time and 

resource constraints they face, few are able to engage in evaluation activities. One IRB 

administrator remarked that, unfortunately, many IRBs and their institutions only become 

proactive about ensuring the integrity of the process once a concern has been brought to 

their attention. 

Federal Reference Points 

Institutions have some obligation, though not extensive, to perform self-assessments. The 

OPRR multiple project assurance requires offices of research administration to perform 

procedural and record-keeping audits not less than once a year.43 The FDA and OPRR do 

not require IRBs to perform evaluations of their effectiveness or efficiency, although both 

encourage such activities. The FDA Information Sheets for IRBs and Clinical 

Investigators include a checklist to help institutions evaluate their own policies and 

procedures; the OPRR guidebook offers some points for IRBs to consider.44 By contrast, 

evaluation requirements are much stronger for research involving animals. All 

Institutional Animal Care and Use Committees are mandated to conduct reviews of 

institutional programs and of animal facilities at least once every six months. 

Promising Approaches 

Perform a Full-scale Evaluation of the Clinical Research Process 

In response to FDA and OPRR concerns about alleged non-compliance with regulations, 

the University of Minnesota commissioned two evaluations. The first evaluation was 

performed by an internal faculty advisory committee composed of several researchers, an 

IRB member, an administrator from Grants and Contracts, and a representative from legal 

counsel. The faculty advisory committee spent a month speaking with IRB 

administrators and chairs, researchers, legal counsel, and external consultants about IRB 

operations and resources. Based on their evaluation, they recommended changes in nine 

strategic areas. 

20 OEI-01-97-00191 IRBs: Promising Approaches 
To further assure itself that the process was functioning well, the University of Minnesota 

hired an outside monitor to assessprotocols that met internal criteria for risk. These 

protocols were assessed for their level of regulatory compliance. In addition, the monitor 

interviewed research investigators about their interactions with the IRB and their 

experiences with the IRB process. The monitor reported all results to the IRB, including 

a set of recommendations. While the IRB had the option of using an internal auditor for 

this task, they felt it was critical to bring in an external monitor. They felt that an external 

monitor provided a greater level of objectivity. According to the IRB, responses to the 

monitor were extremely positive. In fact, resource-permitting, the IRB would like repeat 

this process in the future. 

The two evaluations were helpful in highlighting areas that needed improvement, such as 

increased resources for the IRB, changes in the reporting structure, and more support 

from the University. The resultant changes have helped to create better relations between 

researchers and the IRB. In retrospect, IRB staff felt that if such an evaluation had 

happened earlier it might have prevented difficulties the IRB was experiencing. 

Evaluate a Specific Aspect of IRB Functioning 

Santa Rosa Health Care conducted a year-long evaluation of its continuing review 

procedures to ascertain its compliance with OPRR guidance and to ensure that subjects 

were receiving updated toxicity information in the informed consent document.45 As 

protocols came up for reapproval, the IRB performed an item-by-item comparison of 

toxicities contained in the local consent form to the toxicities contained in the consent 

form supplied by the research base. They also performed a comparison of interim reports 

to the toxicities contained in the local and sample consent forms, a review of drug 

profiles, and a review of product labeling for approved drugs. 

The Santa Rosa Health Care IRB discovered that the majority of protocols had 

incomplete informed consents, that it had been lax in the critical review of initial 

submissions, and that some interim reports failed to provide necessary information so that 

potential risks to study participants could be adequately assessed. As a result, the IRB 

revised the stringency of its initial review of consent forms to include a more thorough 

review of the toxicities listed. The IRB also compiled a reference binder containing a list 

of the more commonly used chemotherapeutic drugs. According to the IRB chair, these 

steps were helpful in strengthening the IRB review process and were also instrumental in 

heightening investigator awareness of the need to submit accurate informed consent 

documents. The evaluation was, however, extremely resource-intensive. The IRB chair 

pointed out that because his IRB had a small workload it was able to put such an 

extensive amount of time and effort into the evaluation--something that larger IRBs might 

not have time for. 

21 OEI-01-97-00191 IRBs: Promising Approaches 
CONCLUSION 

As we noted in the introduction, many IRBs are struggling to keep pace in the rapidly 

changing research environment. Yet, despite the challenges posed by the new research 

environment, a number of IRBs have developed innovative approaches to enhancing their 

effectiveness and efficiency. Many of these approaches are catalogued in this report. 

We did not independently evaluate any approach highlighted in this report. An 

approach’s inclusion does not necessarily mean that it receives our stamp of approval, or 

that it is occurring only at the institution we highlighted. We intend to provide a snapshot 

in time; it is not a definitive study of all IRBs. The report is intended to stimulate 

discussion among the Department, IRBs, and those groups interested in IRB functioning. 

While we have highlighted the work of many IRBs, we may have omitted some 

promising approaches. 

A key challenge to the Federal government is to find ways of giving IRBs the flexibility 

to develop promising approaches while at the same time holding them accountable for 

performance. This a matter we address in our summary report entitled, InstitutionaZ 

Review Boards: A Time For Reform. 

22 OEI-01-97-00191 IRBs: Promising Approaches 
APPENDIX A 

Endnotes 

1. Over time, as clinical research activities spread to other settings, IRBs were established in 

these settings as well. Institutional review boards now exist in a variety of institutions--including 

State governments, the National Institutes of Health, universities, managed care organizations, 

contract-research organizations, and hospitals (of all kinds). Independent IRBs have also arisen 

to review research protocols. 

2. Throughout this report, the terms “research investigator” and “investigator” are used 

interchangeably. They are used to denote persons leading clinical research studies. 

3. The clinical research market is roughly a 4 billion dollar market. It is estimated that about 75 

percent of clinical research is industry-sponsored. (Telephone conversation with Kenneth Getz, 

Director of Center Watch, 26 May 1998.) 

4. While this phenomenon was true for drug studies, the situation was reversed for medical 

device studies. The intraocular lenses studies involved thousands of subjects. Most of the 

studies were conducted at small community hospitals. 

5. Throughout this report, the terms “research plans ” , “research proposals”, and “protocols” are 

used interchangeably. They are used to denote proposals submitted by research investigator to 

the IRB for review. 

6. The primary sources of interaction among IRBs, for those who can afford the fees, are annual 

meetings hosted by Public Responsibility in Medicine and Research (PRIM&R) and Applied 

Research Ethics National Association (ARENA), and FDA- and NIH-sponsored conferences and 

guidance letters. One successful forum for communication has been an electronic mailing list 

sponsored by the Office of Research, Technology and Information at the Medical College of 

Wisconsin. The goal of the forum is to promote discussion of ethical, regulatory, and policy 

concerns with human-subjects research. This has provided an opportunity for over 720 IRBs to 

raise issues they are struggling with and to learn from one another. 

7. Ruth Ellen Bulger, Elizabeth Meyer Bobby, Harvey V Fineberg, editors, Society’s Choices. 

Social and Ethical Decision Making in Biomedicine, (Washington, DC: National Academy Press, 

1995) p. 182. 

8. The National Bioethics Advisory Commission (NBAC) was established by Executive Order 

12975 on October 3, 1995. Section 5 of the executive order establishing NBAC states that as a 

first priority, NBAC shall direct its attention to consideration of protection of the rights and 

welfare of human research subjects. 

9. Some of the promising approaches we address are carried out, not by the IRB, but by other 

parts of the institutions of which the IRB is a part. 

A-l OEI-01-97-00191 IRBs: Promising Approaches 
10. These IRBs are overseeing research at institutions receiving over 1.4 billion dollars of Public 

Health Service (PHS) awards. As of March 1998, these institutions received over 27 percent of 

the PHS dollars awarded extramurally for human subject research. 

11. We use the term “academic health centers” in accord with the following definition offered by 

Blumenthal, et al: “One of 125 institutions in the United States that consist of at least a medical 

school and an owned or closely affiliated clinical facility in which faculty instruct physicians-intraining. These centers classically conduct teaching, patient care and, in many cases, research.” 

(David Blumenthal, Eric G. Campbell, Joel S. Weissman, “The Social Missions of Academic 

Health Centers,” New England Journal of Medicine, Vol. 337,20 November 1997, No. 21, pp. 

1550-53.) 

12. These six institutions alone account for over half a billion dollars of Public Health Service 

(PHS) awards. As of March 1998, these institutions received over 11 percent of the total PHS 

dollars awarded extramurally for human subject research. 

13.45 C.F.R., sec. 46.103(b)(2). 

14. 45 C.F.R., sec. 46.107(a) and 21 C.F.R., sec. 56.107(a). 

15. Office or Protection from Research Risks, Protecting Human Research Subjects: 

Institutional Review Guidebook (U.S. Government Printing Office, 1993), ch. 1, sec. B, p. I-4. 

16. Jeffrey S. Jones et al., “Structure and Practice of Institutional Review Boards in the United 

States,” Academic Emergency Medicine, vol. 3 (August 1996) no. 8, pp. 804809. 

17. This is in addition to determining that the scientific design and methods are adequate for 

achieving the stated objectives. 

18. From here on, references to “researchers” and “investigators” will apply to staff as well. The 

education of research staff is critical as it is often the staff, not the investigators, who conduct the 

daily business of administering research. 

19. These bodies include the National Commission for the Protection of Human Subjects of 

Biomedical and Behavioral Research, the Commission on Research Integrity, Applied Research 

Ethics National Association (ARENA), and the President in his 1997 remarks in apology for the 

study done in Tuskegee. 

20. Dear Colleague letter from the Office of Protection from Research Risks, Subject: 

“Continuing Review -- Institutional and Institutional Review Board Responsibilities”, 

10 January 1995, and Dear Colleague letter from the Office of Protection from Research Risks, 

“Subject: Inclusion of Women and Minorities in Research,” April 25, 1994. 

21. According to the IRBs we interviewed, investigators can be confused about what types of 

research need to be submitted to the IRB. The growth of managed care research, and 

A-2 OEI-01-97-00191 IRBs: Promising Approaches 
particularly quality improvement and outcomes research, is fueling this confusion. 

22. The Internet address is http://helix.nih.gov:SOOl/ohsr/cbti. 

23. A 1996 report by the U.S. General Accounting Office asserted that continuing reviews “are 

typically either superficial or not done at all. According to OPRR officials, IRBs have not always 

understood the requirements for continuing review, and, in other cases, IRB workload demands 

have reduced the quality of this review. In some cases, IRB administrative staff with no 

scientific expertise--not IRB members themselves--review continuing review forms, ensuring 

only that the information has been provided. Heavy workload also necessitates that IRBs rely 

largely on investigators’ self-assessments in conducting continuing reviews” (U.S. General 

Accounting Office, ScientiJic Research: Continued Vigilance Critical to Protecting Human 

Subjects, GAO/HEHS-96-72, March 1996, p. 17-l 8.) Our companion report also raises 

questions about the oversight process. For more information see, U.S. Department of Health and 

Human Services, Office of Inspector General, Institutional Review Boards: Their Role in 

Reviewing Approved Research, OEI-0 l-97-00 190. 

24. 45 C.F.R., sec. 46.109(e) and 21 C.F.R., sec. 56.109(e). 

25. Several recent articles and reports have pointed out the limitations to the informed consent 

process. In 1995, Harold Edgar and David J. Rothman reported that IRBs rarely require 

third-party observers to oversee the consent process or to examine what actually occurs in the 

consent process. (Harold Edgar and David J. Rothman, “The Institutional Review Board and 

Beyond: Future Challenges to the Ethics of Human Experimentation”, The Milbank Quarterly, 

Vol. 73, 1995. No. 4, pp. 489-506.) A 1996 U.S. General Accounting Office study found that 

IRB reviews generally did not involve the direct observation of the research study or the 

informed consent process. (U.S. General Accounting Office, Scientific Research.. Continued 

Vigilance Critical to Protecting Human Subjects, GAO/HEHS-96-72, March 1996, pp. 17-l 8.) 

According to Reiser and Knudson, the reasons for the infrequent inclusion of a third-party 

observer include: (1) that they would interfere with the consent process; (2) that their presence 

would imply that investigators are not trusted to obtain consent; and (3) that they would 

introduce into the research a new problem--that of finding qualified observers to handle oversight 

tasks such as gaining knowledge about the protocol, learning about the subject and the setting of 

research, and, most important, bringing an impartial view to the analysis. (Stanley Joel Reiser 

and Paula Knudson, “Protecting Research Subjects after Consent: The Case for the ‘Research 

Intermediary”‘, IRB, March-April 1993, pp. lo- 11.) 

26. Recommendation 10, part 2 of the Advisory Commission on Human Radiation Experiments 

calls for “Mechanisms for ensuring that the information provided to potential subjects (1) clearly 

distinguishes research from treatment, (2) realistically portrays the likelihood that subjects may 

benefit medically from their participation and the nature of the potential benefit, and (3) clearly 

explains the potential for discomfort and pain that may accompany participation in research.” 

A-3 OEI-01-97-00191 IRBs: Promising Approaches 
Recommendation 10, part 5 calls for “Mechanisms for ensuring that information provided to 

potential subjects clearly identifies the financial implications of deciding to consent to or refuse 

participation in research.” (Advisory Committee on Human Radiation Experiments, FinaZ 

Report, Washington DC: U.S. Government Printing Office, 1995). 

27. For more information about the use of a research intermediary, see Stanley Joe Reiser and 

Paula Knudson “Protecting Research Subjects after Consent: The Case for the ‘Research 

Intermediary”‘, IRB, March-April 1993, pp. lo- 11. 

28. Phase 1 studies are the first of several required clinical trials. Phase 1 trials test the safety of 

a drug or device on volunteer human subjects. 

29. The IRB deals with this by requiring that investigators tell subjects up front that their 

consent forms may be reviewed by the IRB. If this causes a problem for a subject or investigator, 

the name of a subject can be deleted prior to IRB review. 

30. For more information about this continuing review form, see Bruce Gordon and Ernest 

Prentice, “Continuing Review of Research Involving Human Subjects: Approach to the Problem 

and Remaining Areas of Concern”, IRB, Vol. 19, March-April 1997, No. 2, pp. 8-l 1. 

31. 45 C.F.R., sec. 46.109(e) and 21 C.F.R., sec. 56.109 (e). 

32. Based on a 1993 survey of IRBs, Barbara Mishkin commented that “Probably the most 

troublesome problem--and the most frequent--was the absence of substantive review at the time 

of annual renewals. In a number of IRBs, the staff review annual review reports primarily to 

assure that one has been filed and that all spaces on an institutional form have been filled in. In 

some instances, even that review is poorly performed, and staff have submitted to the IRB, and 

the IRB has actually approved, submissions that are clearly incomplete. For example one IRB 

repeatedly re-approved protocols for which the principal investigator (PI) never provided the IND 

(investigational new drug) number assigned by the FDA for a new drug under study. Another 

IRB repeatedly processed forms on which the PI failed to indicate whether the risks to human 

subjects were minimal, moderate, or high.” Barbara Mishkin, “Ethics, Law and Public Policy”, 

Professional Ethics Report (a publication of the American Association for the Advancement of 

Science), Vol. 7, (Spring 1994), No. 2, pp. 4-6. 

33. Office of Protection from Research Risks, Protecting Human Research Subjects. 

Institutional Review Guidebook (U.S. Government Printing Office, 1993), ch. 1, sec. B, p. I-7. 

34. The OPRR sponsors an average of five national educational conferences per year. 

35. Office of Protection from Research Risks, Protecting Human Research Subjects. 

Institutional Review Guidebook (U.S. Government Printing Office, 1993), ch. 1, sec. B, p. I-7. 

36. Office of Protection from Research Risks, Protecting Human Research Subjects. 

Institutional Review Guidebook (U.S. Government Printing Office, 1993), ch. 1, sec. B, p. I-4. 

A-4 OEI-01-97-00191 IRBs: Promising Approaches 
37. The NIH guidelines are located in the Federal Register, (59 Fed. Reg. 14,508, March 28, 

1994.) The institution’s responsibility is expanded upon in the OPRR Dear Colleague letter, 

“Subject: Inclusion of Women and Minorities in Research”, April 25, 1994. The FDA guideline 

can be found in a July 22, 1993 guidance entitled “Guideline for the study and evaluation of 

gender differences in the clinical evaluation of drugs.” 

38. Federal regulations call for at least one IRB member whose concerns are primarily in 

nonscientific areas and one member who is not otherwise affiliated with the institution. Some 

IRBs rely on a single member to satisfy both requirements. 

39. Many IRBs hold their review meetings during business hours. 

40. Many of the persons chosen to serve as bioethicists have a background in science and/or 

ethics. At the NIH, education is provided for new consultants through a one day training session. 

Continuing education and dialogue occurs through group meetings held on alternate months. 

4 1. A 1992 poll of 127 NIH IRB members found 63 percent of those polled believing the 

bioethicist on their board to be very helpful, 32 percent somewhat helpful and 5 percent not 

helpful. In response to the question, “Does the consultant assist you in elucidating or clarifying 

bioethics issues,” 80 percent of the members and 100 percent of the chairpersons responded 

“yes.” (Evan G. DeRenzo and Frederick 0. Bonkovsky, “Bioethics Consultants of the National 

Institutes of Health’s Intramural IRB System: The Continuing Evolution,” IRB, May-June 1993, 

pp. 9-10) 

42. According to Hayes: “If, in fact, IRBs make mistakes from time to time, a thorough, periodic 

evaluation, particularly one done externally, would help to refocus the group’s mission and 

objectives.... The IRB process is simply too important not to include a carefully designed 

evaluation process, which can help boards stay on task and maintain the quality and objectivity of 

their decisions.” (Gregory J. Hayes, Steven C. Hayes, and Thane Dykstra, “A Survey of 

University Institutional Review Boards: Characteristics, Policies, and Procedures”, IRB, vol. 17, 

May-June 1995, no. 3, pp. l-6) 

43. OPRR Multiple Project Assurance, 1997, Part 2, Section II. 

44. The encouragement for self-evaluation can be found in (1) the FDA Information Sheets for 

IRBs and Clinical Investigators, 1995, appendix H, and (2) the Office of Protection from 

Research Risks, Protecting Human Research Subjects: Institutional Review Guidebook (U.S. 

Government Printing Office, 1993), ch. 1, sec. B, pp. I-7 through I-8. 

45. For more information about this evaluation, see Mary S. Adams and Dennis A. Conrad, 

“Annual Review: Observed Deficiencies and Suggested Corrections”, IRB, vol. 18, NovemberDecember 1996, no. 6, pp. l-4. 

A-5 OEI-01-97-00191 IRBs: Promising Approaches