[Memo Report] 
October 23, 1998 
June Gibbs Brown 
Inspector General 
Final Report on Low-Volume Institutional Review Boards (OEI-01-97-00194) 
See Addressees Below 
In this memorandum, I would like to share with you the results of a brief review we conducted 
of low-volume institutional review boards (IRBs).  We did this as a follow-up to our recent 
study of challenges facing IRBs. 
In general, in our June 1998 summary report, we concluded that the IRB system is in jeopardy 
and made numerous recommendations for reforming the system.1  While the findings and 
recommendations of these reports applied to all IRBs, they had particular relevance to academic 
health centers whose primary focus is research. 
While conducting the research for these reports, we became aware that there are some unique 
challenges facing hospital-based IRBs outside of the academic health centers.  Because these 
IRBs tend to oversee considerably fewer research protocols than those at the large research 
centers, we refer to them as low-volume IRBs; in this memorandum, we define low-volume 
IRBs as those that conduct less than 125 initial reviews annually.  This definition most likely 
encompasses a majority of the 3,000-5,000 IRBs in this country.  However, given their smaller 
caseloads, low-volume IRBs probably oversee the minority of protocols subject to IRB review. 
These IRBs typically reside in hospitals that are community-focused and lack a research culture.2 
In brief, our results indicate that low-volume IRBs, like others, face significant threats to their 
effectiveness; they review too much, too quickly, with too little expertise and conduct minimal 
review of approved research.  In addition, they face conflicts that threaten their independence, 
provide little training for investigators and board members, and devote little attention to 
evaluating IRB effectiveness.  Finally, they face major changes in the research environment. We 
conclude that the same recommendations set forth in our prior report apply to low-volume 
IRBs.  In implementing these recommendations, we suggest that oversight agencies pay special  Page 2 - See Addresses Below 
attention to the particular needs of low-volume IRBs.  Specifically, we recommend that the 
Office of Protection from Research Risks (OPRR) and the Food and Drug Administration 
address low-volume IRBs relative isolation from the rest of the IRB community by fostering 
collaborations between low-volume IRBs and others. 
Methodology 
We developed an awareness of the special challenges facing low-volume IRBs through our 
contacts with various IRB officials, as well as through our review of a recent Canadian 
commission report that addressed issues specifically pertaining to low-volume IRBs.3  We 
decided to conduct a limited follow-up inquiry to assess more fully the applicability of the 
findings from our earlier IRB reports to low-volume IRBs. 
We based this inquiry primarily on a judgmental sample of 12 low-volume IRBs.  We identified 
these IRBs through references from participants in our prior IRB research.  In choosing the 
IRBs to include in our sample, we attempted to attain both geographical diversity and a range of 
hospital types in order to broaden the applicability of our findings.  Our sample of IRBs 
conducted a median of 44 annual initial reviews, ranging from a low of 5 to a high of 124.4  We 
obtained data and conducted interviews with representatives from each IRB in our sample.  In 
addition, we interviewed several experienced IRB officials who offer educational consulting to 
low-volume IRBs.  Finally, we analyzed the data in a recent IRB-related National Institutes of 
Health (NIH) report.5 
Low-Volume IRBs, Like Other IRBs, are At-Risk 
Our prior report identified six basic factors that jeopardize the effectiveness of IRBs.  In this 
follow-up probe, we have found further confirmation that these factors apply to low-volume 
IRBs.  In interviewing low-volume IRB members and staff, we were impressed by the integrity 
and dedication that they have brought to the task of protecting human subjects.  They have 
made, and continue to make, important contributions.  However, systemic weaknesses exist and 
may be intensifying.  In Table 1 and the text below, we list five of the factors presented in our 
prior report and review their applicability to low-volume IRBs.  Subsequently, in Table 2, we 
address the sixth factor concerning the changing research environment.  Page 3 - See Addressees Below 
TABLE 1: APPLICABILITY OF KEY FINDINGS FROM OUR PRIOR REPORT 
FINDING  APPLICABILITY TO LOW-VOLUME IRBS 
IRBs review too much, too  Moderate. Though their protocol volume has increased only slightly, 
quickly, with too little  workload pressures still exist due to insufficient resources. Furthermore, 
expertise.  these IRBs tend to be isolated, with limited expertise in both scientific and 
ethical areas. Because they review so few protocols, reviewers breadth of 
experience with human-subject protection issues is lacking, potentially 
affecting their ability to discern subtle issues/violations. 
IRBs conduct minimal  Substantial. They tend to devote only a small proportion of IRB meetings to 
continuing review of approved  continuing review. Their isolation and limited expertise may make them less 
research.  likely to identify problems; they tend to raise fewer questions and require 
fewer modifications. Continuing reviews are virtually all paper-based. 
IRBs face conflicts that  Substantial.  Commercial sponsorship provides an increasingly significant 
threaten their independence.  source of revenue for many hospitals. This situation can intensify pressures 
to accommodate a hospitals financial interests.  Conflicts of interest may 
arise from these IRBs having one staff person acting in multiple conflicting 
roles. They have few community members, though they seem to have an 
easier time attracting nonaffiliated and nonscientific members than high-
volume IRBs. 
IRBs and their institutions  Substantial. Low-volume IRBs are often isolated, underfunded, and unable 
provide little training for  to provide training. Few have the funding to access outside training for 
investigators and board  members, unless they charge sponsors for reviews. Training of investigators 
members.  is minimal or non-existent. 
IRBs devote little attention to  Substantial. Few perform any self-evaluation, though many were receptive 
evaluating IRB effectiveness.  to this idea. A small number have hired outside consultants. 
They review too much, too quickly, with too little expertise.  It appears that low-volume 
IRBs have not been experiencing significant increases in protocol volume.6  However, the 
paucity of these IRBs resources may limit their effectiveness in protecting human subjects.  One 
IRB administrator in our sample described her IRB as merely a function; the board had neither 
specific resources allocated for it nor specific office space devoted to it.  Many of the IRBs in 
our sample had less than one full-time employee handling all of the IRBs administrative tasks. 
Many IRB officials and experts expressed concern as to whether low-volume IRBs have 
sufficient expertise to recognize complex scientific or ethical issues when they arise.  One IRB 
consultant observed a low-volume IRB review a gene therapy protocol that she considered to 
contain controversial ethical issues; these were not addressed by the IRB.  The IRBs that review 
extremely low volumes of protocols may be particularly lacking in expertise.  Some experts 
believe that IRBs that review a very small number of protocols each year do not develop what 
one IRB official referred to as a perception of risk, an intuition as to how risky one protocol is 
relative to others.  Page 4 - See Addressees Below 
They conduct minimal continuing review of approved research.  Low-volume IRBs tend to 
devote only a small portion of their meeting time to continuing review, despite the smaller 
number of protocols they must review.  Likewise, low-volume IRBs make few modifications to 
protocols during continuing review.  As with high-volume IRBs, more than 80 percent of these 
IRBs continuing reviews are approved as submitted.7  The IRBs may be less inclined to alter 
continuing protocols because they lack vital external information, including feedback from Data 
Safety Monitoring Boards (DSMBs) and meaningful adverse event reports, needed for judging 
the ongoing safety of the study.  Because hospitals represented by low-volume IRBs often enroll 
just one or two patients in a given multi-center trial, this dearth of external information is 
especially detrimental to their IRBs continuing review process. 
In addition, continuing review poses special problems for many low-volume IRBs, as they often 
do not have an adequate infrastructure for overseeing this process.  Many have difficulty 
tracking continuing reviews because they lack adequate resources, such as a computerized 
protocol management system.  The annual review requirement creates particular logistical 
difficulties for those IRBs that meet less than once per month. 
They face conflicts that threaten their independence.  Commercially-sponsored protocols 
can generate significant revenues for participating hospitals and investigators.  As a result, IRBs 
may experience pressures to 
accommodate these interests while 
Funding Sources of Protocols Approved 
still upholding their responsibilities to 
by IRBs in Our Sample 
protect human subjects.  Such 
Federal government as part of Cooperative Group Protocols  37% 
pressures are quite relevant to low-
Pharmaceutical manufacturers or their representatives  33% 
volume IRBs, as a high percentage of 
Hospitals internal sources  14% 
their research, a mean of 33 percent 
Device manufacturers or their representatives  4% 
of our sample in 1997, is industry-
Other Federal sources  4% 
sponsored (see insert).  Many IRBs 
Foundations  2% 
reported that they have experienced 
Other  6% 
IRB shopping by commercial 
sponsors, and some have felt 
pressure to accept research because 
of its potential for generating hospital revenue. 
In addition, the limited resources available to low-volume IRBs may result in one person being 
responsible for several conflicting duties.  We spoke to one IRB administrator who had, at one 
point, simultaneously acted as the data manager, clinical coordinator, and IRB coordinator for a 
hospital.  Another IRB in our sample included among its membership the entire hospital board. 
Low-volume IRBs have few nonaffiliated members, though the low-volume IRBs in our sample 
seemed to experience less difficulty attracting and maintaining community members for their 
IRBs than the IRBs in our prior research.  Nonaffiliated IRB members, as we noted in our prior 
report, can provide a helpful counter-balance to institutional interests.  Page 5 - See Addressees Below 
They and their institutions provide little training for investigators and board members. 
Training is especially important for these IRBs, particularly those overseeing a very low volume 
of protocols, because they tend to be relatively isolated from the rest of the IRB community. 
The minimal training primarily results from a lack of resources, since many of these low-volume 
IRBs are only budgeted to cover basic operational costs. 
Many low-volume IRBs are removed from informal information networks, such as discussions 
with research colleagues, and other channels generally available to IRBs at larger academic 
health centers.  This isolation is compounded by their occasional omission from more formal 
information networks, such as those facilitated by NIHs OPRR; OPRR holds educational 
conferences almost exclusively at high-volume multiple project assurance (MPA) institutions and 
sends some educational materials only to MPA institutions.  In addition, many of the IRB chairs 
and administrators in our sample had questions about IRB regulations, but were unaware of 
available resources for accessing this information, such as  the Public Responsibility in Medicine 
and Research (PRIM&R), the Applied Research Ethics National Association (ARENA)8, or 
MCWIRB, an online IRB-focused discussion forum. 
They devote little attention to evaluating IRB effectiveness.  Few of the IRBs in our sample 
have conducted internal evaluations.  However, many have discussed methods, such as 
performing on-site reviews of the consent process and bringing in consultants, for expanding 
upon the current paper-based review process.  The IRBs in our sample reported that they have 
been hindered more by the cost of these undertakings than by lack of interest in pursuing them. 
They face major changes in the research environment. 
TABLE 2: A CHANGING ENVIRONMENT FOR IRBS 
CHANGES IDENTIFIED IN  RELEVANCE OF THE CHANGES TO LOW-VOLUME IRBS 
PRIOR OIG REPORT 
Increased Commercialization  Substantial. Many have experienced IRB shopping and pressures from 
of Research  their institutions and/or sponsors for quick reviews for commercial sponsors. 
Proliferation of Multi-Center  Substantial. Adverse-event reports are difficult to interpret and sponsors 
Trials  provide little additional information about ongoing trials. 
New Types of Research  Moderate. Though generally not involved in cutting edge research, many 
of these hospitals are expanding into new arenas of research. This research 
often requires new, specialized expertise and introduces new ethical issues. 
Expansion of Managed Care  Moderate. Managed care is a limited, but growing presence in these 
hospitals. Where it exists, managed care cost pressures limit the allocation of 
personnel and other resources for IRB-related activities. 
Increased Number of  Moderate. While these IRBs are not facing significant increases in protocol 
Proposals  volume, lack of sufficient IRB resources may preclude them from conducting a 
complete and timely review of their existing workload. 
Rise of Patient Consumerism  Minor. Some IRBs have encountered patient demands for certain unproven 
therapies, though most have experienced little or no patient consumerism.  Page 6 - See Addressees Below 
Low-volume IRBs have been particularly affected by the proliferation of multi-center trials and 
the increased commercialization of research.  These changes have introduced research into new 
settings, including hospitals traditionally focused on clinical care.  This research often exposes 
IRBs to new ethical and scientific issues that may exceed their current realm of expertise. 
Managed care has not yet penetrated many of the hospitals in our sample and therefore has not 
affected their IRBs as much as IRBs housed in large academic health centers.  However, almost 
all of the hospitals in our sample have experienced the effects of shrinking budgets and, 
consequently, a shortage of IRB resources.  Some changes, such as patient consumerism and 
new types of biomedical research are not yet having a significant impact on low-volume IRBs. 
Though all of the aforementioned changes have affected low-volume IRBs, these changes have 
had a slower and less intense impact than they have had on large academic health centers. 
Conclusion 
We have determined in this follow-up inquiry that the findings of our prior report are generally 
quite applicable to low-volume IRBs.  Therefore, we reiterate the importance of the 
recommendations stated in that report.  Although there are some differences in how and to what 
degree these findings apply, the overall picture that emerges clearly indicates that the call for 
reform we issued in our prior report applies to the broad universe of IRBs.  In essence, these 
recommendations are as valid for community hospital IRBs responsible for reviewing a few 
dozen protocols a year as they are for academic health center IRBs responsible for reviewing a 
few thousand. 
This similarity, however, does not alter the fact that low-volume IRBs have some distinctive 
characteristics that warrant special attention in carrying out these recommendations.  The most 
notable of these characteristics is a sense of isolation.  Although we recognize the limited 
resources that FDA and OPRR have to conduct educational outreach, we suggest that they pay 
special attention to low-volume IRBs and the issue of isolation when conducting this outreach. 
Such outreach could be accomplished through the issuance of FDA informational letters and 
OPRR Dear Colleague letters, the inclusion of additional material on web-sites, the 
preparation and distribution of special video material, or the use of existing networks, such as 
ARENA, PRIM&R, and cooperative research groups. 
As part of these efforts, we suggest that OPRR and FDA focus on fostering collaborations 
between low-volume IRBs and other IRBs.  Closer linkages with more experienced IRBs would 
strengthen their participation in the broader research community.  Such collaborations might 
involve more experienced IRBs assisting low-volume IRBs by conducting training sessions, 
providing advice in special circumstances, and even in carrying out protocol reviews.  We 
believe that the isolation of low-volume IRBs can be addressed more effectively through joint 
efforts among IRBs than through central initiatives by the Federal Government.  Page 7 - See Addressees Below

If you have any questions about this memorandum report, or about our work on IRBs in

general, please feel free to call me or George Grob or have your staff contact Mary Beth Clarke

at (202) 619-2481.

Attachment

Addressees:

Harold E. Varmus, M.D.

Director 

National Institutes of Health

Michael A. Friedman, M.D.

Lead Deputy Commissioner 

Food and Drug Administration
 Appendix A:  Low-Volume IRB Promising Approaches 
In conducting interviews with IRB chairs and administrators, we encountered a number of 
promising approaches for handling the substantial responsibility of low-volume IRBs.  We 
present some of them in this appendix.  In determining what to characterize as a promising 
approach, we depended on the judgements of IRB members, administrators, and consultants, as 
well as our own judgement of whether the approach was significantly novel and important to 
warrant attention.  The innovative practices included here are specifically geared towards 
combating the isolation of low-volume IRBs.  In addition to the promising approaches listed 
here, we devoted an entire prior report, Institutional Review Boards: Promising Approaches 
(OEI-01-97-00191), to describing innovative IRB practices.  While most of the ideas and 
techniques identified in that report were found in large academic health centers, we believe that 
many of those promising approaches could also be applied to low-volume IRBs. 
We did not independently evaluate any practice highlighted in this appendix.  Furthermore, a 
practices inclusion does not necessarily mean that it receives our stamp of approval; nor are we 
claiming in highlighting a promising approach at one institution that other institutions have not 
developed similar approaches.  We intend to provide a snapshot of the types of innovations 
taking place in IRBs in order to fuel discussion among interested parties; it is by no means a 
definitive study of all IRBs. 
Using consultants 
Procedural.  Several IRBs have brought in outside consultants, generally people who have been 
IRB professionals at large, academic medical centers for many years, to advise on ways that the 
IRB could improve its efficiency and effectiveness.  These consultants bring a broad perspective 
and a great deal of knowledge and experience to low-volume IRBs, both in substantive areas, 
such as the ethics of human-subject research, and in procedural issues relating to IRBs.  These 
consultants recommendations tend to be much more performance-based than those espoused by 
FDA inspectors.  All of the IRBs in our sample who had hired consultants found them helpful. 
Scientific.  In order to ensure the quality and appropriateness of a protocol, one IRB chair often 
asks a hospital physician, who specializes in the same field as the investigator submitting the 
protocol in question, to review that protocol.  He finds this to be an effective way to verify the 
robustness of the design, the accuracy of the specialized information, and the relevance of this 
study in light of current work in the field. 
Training at nearby hospitals 
A-1
 Though low-volume IRBs tend to have little, if any, budget to provide training for investigators 
and/or members, many of these IRBs are located close to research hospitals which do sponsor 
such training.  One IRB in our sample regularly provides opportunities for members and 
investigators to attend training sessions at a local large academic medical center. 
Including an experienced member of a local high-volume IRB on a low-volume IRB 
One way of combating the sense of isolation that many of these low-volume IRBs feel is to 
recruit someone who sits on the IRB of a nearby high-volume hospital onto their own IRB. 
Including a dual member on their IRB is a low-cost method for low-volume IRBs to stay 
abreast of ever-changing practices and regulations. The IRB in our sample that included such a 
member reported that this member added an unprecedented continuing education component to 
their meetings.  Furthermore, she had invited any interested members to a meeting at the high-
volume IRB on which she also serves, to observe their procedures and witness how they handle 
certain issues. 
Charging commercial sponsors 
Some have begun charging a small fee to commercial sponsors.  IRBs have reported using this 
money for a wide array of IRB-related activities including education and training of members. 
However, we recognize that, although this generates needed funds for many of these IRBs, it 
places much of the financial burden on commercial sponsors.  Although such a practice 
potentially could create tensions between investigators and commercial sponsors, the IRBs 
currently engaging in this practice report that charging a fee to commercial sponsors has not 
discouraged any research thus far. 
A-2
 Appendix B:  Endnotes 
1. The general recommendations from our prior report, Institutional Review Boards:  A Time for 
Reform (OEI-01-97-00193), were directed jointly to NIH and FDA, and are listed below.  Each 
of these was followed by a number of more specific operational recommendations. 
C	 Recast Federal IRB requirements so that they grant IRBs greater flexibility and hold them 
more accountable for results. 
C  Strengthen continuing protections for human subjects participating in research. 
C	 Enact Federal requirements that help ensure that investigators and IRB members are 
adequately sensitized to human subject protections. 
C	 Help insulate IRBs from conflicts that can compromise their mission in protecting human 
subjects. 
C	 Recognize the seriousness of the workload pressures that many IRBs face and take actions 
that aim to moderate them. 
C  Reengineer the Federal oversight process. 
2. There are a number of low-volume independent IRBs that are not part of a hospital 
organization.  Although we do not consider independent IRBs in this memorandum, they are 
examined in our prior report, Institutional Review Boards: Emergence of Independent Boards 
(OEI-01-97-00192). 
3. National Council on Bioethics in Human Research, Protecting and Promoting the Human

Research Subject: A Review of the Function of Research Ethics Boards in Canadian Faculties of

Medicine, NCBHR Communique CNBRH, Vol.6 (1995) No.1.  This Canadian commission

report raised concerns about low-volume IRBs, stating that those that oversee a very low number

of protocols are unable to conduct reviews with adequate breadth and depth.  The report

recommended that IRBs which review less than 50 proposals annually should merge with an IRB

from another institution.

4. Total annual initial reviews include both full board and expedited reviews.

5. NIH Office of Extramural Research, Evaluation of NIH Implementation of Section 491 of the

Public Health Service Act, Mandating a Program of Protection for Research Subjects, 

June 15, 1998.  This is a broad, data-rich survey of 491 IRBs that held multiple project assurances

(MPAs) in 1995.

6. For the IRBs in our sample, initial reviews increased by a mean of 16 percent between 1993

and 1997, in contrast to the 42 percent increase found at the 6 academic medical centers cited in

our previous report.

7.  NIH Office of Extramural Research, Evaluation of NIH Implementation of Section 491 of the

B-1
 Public Health Service Act, Mandating a Program of Protection for Research Subjects, 
June 15, 1998. 
8. The Applied Research Ethics National Association (ARENA) is a subsidiary of the Public 
Responsibility in Medicine and Research (PRIM&R), which is a nonprofit organization dedicated 
to promoting the ethical conduct of research.  ARENA is a professional association whose 
members include administrators and members of Institutional Review Boards and Institutional 
Animal Care and Use Committees. 
B-2