[Memo Report] 

October 23, 1998 

June Gibbs Brown 

Inspector General 

Final Report on Low-Volume Institutional Review Boards (OEI-01-97-00194) 

See Addressees Below 

In this memorandum, I would like to share with you the results of a brief review we conducted 

of low-volume institutional review boards (IRBs). We did this as a follow-up to our recent 

study of challenges facing IRBs. 

In general, in our June 1998 summary report, we concluded that the IRB system is in jeopardy 

and made numerous recommendations for reforming the system.1 While the findings and 

recommendations of these reports applied to all IRBs, they had particular relevance to academic 

health centers whose primary focus is research. 

While conducting the research for these reports, we became aware that there are some unique 

challenges facing hospital-based IRBs outside of the academic health centers. Because these 

IRBs tend to oversee considerably fewer research protocols than those at the large research 

centers, we refer to them as “low-volume” IRBs; in this memorandum, we define low-volume 

IRBs as those that conduct less than 125 initial reviews annually. This definition most likely 

encompasses a majority of the 3,000-5,000 IRBs in this country. However, given their smaller 

caseloads, low-volume IRBs probably oversee the minority of protocols subject to IRB review. 

These IRBs typically reside in hospitals that are community-focused and lack a research culture.2 

In brief, our results indicate that low-volume IRBs, like others, face significant threats to their 

effectiveness; they review too much, too quickly, with too little expertise and conduct minimal 

review of approved research. In addition, they face conflicts that threaten their independence, 

provide little training for investigators and board members, and devote little attention to 

evaluating IRB effectiveness. Finally, they face major changes in the research environment. We 

conclude that the same recommendations set forth in our prior report apply to low-volume 

IRBs. In implementing these recommendations, we suggest that oversight agencies pay special 
Page 2 - See Addresses Below 

attention to the particular needs of low-volume IRBs. Specifically, we recommend that the 

Office of Protection from Research Risks (OPRR) and the Food and Drug Administration 

address low-volume IRBs’ relative isolation from the rest of the IRB community by fostering 

collaborations between low-volume IRBs and others. 

Methodology 

We developed an awareness of the special challenges facing low-volume IRBs through our 

contacts with various IRB officials, as well as through our review of a recent Canadian 

commission report that addressed issues specifically pertaining to low-volume IRBs.3 We 

decided to conduct a limited follow-up inquiry to assess more fully the applicability of the 

findings from our earlier IRB reports to low-volume IRBs. 

We based this inquiry primarily on a judgmental sample of 12 low-volume IRBs. We identified 

these IRBs through references from participants in our prior IRB research. In choosing the 

IRBs to include in our sample, we attempted to attain both geographical diversity and a range of 

hospital types in order to broaden the applicability of our findings. Our sample of IRBs 

conducted a median of 44 annual initial reviews, ranging from a low of 5 to a high of 124.4 We 

obtained data and conducted interviews with representatives from each IRB in our sample. In 

addition, we interviewed several experienced IRB officials who offer educational consulting to 

low-volume IRBs. Finally, we analyzed the data in a recent IRB-related National Institutes of 

Health (NIH) report.5 

Low-Volume IRBs, Like Other IRBs, are At-Risk 

Our prior report identified six basic factors that jeopardize the effectiveness of IRBs. In this 

follow-up probe, we have found further confirmation that these factors apply to low-volume 

IRBs. In interviewing low-volume IRB members and staff, we were impressed by the integrity 

and dedication that they have brought to the task of protecting human subjects. They have 

made, and continue to make, important contributions. However, systemic weaknesses exist and 

may be intensifying. In Table 1 and the text below, we list five of the factors presented in our 

prior report and review their applicability to low-volume IRBs. Subsequently, in Table 2, we 

address the sixth factor concerning the changing research environment. 
Page 3 - See Addressees Below 

TABLE 1: APPLICABILITY OF KEY FINDINGS FROM OUR PRIOR REPORT 

FINDING APPLICABILITY TO LOW-VOLUME IRBS 

IRBs review too much, too 

quickly, with too little 

expertise. 

Moderate. Though their protocol volume has increased only slightly, 

workload pressures still exist due to insufficient resources. Furthermore, 

these IRBs tend to be isolated, with limited expertise in both scientific and 

ethical areas. Because they review so few protocols, reviewers’ breadth of 

experience with human-subject protection issues is lacking, potentially 

affecting their ability to discern subtle issues/violations. 

IRBs conduct minimal 

continuing review of approved 

research. 

Substantial. They tend to devote only a small proportion of IRB meetings to 

continuing review. Their isolation and limited expertise may make them less 

likely to identify problems; they tend to raise fewer questions and require 

fewer modifications. Continuing reviews are virtually all paper-based. 

IRBs face conflicts that 

threaten their independence. 

Substantial. Commercial sponsorship provides an increasingly significant 

source of revenue for many hospitals. This situation can intensify pressures 

to accommodate a hospital’s financial interests. Conflicts of interest may 

arise from these IRBs having one staff person acting in multiple conflicting 

roles. They have few community members, though they seem to have an 

easier time attracting nonaffiliated and nonscientific members than highvolume IRBs. 

IRBs and their institutions 

provide little training for 

investigators and board 

members. 

Substantial. Low-volume IRBs are often isolated, underfunded, and unable 

to provide training. Few have the funding to access outside training for 

members, unless they charge sponsors for reviews. Training of investigators 

is minimal or non-existent. 

IRBs devote little attention to 

evaluating IRB effectiveness. 

Substantial. Few perform any self-evaluation, though many were receptive 

to this idea. A small number have hired outside consultants. 

They review too much, too quickly, with too little expertise. It appears that low-volume 

IRBs have not been experiencing significant increases in protocol volume.6 However, the 

paucity of these IRBs’ resources may limit their effectiveness in protecting human subjects. One 

IRB administrator in our sample described her IRB as merely a “function”; the board had neither 

specific resources allocated for it nor specific office space devoted to it. Many of the IRBs in 

our sample had less than one full-time employee handling all of the IRB’s administrative tasks. 

Many IRB officials and experts expressed concern as to whether low-volume IRBs have 

sufficient expertise to recognize complex scientific or ethical issues when they arise. One IRB 

consultant observed a low-volume IRB review a gene therapy protocol that she considered to 

contain controversial ethical issues; these were not addressed by the IRB. The IRBs that review 

extremely low volumes of protocols may be particularly lacking in expertise. Some experts 

believe that IRBs that review a very small number of protocols each year do not develop what 

one IRB official referred to as a “perception of risk,” an intuition as to how risky one protocol is 

relative to others. 
Page 4 - See Addressees Below 

They conduct minimal continuing review of approved research. Low-volume IRBs tend to 

devote only a small portion of their meeting time to continuing review, despite the smaller 

number of protocols they must review. Likewise, low-volume IRBs make few modifications to 

protocols during continuing review. As with high-volume IRBs, more than 80 percent of these 

IRBs’ continuing reviews are approved as submitted.7 The IRBs may be less inclined to alter 

continuing protocols because they lack vital external information, including feedback from Data 

Safety Monitoring Boards (DSMBs) and meaningful adverse event reports, needed for judging 

the ongoing safety of the study. Because hospitals represented by low-volume IRBs often enroll 

just one or two patients in a given multi-center trial, this dearth of external information is 

especially detrimental to their IRB’s continuing review process. 

In addition, continuing review poses special problems for many low-volume IRBs, as they often 

do not have an adequate infrastructure for overseeing this process. Many have difficulty 

tracking continuing reviews because they lack adequate resources, such as a computerized 

protocol management system. The annual review requirement creates particular logistical 

difficulties for those IRBs that meet less than once per month. 

They face conflicts that threaten their independence. Commercially-sponsored protocols 

can generate significant revenues for participating hospitals and investigators. As a result, IRBs 

may experience pressures to 

accommodate these interests while 

still upholding their responsibilities to 

protect human subjects. Such 

pressures are quite relevant to lowvolume IRBs, as a high percentage of 

their research, a mean of 33 percent 

of our sample in 1997, is industrysponsored (see insert). Many IRBs 

reported that they have experienced 

“IRB shopping” by commercial 

sponsors, and some have felt 

pressure to accept research because 

Funding Sources of Protocols Approved 

by IRBs in Our Sample 

Federal government as part of Cooperative Group Protocols 37% 

Pharmaceutical manufacturers or their representatives 33% 

Hospital’s internal sources 14% 

Device manufacturers or their representatives 4% 

Other Federal sources 4% 

Foundations 2% 

Other 6% 

of its potential for generating hospital revenue. 

In addition, the limited resources available to low-volume IRBs may result in one person being 

responsible for several conflicting duties. We spoke to one IRB administrator who had, at one 

point, simultaneously acted as the data manager, clinical coordinator, and IRB coordinator for a 

hospital. Another IRB in our sample included among its membership the entire hospital board. 

Low-volume IRBs have few nonaffiliated members, though the low-volume IRBs in our sample 

seemed to experience less difficulty attracting and maintaining community members for their 

IRBs than the IRBs in our prior research. Nonaffiliated IRB members, as we noted in our prior 

report, can provide a helpful counter-balance to institutional interests. 
Page 5 - See Addressees Below 

They and their institutions provide little training for investigators and board members. 

Training is especially important for these IRBs, particularly those overseeing a very low volume 

of protocols, because they tend to be relatively isolated from the rest of the IRB community. 

The minimal training primarily results from a lack of resources, since many of these low-volume 

IRBs are only budgeted to cover basic operational costs. 

Many low-volume IRBs are removed from informal information networks, such as discussions 

with research colleagues, and other channels generally available to IRBs at larger academic 

health centers. This isolation is compounded by their occasional omission from more formal 

information networks, such as those facilitated by NIH’s OPRR; OPRR holds educational 

conferences almost exclusively at high-volume multiple project assurance (MPA) institutions and 

sends some educational materials only to MPA institutions. In addition, many of the IRB chairs 

and administrators in our sample had questions about IRB regulations, but were unaware of 

available resources for accessing this information, such as the Public Responsibility in Medicine 

and Research (PRIM&R), the Applied Research Ethics National Association (ARENA)8

, or 

MCWIRB, an online IRB-focused discussion forum. 

They devote little attention to evaluating IRB effectiveness. Few of the IRBs in our sample 

have conducted internal evaluations. However, many have discussed methods, such as 

performing on-site reviews of the consent process and bringing in consultants, for expanding 

upon the current paper-based review process. The IRBs in our sample reported that they have 

been hindered more by the cost of these undertakings than by lack of interest in pursuing them. 

They face major changes in the research environment. 

TABLE 2: A CHANGING ENVIRONMENT FOR IRBS 

CHANGES IDENTIFIED IN 

PRIOR OIG REPORT 

RELEVANCE OF THE CHANGES TO LOW-VOLUME IRBS 

Increased Commercialization 

of Research 

Substantial. Many have experienced “IRB shopping” and pressures from 

their institutions and/or sponsors for quick reviews for commercial sponsors. 

Proliferation of Multi-Center 

Trials 

Substantial. Adverse-event reports are difficult to interpret and sponsors 

provide little additional information about ongoing trials. 

New Types of Research Moderate. Though generally not involved in “cutting edge” research, many 

of these hospitals are expanding into new arenas of research. This research 

often requires new, specialized expertise and introduces new ethical issues. 

Expansion of Managed Care Moderate. Managed care is a limited, but growing presence in these 

hospitals. Where it exists, managed care cost pressures limit the allocation of 

personnel and other resources for IRB-related activities. 

Increased Number of 

Proposals 

Moderate. While these IRBs are not facing significant increases in protocol 

volume, lack of sufficient IRB resources may preclude them from conducting a 

complete and timely review of their existing workload. 

Rise of Patient Consumerism Minor. Some IRBs have encountered patient demands for certain unproven 

“therapies,” though most have experienced little or no patient consumerism. 
Page 6 - See Addressees Below 

Low-volume IRBs have been particularly affected by the proliferation of multi-center trials and 

the increased commercialization of research. These changes have introduced research into new 

settings, including hospitals traditionally focused on clinical care. This research often exposes 

IRBs to new ethical and scientific issues that may exceed their current realm of expertise. 

Managed care has not yet penetrated many of the hospitals in our sample and therefore has not 

affected their IRBs as much as IRBs housed in large academic health centers. However, almost 

all of the hospitals in our sample have experienced the effects of shrinking budgets and, 

consequently, a shortage of IRB resources. Some changes, such as patient consumerism and 

new types of biomedical research are not yet having a significant impact on low-volume IRBs. 

Though all of the aforementioned changes have affected low-volume IRBs, these changes have 

had a slower and less intense impact than they have had on large academic health centers. 

Conclusion 

We have determined in this follow-up inquiry that the findings of our prior report are generally 

quite applicable to low-volume IRBs. Therefore, we reiterate the importance of the 

recommendations stated in that report. Although there are some differences in how and to what 

degree these findings apply, the overall picture that emerges clearly indicates that the call for 

reform we issued in our prior report applies to the broad universe of IRBs. In essence, these 

recommendations are as valid for community hospital IRBs responsible for reviewing a few 

dozen protocols a year as they are for academic health center IRBs responsible for reviewing a 

few thousand. 

This similarity, however, does not alter the fact that low-volume IRBs have some distinctive 

characteristics that warrant special attention in carrying out these recommendations. The most 

notable of these characteristics is a sense of isolation. Although we recognize the limited 

resources that FDA and OPRR have to conduct educational outreach, we suggest that they pay 

special attention to low-volume IRBs and the issue of isolation when conducting this outreach. 

Such outreach could be accomplished through the issuance of FDA informational letters and 

OPRR “Dear Colleague” letters, the inclusion of additional material on web-sites, the 

preparation and distribution of special video material, or the use of existing networks, such as 

ARENA, PRIM&R, and cooperative research groups. 

As part of these efforts, we suggest that OPRR and FDA focus on fostering collaborations 

between low-volume IRBs and other IRBs. Closer linkages with more experienced IRBs would 

strengthen their participation in the broader research community. Such collaborations might 

involve more experienced IRBs assisting low-volume IRBs by conducting training sessions, 

providing advice in special circumstances, and even in carrying out protocol reviews. We 

believe that the isolation of low-volume IRBs can be addressed more effectively through joint 

efforts among IRBs than through central initiatives by the Federal Government. 
Page 7 - See Addressees Below

If you have any questions about this memorandum report, or about our work on IRBs in

general, please feel free to call me or George Grob or have your staff contact Mary Beth Clarke

at (202) 619-2481.

Attachment

Addressees:

Harold E. Varmus, M.D.

Director 

National Institutes of Health

Michael A. Friedman, M.D.

Lead Deputy Commissioner 

Food and Drug Administration
Appendix A: Low-Volume IRB Promising Approaches 

In conducting interviews with IRB chairs and administrators, we encountered a number of 

promising approaches for handling the substantial responsibility of low-volume IRBs. We 

present some of them in this appendix. In determining what to characterize as a promising 

approach, we depended on the judgements of IRB members, administrators, and consultants, as 

well as our own judgement of whether the approach was significantly novel and important to 

warrant attention. The innovative practices included here are specifically geared towards 

combating the isolation of low-volume IRBs. In addition to the promising approaches listed 

here, we devoted an entire prior report, Institutional Review Boards: Promising Approaches 

(OEI-01-97-00191), to describing innovative IRB practices. While most of the ideas and 

techniques identified in that report were found in large academic health centers, we believe that 

many of those promising approaches could also be applied to low-volume IRBs. 

We did not independently evaluate any practice highlighted in this appendix. Furthermore, a 

practice’s inclusion does not necessarily mean that it receives our stamp of approval; nor are we 

claiming in highlighting a promising approach at one institution that other institutions have not 

developed similar approaches. We intend to provide a snapshot of the types of innovations 

taking place in IRBs in order to fuel discussion among interested parties; it is by no means a 

definitive study of all IRBs. 

Using consultants 

Procedural. Several IRBs have brought in outside consultants, generally people who have been 

IRB professionals at large, academic medical centers for many years, to advise on ways that the 

IRB could improve its efficiency and effectiveness. These consultants bring a broad perspective 

and a great deal of knowledge and experience to low-volume IRBs, both in substantive areas, 

such as the ethics of human-subject research, and in procedural issues relating to IRBs. These 

consultants’ recommendations tend to be much more performance-based than those espoused by 

FDA inspectors. All of the IRBs in our sample who had hired consultants found them helpful. 

Scientific. In order to ensure the quality and appropriateness of a protocol, one IRB chair often 

asks a hospital physician, who specializes in the same field as the investigator submitting the 

protocol in question, to review that protocol. He finds this to be an effective way to verify the 

robustness of the design, the accuracy of the specialized information, and the relevance of this 

study in light of current work in the field. 

Training at nearby hospitals 

A-1
Though low-volume IRBs tend to have little, if any, budget to provide training for investigators 

and/or members, many of these IRBs are located close to research hospitals which do sponsor 

such training. One IRB in our sample regularly provides opportunities for members and 

investigators to attend training sessions at a local large academic medical center. 

Including an experienced member of a local high-volume IRB on a low-volume IRB 

One way of combating the sense of isolation that many of these low-volume IRBs feel is to 

recruit someone who sits on the IRB of a nearby high-volume hospital onto their own IRB. 

Including a “dual” member on their IRB is a low-cost method for low-volume IRBs to stay 

abreast of ever-changing practices and regulations. The IRB in our sample that included such a 

member reported that this member added an unprecedented continuing education component to 

their meetings. Furthermore, she had invited any interested members to a meeting at the highvolume IRB on which she also serves, to observe their procedures and witness how they handle 

certain issues. 

Charging commercial sponsors 

Some have begun charging a small fee to commercial sponsors. IRBs have reported using this 

money for a wide array of IRB-related activities including education and training of members. 

However, we recognize that, although this generates needed funds for many of these IRBs, it 

places much of the financial burden on commercial sponsors. Although such a practice 

potentially could create tensions between investigators and commercial sponsors, the IRBs 

currently engaging in this practice report that charging a fee to commercial sponsors has not 

discouraged any research thus far. 

A-2
Appendix B: Endnotes 

1. The general recommendations from our prior report, Institutional Review Boards: A Time for 

Reform (OEI-01-97-00193), were directed jointly to NIH and FDA, and are listed below. Each 

of these was followed by a number of more specific operational recommendations. 

C Recast Federal IRB requirements so that they grant IRBs greater flexibility and hold them 

more accountable for results. 

C Strengthen continuing protections for human subjects participating in research. 

C Enact Federal requirements that help ensure that investigators and IRB members are 

adequately sensitized to human subject protections. 

C Help insulate IRBs from conflicts that can compromise their mission in protecting human 

subjects. 

C Recognize the seriousness of the workload pressures that many IRBs face and take actions 

that aim to moderate them. 

C Reengineer the Federal oversight process. 

2. There are a number of low-volume independent IRBs that are not part of a hospital 

organization. Although we do not consider independent IRBs in this memorandum, they are 

examined in our prior report, Institutional Review Boards: Emergence of Independent Boards 

(OEI-01-97-00192). 

3. National Council on Bioethics in Human Research, “Protecting and Promoting the Human

Research Subject: A Review of the Function of Research Ethics Boards in Canadian Faculties of

Medicine,” NCBHR Communique CNBRH, Vol.6 (1995) No.1. This Canadian commission

report raised concerns about low-volume IRBs, stating that those that oversee a very low number

of protocols are unable to conduct reviews with adequate breadth and depth. The report

recommended that IRBs which review less than 50 proposals annually should merge with an IRB

from another institution.

4. Total annual initial reviews include both full board and expedited reviews.

5. NIH Office of Extramural Research, “Evaluation of NIH Implementation of Section 491 of the

Public Health Service Act, Mandating a Program of Protection for Research Subjects,” 

June 15, 1998. This is a broad, data-rich survey of 491 IRBs that held multiple project assurances

(MPAs) in 1995.

6. For the IRBs in our sample, initial reviews increased by a mean of 16 percent between 1993

and 1997, in contrast to the 42 percent increase found at the 6 academic medical centers cited in

our previous report.

7. NIH Office of Extramural Research, “Evaluation of NIH Implementation of Section 491 of the

B-1
Public Health Service Act, Mandating a Program of Protection for Research Subjects,” 

June 15, 1998. 

8. The Applied Research Ethics National Association (ARENA) is a subsidiary of the Public 

Responsibility in Medicine and Research (PRIM&R), which is a nonprofit organization dedicated 

to promoting the ethical conduct of research. ARENA is a professional association whose 

members include administrators and members of Institutional Review Boards and Institutional 

Animal Care and Use Committees. 

B-2