Is Volume Related to Outcome in Health Care? A Systematic Review and Methodologic Critique of the Literature

Measuring and understanding the association between volume and outcome in the delivery of health services has been the focus of much research since the 1980s (1, 2). Recently, purchasers and consumer groups have taken an interest in the wider public dissemination of data on hospital volume for specific surgical procedures and medical conditions (3-5). Because high volume has been associated with better health outcomes for a variety of procedures and conditions, these groups recommend that consumers or health plans use data on volume to choose high-quality hospitals and physicians. Because reliable, valid, and timely information about health outcomes is lacking, data on volume are even more attractive. The wisdom of this policy, however, depends critically on the strength, magnitude, and meaning of the association between volume and outcome in health care. During the past 20 years, a large body of research has focused on measuring and explaining the association between patient outcomes and the volume of health services provided by hospitals and physicians (1, 2, 6-11). Many [but not all] studies have documented that higher volume is associated with better outcomes. Two principal hypotheses have been advanced to explain these relationships: 1) Physicians (and hospitals) develop more effective skills if they treat more patients [practice makes perfect] or 2) physicians (and hospitals) achieving better outcomes receive more referrals and thus accrue larger volumes (selective referral) (2, 6, 9, 10). Past reviews of this subject have focused only on specific procedures or conditions (12-14) or on effects of hospital volume (12, 15), have included nonpopulation-based studies (14-16), or were done before many recent primary studies (12, 16). In addition, to our knowledge, no comprehensive review has systematically analyzed and described the methodologic rigor of this literature. This project was undertaken to address some of these limitations as part of a 2000 joint workshop of the U.S. Institute of Medicine and the National Cancer Policy Board (17). We set out to conduct a systematic review of the research evidence linking volume and outcome in health care, to summarize and describe the methodologic rigor of the existing literature, and to highlight the research and policy implications of these findings. Methods Study Identification and Selection We searched MEDLINE for English-language articles published between 1 January 1980 and 31 December 2000 that assessed the relationship between volume and outcome in health care. Our search algorithms combined the Medical Subject Headings (MeSH terms) and text words outcome, outcome assessment, outcome and process assessment, volume, utilization, frequency, statistics, and regionalization with names of clinical conditions and procedures (Appendix). Bibliographies were reviewed to identify other articles of interest, and experts were contacted about missed or unpublished studies. We also searched the Cochrane Collaboration database. For our systematic review, we selected English-language articles in which hospital or physician volume was an independent variable, health outcomes (for example, death, stroke, or clinical complications) were the dependent variable, samples were community- or population-based, and patients had primarily been treated since 1980. When multiple articles used the same data, we included the most recent or complete article. We excluded studies published before 1980 because clinical and surgical care and outcomes have changed considerably since then for nearly all of the topics we reviewed. We excluded studies from single institutions, voluntary registries, or other convenience samples because of concerns about reporting bias and generalizability. Our initial search also identified articles about volume and outcome in trauma, newborn intensive care, and organ transplantation. We did not include these topics in our review because these services have already been extensively regionalized or are otherwise highly regulated by government or other administrative bodies. Review Methods Two of the authors reviewed each article independently, using a standardized abstraction form to record study characteristics, methodologic attributes, and research results. Reviewers were not blinded to journal, authors, or findings. The degree of initial agreement between reviewers was very high ( = 0.82 to 1.0). Any discrepancies were resolved by discussion. A third reviewer was used in rare instances when the original two reviewers did not reach full agreement. We critically read each study with an eye toward assessing its methodologic rigor as well as describing its results. Our methodologic review focused on key design attributes that influence the validity and generalizability of systematic reviews and the studies on which they are based (18, 19). Some preexisting patient factors, such as disease severity and burden of comorbid illness, are clear determinants of outcome independent of volume. Because it is necessary to account for differences in such factors to make valid comparisons between high- and low-volume providers, we assessed the quality of risk-adjustment techniques (12, 14, 20). We considered four levels of risk adjustment (none, models based on administrative data alone, models that included some clinical data, and clinical models with good calibration [HosmerLemeshow test]) and discrimination (c-statistic 0.75) (20). In addition, we noted whether the investigators controlled for any specific processes of care (medical therapies, surgical techniques, or adjunctive procedures) that are known to result in better outcomes. Increasing evidence shows that high-volume providers may more consistently use proven effective therapies, such as aspirin or -blockers in myocardial infarction (21) or adjuvant therapy in breast cancer (22). We also noted whether the investigators had explicitly measured the appropriateness of patient selection for procedural conditions. Although there has been little investigation of this issue, one study of carotid endarterectomy found that high-volume surgeons had a higher proportion of operations for inappropriate indications compared with low-volume surgeons (23). We ascertained four other analytic attributes: the type of outcomes assessed, the number of hospitals and physicians in each study (an indicator of the potential for variation at the two provider levels of interest), the number of volume strata examined, and the unit of analysis (whether effects of hospital and physician volume were assessed). Finally, samples that included all patients undergoing a specific procedure were considered more generalizable to the overall population than studies restricted to patients with one type of insurance. Statistical Analysis Studies that found a statistically significant mortality advantage for high-volume providers (P < 0.05) were considered positive. For each study, we recorded the definition of high and low volume (cases per year) and the overall mortality rate of the sample. All summary death rates reflect in-hospital mortality unless otherwise specified. We calculated the absolute difference in mortality rate between the highest and lowest volume strata reported and expressed this as the number of excess deaths per 100 cases attributable to low volume. We report medians and ranges for high- and low-volume thresholds, mean mortality rates, and absolute differences in mortality rates. A few studies assessed volume as a continuous variable, making it impossible to calculate a mortality difference between high- and low-volume providers. We included data on whether these studies demonstrated statistically significant associations in our summary. We used chi-square tests to assess associations between positive findings and methodologic attributes. All analyses were done by using SAS software, version 6.12 (SAS Institute, Inc., Cary, North Carolina). Results Methodologic Characteristics of Included Studies Our systematic review identified 254 articles. Eighteen articles reported on more than one procedure or condition. Because we summarized the literature by topic, we evaluated each article that studied more than one procedure as more than one study. We reviewed a total of 272 studies, of which 135 (53%) met all inclusion criteria. These 135 studies covered 27 clinical topics. Studies were most commonly excluded because volume was not an independent variable (n = 43), the sample was not population based (n = 40), or the study data were obtained before 1980 (n = 29). Studies were also excluded if no primary health outcomes were assessed, if mixed outcomes were assessed (for example, death plus long length of stay), or if data were duplicative. We found the literature to be extremely heterogeneous, even for a given procedure, making formal meta-analysis impossible. The methodologic characteristics of the 135 studies we reviewed are described in Table 1. Overall, 124 studies were from the United States, 6 were from Canada, 3 were from Europe, and 2 were from the United Kingdom. Half of all the studies we identified were published since 1998, underscoring the growing interest in this topic. The vast majority of reports were based on state or national hospital-discharge databases and had sample sizes that exceeded 1000 and included 20 or more hospitals and 50 or more physicians. For 79% of studies, the primary outcome was death, usually inpatient death. Many studies, however, reported 30-day mortality rates, and several articles on cancer reported long-term survival. Twenty-one percent of studies measured more than just death. For example, all 12 studies of coronary angioplasty also reported rates of emergency coronary artery bypass surgery, and half of the investigations of carotid endarterectomy reported stroke or other neurologic outcomes. Ninety percent of studies analyzed at least three categories of volume, usually tertiles or quartiles, and a