Adjustments for Center in Multicenter Studies: An Overview

Multicenter studies offer powerful methods for understanding the effects of treatment and exposure on patient outcomes. Although convenient and expedient, simple pooling of data as if they arose from a single population can produce incorrect results. More complex methods are needed for analyzing data characterized by patients clustered within centers. A hospital, ward, clinic, physician's office, neighborhood, housing complex, or family, in which patients are naturally grouped, might qualify as a center for the purposes of design and analysis. Typically, individuals within centers are more similar than those from different centers. Siblings share genes and environment more than do strangers. Patients in one hospital experience common treatment protocols delivered by shared clinicians. In statistical terms, observations within a center are correlated; those in different centers are independent. Although well known among statisticians and epidemiologists (18), this failure of a key methodologic assumption in standard statistics, the independence of observations, is overlooked in manuscripts and published articles. Authors and readers recognize the general principle of confounding by covariates (age) in the association between treatments (hormone replacement therapy) and outcomes (stroke). Effect modification (smokers might be at greater risk from oral contraceptives than non-smokers) is also a well-known feature of clinical studies. Less appreciated is the potential for study center to produce confounding or effect modification when treatments are administered across several centers. Failure to consider the center in an analysis might result in incorrect P values and confidence intervals (because of clustering), biased estimates (because of uncontrolled confounding), and unrecognized heterogeneity across centers (because of effect modification). For purposes of exposition, we use the common example of the patient as the observation and the hospital, clinic, or physician's practice as the center (Table 1). Table 1. Key Points: Effects of Ignoring Center on Study Design and Analysis Historical Development Early applications of statistical methods for clustered data arose from the specialized field of survey research (9, 10), where the practice of sampling clusters of individuals in entire households or census blocks necessitated new statistical methods. A parallel development occurred in the social sciences and education, where children are naturally grouped in classrooms (11). In 1978, Cornfield was credited with an early assessment of the implications of clustering in biomedical studies (3). Despite these developments, reviews document the inattention of published studies to departures from the assumption behind most statistical methodsindependence of observations (1, 1215). This inattention perhaps reflected the now-remedied dearth of commercially available software and the inadequate documentation of early pertinent findings from survey literature. Effects on P Values and Confidence Intervals When patients are not independent observations, P values and confidence intervals generated from standard statistical methods can be incorrect. Without an adjustment for clustering, results might appear statistically significant when they are not, or vice versa. For that reason, investigators need to identify centers, incorporate the concept of centers into their designs, estimate the design effect, and adjust confidence intervals and P values appropriately. Identifying the Center In traditional randomized multicenter studies, the identity of the centers is obvious; in other designs, it is not. For example, Dexter and colleagues (16) chose physician as the center and the physicianpatient pairs as the observation in their study of the effectiveness of computerized reminders on the quality of discussion about advance directives. This choice was reasonable because patients might have similar (correlated) success in their dialogues with their common physician. In an observational study of hand-washing behavior among clinicians at a single hospital (17), the authors selected a combination of period of observation and ward as the center and the opportunity to wash as the observation. Although the authors might have selected a different definition of center (and observation), it was reasonable to assume a common correlation of the probability of hand-washing within the ward during a shift, perhaps because of common personnel and shared time pressures. Relating the Center to the Design In traditional multicenter trials, patients are randomly assigned to different treatments within each center, and estimates of treatment effect are within-center comparisons. In a cluster-randomization design, by contrast, all the patients in a practice, hospital, organization, or entire community are assigned to the same treatment (1, 18, 19). Comparisons are effectively done among communities, although data are collected and analyzed on each person. This study design appears typically in physician-practicebased interventions, as in a comparison of individualized office systems to improve the rate of breast cancer screening (20). In many studies of computer-generated reminders, for example, the intervention group of physicians receives a reminder for all patients, whereas the control group of physicians receives no reminders for any patients (2). The intervention (the reminder) is therefore a center-level (among-center) comparison. Ethics, politics, logistics, or the need to avoid contamination of treatment arms often dictates this design. Authors should indicate the unit of randomization, patient or center, in reports of clinical trials (21). Design will therefore control whether key comparisons should be within or among centers. The Design Effect A measure of the degree of clustering is the design effect (10, 22, 23). When patients do not cluster, the design effect is 1.0 and standard statistical methods should produce correct confidence intervals. We shall first look at the design effect for estimates of the overall risk for an outcome, such as disease or adverse event. Then we shall consider the design effect in the presence of covariates, such as treatment (exposure) or other patient- or center-level factors. When no covariates are present, the design effect for an estimate of the overall risk for an outcome usually increases as the within-center correlation and the number of patients per center increase. A study with a design effect of 2.0 has double the variance and half the effective sample size of a comparable study with no clustering. Without an adjustment for clustering, confidence intervals on estimates of patient risk are too narrow. A striking example appears in the analysis by Laine and associates (24) of rates of combined (right and left) cardiac catheterization among 41 000 Medicare patients in 73 Pennsylvania hospitals. A simple estimate of the 95% confidence interval (33.7% to 34.7%) of the overall rate of 34.2% suggests great precision. The calculation assumes incorrectly that patients' outcomes are independent observations. The hospital-specific rates vary widely, however, from 2% to 98%. This variation provides strong evidence that patients at the same hospital are correlated. Two patients in one hospital are more likely to have (or not to have) a combined procedure than two patients at different hospitals. This clustering of large numbers of patients into centers inflates variance, as represented by a design effect of 370 in this example. The large variance reflects the impact of hospital selection on the result; a slightly different sample of hospitals might produce a substantially different overall rate. The study is far less powerful than a study with 41 000 independent patients would have been. The effective sample size (25) is only 111 (41 000/370). Adjusted for this correlation of patients within hospitals, the confidence interval for the overall rate becomes 25% to 43%. A simple analysis would overstate statistical precision substantially. Design effects for treatment, exposure, or other covariates are more complex. A multivariable model, with several predictors of outcome, can have several design effects, one for each treatment effect and covariate. For example, in the preceding example, the design effect was 169 for the hospital location covariate but only between 25 and 57 for the hospital size categories. The size of the design effect depends in part on whether the covariate applies to the patient or to the center. If all patients in a center receive identical treatment, as in a computerized physician reminder study, the design effect for reminders (a center-level factor) can be substantial. By contrast, in the same study, if the investigator wants to examine the effect of patient sex on outcomes, the design effect might be much closer to 1.0, especially if the distribution of patients by sex is about the same across centers. The design effect for a treatment variable also can fall below 1.0 in some instances of perfectly balanced studies, with equal proportions of patients assigned to treatment and control groups at each center (6, 26). In that case, standard statistical methods can overstate the width of confidence intervals and mask statistically significant results. Therefore, study design, the comparison of interest, and the degree of correlation of outcomes affect confidence intervals and P values. Details for computing the design effect in simple cases are given by Kerry and Bland (22, 23). One can test for the need to adjust for clustering by comparing the design effect with 1.0. In a study of the incidence of adverse drug reactions among 3137 patients admitted to 62 departments in 33 hospitals, Pouyanne and coworkers considered a design effect below 1.5 to be negligible (27). However, failure to adjust for a design effect of 1.5 means that the 95% confidence interval as computed by standard statistical p