Breast-conservation therapy involves a number of physician decisions not required for mastectomy, including the selection of patients for breast conservation, the amount of tissue to be removed from the area surrounding the tumor, the details of administering radiation, and so forth. (See Sacks and Baum, 1993; Winchester and Cox, 1992; Harris et al., 1990; NIH, 1991.) And since breast-conservation therapy involves radiation, its implementation would logically vary depending upon the availability of appropriate radiation equipment and expertise in operating that equipment. Breast-conservation therapy also requires “careful long-term breast monitoring” in order to identify and treat local recurrences in the breast that was subjected to lumpectomy (NIH, 1991). All these treatment-implementation factors can potentially affect breast-conservation patients’ survival—and may not be the same in randomized studies and in medical practice. At least, the typical treatments given in day-to-day medical practice could fall short of the presumably consistent and high-quality treatments provided by a single prestigious research center, such as the National Cancer Institute (NCI). Some randomized studies are conducted at single centers, while others are conducted at diverse sites (that is, multiple centers). To more closely approximate day-to-day medical practice, multicenter studies have, in some instances, intentionally involved “community surgeons.” For this reason—and also because the treatments given in multicenter studies may vary from one center to another—multicenter studies’ results may more closely approximate results in medical practice than the results of single-center studies at prestigious institutions. But unlike medical practice, both single-center and multicenter studies stipulate that participating physicians follow a set of prespecified procedures. The question remains, then, as to whether or not breast conservation therapy has produced results similar to mastectomy in day-to-day medical practice. Randomized clinical studies are the “gold standard” of medical research. Random assignment essentially equates patients in the two treatment groups. Because the two groups should not differ on variables related to cancer survival, their outcomes can be directly compared, and any difference in survival can be attributed to the difference in treatment. In contrast, the statistical analysis of cases from a medical practice database represents a potential “window” on how well breast-conservation therapy has, in fact, worked in community medical practice. But the results of such analyses may be less conclusive because of their vulnerability to hidden selection bias. (See Byar, 1980; Office of Technology Assessment, 1994.) Briefly, in day-to-day medical practice, patients and physicians freely choose between treatments; a database analyst must, therefore, attempt to control for the potentially differing characteristics of patients who received breast-conservation therapy and those who received mastectomy. In this report, we have made all possible efforts to minimize the impact of selection bias, as described below. The analyses presented here are based on a unique combination of meta-analysis (to summarize randomized studies’ results), statistical analysis of records from a medical practice database, and cross design comparison of results. To our knowledge, this is the first time such an approach has been used in the area of breast cancer treatment. In all analyses presented here, breast-conservation therapy is defined as including lumpectomy, nodal dissection, and radiation. With respect to time frame, the randomized studies enrolled and treated patients from 1972 to 1989, and the medical practice cases selected for this analysis were diagnosed from 1983 to 1985. Because of limitations in the medical practice database (discussed in appendix I), all our analyses use the outcome criterion of 5-year survival and examine node-negative patients only. The medical practice data were drawn from the National Cancer Institute’s Surveillance, Epidemiology, and End Results (SEER) database. SEER archives records for almost all cancer patients residing in five states— Connecticut, Hawaii, Iowa, New Mexico, and Utah—and four metropolitan areas—Atlanta, Detroit, San Francisco-Oakland, and Seattle -Puget Sound. (See Hankey et al., 1992.) Our analysis consisted of three major steps. In step 1, we performed a meta-analysis to summarize randomized studies’ results and obtain summary figures that can be compared to medical practice results. We conducted meta-analyses separately for the single-center studies and for the more generalizable multicenter studies to determine if similarity of survival following breast-conservation therapy and mastectomy holds for both kinds of studies. In step 2, we obtained information on the survival of breast-conservation and mastectomy patients in day-to-day medical practice. Specifically, from the SEER database, we drew records for a relatively homogeneous set of patients who, on the basis of several characteristics, were comparable to those enrolled in randomized studies. For this group of SEER patients, we conducted an analysis of survival following breast-conservation therapy and mastectomy. SEER results were adjusted for tumor size and several other variables so that patients who had received breast-conservation therapy would be “matched” to those who had received mastectomy. The matching was intended to minimize the effects of differing characteristics of patients who received breast-conservation therapy and mastectomy. In addition, a sensitivity analysis was performed to check for selection bias on life-threatening factors unrelated to cancer (such as heart-disease). In step 3, we compared (1) the summary results for the single-center and multicenter randomized studies to (2) the results of our analysis of cases selected from the SEER medical practice data. We also considered the logic of our analyses and, in particular, whether the resulting evidence was sufficient to conclude that—in day-to-day medical practice— breast- conservation therapy has been followed by survival similar to that observed for mastectomy. Throughout step 3, we drew on the principles of “cross design synthesis.” In this report, we use the term “similar” when the observed difference between the survival rates (1) is not statistically significant and (2) has an absolute value of less than 1.5 percentage points. Conversely, when a comparison of survival rates shows a difference of 1.5 percentage points or larger—and that difference is also statistically significant—we state that one rate is higher (or lower) than the other. Step 1 (the analysis of randomized studies) began with the identification of relevant single-center and multicenter studies through bibliographic searches and a survey of U.S. breast cancer researchers. Our inclusion criteria were as follows: randomization of enrolled patients to alternative treatments—breast-conservation therapy or mastectomy; breast-conservation therapy that included lumpectomy, nodal dissection, and radiation; no confounding treatments (such as the administration of an additional therapy to one treatment group); availability of 5-year survival rates by treatment group among node-negative patients (either previously published in a scholarly research journal or provided at our request); and published in English (if a non-U.S. study). Six studies—three single-center and three multicenter studies—met these criteria. (See table 1.) Almost 2,500 node-negative breast cancer patients were enrolled and treated in these randomized studies. The treatment effect—that is, the effect of breast-conservation therapy relative to mastectomy—is represented by a comparison of survival following breast-conservation therapy to survival following mastectomy. (See table 2.) by subtracting the 5-year survival rate for mastectomy patients from the 5-year survival rate for breast-conservation patients to determine the difference between the rates; and by calculating the odds ratio (dividing the odds of surviving with breast-conservation therapy by the odds of surviving with mastectomy). As indicated in table 2, the breast-conservation and mastectomy treatment groups experienced similar survival rates in each of the studies, and the odds ratios are close to 1 (the point of equivalence). The confidence intervals for the odds ratios all overlap 1, indicating no statistically significant difference in survival odds for the two treatments. However, the confidence intervals surrounding these estimates are quite broad, indicating a lack of precision in the individual-study estimates. (The U.S.-NSABP figures in table 2 are taken from recalculations published by an NCI contractor in March 1994. The recalculations were published following charges of fraudulent data collection at one U.S.-NSABP center; they exclude the data from that center.) A meta-analysis combining the results for node-negative patients across studies gives more precise estimates of the treatment effect. Table 3 shows meta-analysis results summarizing the treatment effect for single-center studies, multicenter studies, and both types of studies taken together. In addition, table 3 shows meta-analysis results calculated in two ways: (1) including the U.S.-NSABP recalculations published in March 1994 and (2) omitting U.S.-NSABP results entirely. Similar rates of 5-year survival characterized the breast-conservation therapy and mastectomy groups—not only in single-center studies (93.7 percent for breast-conservation patients and 93.7 percent for mastectomy patients), but also in multicenter studies, which may more closely approximate medical practice (88 percent for breast conservation and 88 percent for mastectomy, omitting U.S.-NSABP). Again, the odds ratios are close to 1, and the confidence intervals all overlap 1, indicating no statistically significant difference for any group of randomized studies. Finally, referring again to tables 2 and 3, the 5-year survival rates appear to be higher in single-center studies than in multicenter studies. This could be because of more effective treatments in single-center studies, varying tumor-size limits across the studies, or hidden cross-study differences in patient prognoses prior to treatment. (Step 3 presents more precise comparisons of combined-treatments survival rates.)