Response to Commentaries on Silber et al.: Aggressive Treatment Styles and Surgical Outcomes

We thank Amber Barnato (Barnato 2010), and Elliot Fisher and Jonathan Skinner (Fisher and Skinner 2010) for their thoughtful comments on our paper (Silber et al. 2010). Common themes discussed by Barnato, and Fisher and Skinner concerned the use of 30-day versus 365-day mortality and the effect size associated with aggressiveness. We believe it is appropriate to focus on results from the analysis of 30-day mortality and not 365-day mortality, as do Barnato, and Fisher and Skinner when they interpret our findings. An important reason for our preference is that the 30-day window is most closely aligned to the research question of interest, which is to assess how differences in hospital treatment styles, as measured by Dartmouth end-of-life measures, affect surgical quality (see equation 2 in our paper [Silber et al. 2010]). Examining a period beyond 30 days increases the likelihood that the association between aggressiveness and surgical outcomes will be confounded by effects on survival of treatment by other providers. In addition, it is important to understand why the odds ratio associated with 365-day mortality is smaller (closer to 1) than the 30-day odds ratio (see Table 4). The crude death rates in our overall analysis went from about 4.5 percent at 30 days to 13.9 percent at 365 days; survival by elderly patients on average declined, in absolute terms, 9.4 percent between day 30 and day 365. This mortality decline, which occurred at the same rate for patients treated at both more and less aggressive hospitals, moved the odds ratio comparing patients from more aggressive to less aggressive hospitals from 0.93 in the 30- day analysis to 0.99 in the 365-day analysis. Extending this logic, we note that the odds ratio will always approach 1.0 as the time period is extended, because all patients eventually die. Suppose we had used a sample of younger patients with a lower competing risk of death. If the 30-day mortality odds ratios between more and less aggressive hospitals were the same as in our study, then the 365-day odds ratio would have been more similar to the 30-day results. In both cases more aggressiveness would have been associated with equally better quality of care. One may be tempted to conclude, as do both Fisher and Skinner and Barnato, that the 365-day mortality odds ratio indicates that there is little difference in quality between more aggressive and less aggressive hospitals. We believe this would be a mistake. The aim of surgery is to correct a problem that requires a surgical intervention while causing as little morbidity and mortality as possible. Surgery, for the procedures we studied, is not performed with the expectation that it will alter the course of one's life span beyond the scope of that surgical condition. Given that we demonstrated that the gains in 30-day survival are stable, the 30-day mortality benefit is not trivial. The attributable fraction (MacMahon and Trichopoulos 1996) of deaths is 7.5 percent for a change in aggressiveness from the 25th to the 75th percentile. In other words, moving from the 25th to 75th percentile would save 7.5 percent of deaths that occur at the lower percentile of aggressiveness. Furthermore, we have only reported these results in the middle of the distribution for aggressiveness. Larger benefits occur when comparing greater changes in aggressiveness (for a normal distribution the 25th and 75th percentiles are separated by 1.35 standard deviations, but the 5th and 95th percentiles are separated by about 3.3 standard deviations). We also want to note that Fisher and Skinner's graph of spending versus outcome (figure 1 in Fisher and Skinner 2010) is hypothetical and intended to make a point that reflects their beliefs. This figure, and much of the comment by Fisher and Skinner, has little direct bearing on our study. As for the cost-effectiveness analysis discussed by Barnato, we would point out that it is based on a mistaken premise that a U.S.$10,000 increase in Dartmouth Atlas end-of-life spending is the actual change in resource use (i.e., spending) that was associated with the improvement in surgical outcomes we observed. This is not the case. The Dartmouth Atlas measure of end-of-life spending is correlated with actual spending on surgical patients, but it is not a one-to-one mapping, so it is difficult to comment on this calculation. Fisher and Skinner argue that the real inefficiency in the U.S. health care system is the failure of providers to adopt low-cost, high-quality treatment practices. While this is an interesting argument that may or may not be valid, it is not the argument we evaluated. We examined whether surgical outcomes differed by type of hospital, as classified by Dartmouth end-of-life measures. These Dartmouth measures have been shown to be positively correlated with actual spending on surgical patients in our sample. As noted, our results indicated that being treated at a more aggressive hospital—greater end-of-life resource use—is associated with a significant and nontrivial reduction in 30-day mortality. We believe that this finding suggests that greater spending on surgical patients is relatively productive, although reasonable people can differ on exactly how valuable they believe such spending to be. Our findings reinforce the point that Fisher and Skinner make that what “… is needed is an understanding of mechanisms: It matters what you spend the money on.” Our study showed that spending money to save surgical patients may have significant benefits. The difficult task is identifying productive and unproductive spending. The Dartmouth group believes this is easy to accomplish, but we are less sure. Some readers may have thought, perhaps, because of the Dartmouth group's prior research, that more aggressive treatment of surgical patients was a waste or even detrimental (Consumer Reports Health.org 2008). Our study suggests that this is not the case. Finally, a likely point of agreement between the authors and discussants follows. It would seem that the recent federal emphasis on Comparative Effectiveness research is required if we are to have any chance of achieving a careful reduction in resource use that maintains the quality of our health care.