A third dimension to uncloud the funnel plot of operator quality

Our society is demanding evidence that hospitals and individual physicians are providing high-quality care. Coronary interventions are an easily identifiable discrete episode of care, expensive, frequent, and also have some relatively well-defined outcomes compared with many illnesses and medical encounters. As cardiovascular interventionalists, therefore, our practices and outcomes will continue to receive increasing scrutiny and reporting. Surveillance programs like the State of New York’s will likely be extended nationwide and ultimately will include all endovascular interventions [1]. Kunadian et al.’s report in this issue of CCI is an effort to establish a better approach to flag hospital and operator outlier outcomes not likely due to chance alone for closer scrutiny [2]. The funnel plot concept as first well-described by Richard Light, Matthias Egger, and others was intended to detect publication and other methodological biases in meta-analyses [3,4]. Plotting the underlying trials’ effect sizes against sample size should produce a symmetric funnel-shaped plot in the absence of significant biases. This is because the precision of the estimate of the treatment effect should increase with sample size. If this pattern does not exist, then bias should be suspected. Similarly, a hospital or operator’s observed performance should approach the expected performance (i.e., treatment effect) as case volume (i.e., sample size) increases. This represents an improved method for analyzing and reporting outcomes data as a putative measure of physician and hospital performance and quality. In addition to being intuitively and visually appealing, it partially corrects for the expected increased variance from chance alone with smaller sample sizes [5]. Of course an implicit assumption is that the operator and hospital are important incremental (and perhaps even modifiable) determinants of patient outcome. Within the spectrum of US practice with the existing checks and balances, is this a valid assumption? Or is it all ‘‘the substrate’’ that the patient literally brings to the catheterization table that mainly determines our outcomes? Consider the following Bayesian paradigm to understand our daily practices. The baseline characteristics of the candidate patients for PCI we see (referred to us) creates an unknowable prior probability distribution for patient outcome. Outcomes 5 function(true patient risk). Next, we as operators assess this risk profile. This complex process is both intuitive and explicit, often in the setting of incomplete information and time constraints. A host of demographic, medical (comorbidities and competing risks) and other relevant factors (likelihood of medication compliance) form our calculus. This is more or less reflective of true risk. Operator expected outcomes 5 function (perceived riskjtrue risk) which is conditional on true unknowable risk. Proceed to PCI? The operator’s individual risk tolerance threshold then comes to bear. This may be influenced by real world secondary nonmedical factors: an unavoidable emergency case, referral patterns and pressure, how long will this procedure take, what else must I do today, will it bankrupt my hospital, what will be the medico-legal consequences of a bad outcome, etc. Realistically these factors also play a role. This risk threshold varies considerably among operators based on experience, skill, training, support systems, confidence, and general risk averseness. If function (perceived riskj true risk) < function (acceptable risk thresholdjindividual operator), then PCI is attempted. This determines the operator’s case-mix and prior baseline risk distribution or profile. Operator, team, and support care team/hospital resources and skill then act upon this prior risk distribution to determine the posterior outcome distribution. Actual outcomes 5 function (perceived riskjtrue risk) < function (acceptable risk thresholdjindividual opera-