Towards improved risk scores: the quest for the grail continues Non-compaction cardiomyopathy with diffuse left coronary artery ﬁstulae as a rare cause of congestive heart failure

Barili et al. have reported the first external validation of the recently published EuroSCORE II scoring system. A new scoring system was needed to replace the original logistic EuroSCORE, not only because it was elaborated . 15 years ago, but, more importantly, because a number of papers showed that the EuroSCORE system was poorly calibrated when applied to contemporary data sets. This means that there were discrepancies between predicted and observed operative mortality, with a trend to overestimate the operative risk. The EuroSCORE II system was recently shown to achieve a similar discrimination to the original EuroSCORE, but to be better calibrated. In the initial study, validation was performed using a different data set from the one used to elaborate the scoring system, but this was not an independent validation since the validation and derivation samples were produced from the same data collection and from the same centres. The main conclusions of the external validation are that the EuroSCORE II system achieves a similar discrimination to EuroSCORE, but not a better calibration. The strength of the study by Barili et al. is that the authors have performed an external validation in a large data set prospectively collected from different types of hospitals. The discriminatory properties of the EuroSCORE system have never been a major source of concern. The area under the receiver operating characteristic (ROC) curve was 0.82 for EuroSCORE II in the external validation and 0.81 in the internal validation. These values are close to the discrimination obtained with the original EuroSCORE and with the Society for Thoracic Surgeons (STS) score. Therefore, the fact that discrimination does not appear to be improved with the EuroSCORE II system is not a drawback in itself. On the other hand, external validation leads Barili et al. to conclude that the calibration properties of EuroSCORE II do not seem to be significantly improved as compared with EuroSCORE. This statement seems to go too far given the results of the external validation. The statistical significance of goodness-of-fit tests, such as the Hosmer–Lemeshow test, actually means that there are significant differences between the numbers of observed and predicted deaths. However, discrepancies between observed and predicted deaths in a particular subgroup of patients, such as high-risk patients, may lead to statistically significant differences of the overall test, in particular when using large data sets with a large number of events. Therefore, the P-value of 0.001 in the external validation does not summarize, per se, all of the calibration properties of the EuroSCORE II system. Rather than overall statistical tests, the graphical representation of observed vs. predicted mortality is of great interest to assess the usefulness and limitations of a scoring system in clinical practice. As pointed out by the authors, the graphical representation shows that EuroSCORE constantly overestimates the operative mortality, whatever the risk considered. This is not fully in accordance with previous analyses which showed that the miscalibration of EuroSCORE was mainly observed in high-risk patients. –7 On the other hand, there is a good agreement between observed and predicted operative mortality with the EuroSCORE II system when considering patients with a predicted operative mortality of ,30%. This is of considerable interest since patients with a predicted operative mortality of ,30% account for most patients who are operated on. Unfortunately, the distribution of operative risk and the detailed description of the population are not provided in this validation sample. In the EuroSCORE II population, observed operative mortality was 18% (280/1595) in the highest risk decile. Therefore, the external validation suggests that EuroSCORE II leads to a much more accurate estimation of operative mortality than EuroSCORE for the vast majority of patients. Nevertheless, a poor calibration and an overestimation of the operative risk are still observed with EuroSCORE II in high-risk patients. This has an important impact in practice since risk assessment has recently gained importance in this particular subgroup with the development of percutaneous treatment of valvular heart disease.