ORIGINAL ARTICLE

European Journal of Cardio-Thoracic Surgery 41 (2012) 734–745
doi:10.1093/ejcts/ezs043

EuroSCORE II†
Samer A.M. Nashef a,*, François Roques b, Linda D. Sharples c, Johan Nilsson d, Christopher Smith a,
Antony R. Goldstone e and Ulf Lockowandt fa

Papworth Hospital, Cambridge, UK
University Hospital Centre (CHU), Fort de France, Martinique, France
Medical Research Council, Biostatistics Unit, Cambridge, UKdDivision of Cardiothoracic Surgery, Skåne University Hospital, Lund, SwedeneDepartment of Radiology and Nuclear Medicine, Castle Hill Hospital, Hull, UKf
Karolinska Hospital, Stockholm, SwedenbcReceived 13 October 2011; received in revised form 5 January 2012; accepted 6 January 2012

Abstract
OBJECTIVES: To update the European System for Cardiac Operative Risk Evaluation (EuroSCORE) risk model.
METHODS: A dedicated website collected prospective risk and outcome data on 22 381 consecutive patients undergoing major cardiac
surgery in 154 hospitals in 43 countries over a 12-week period (May–July 2010). Completeness and accuracy were validated during
data collection using mandatory ﬁeld entry, error and range checks and after data collection using summary feedback conﬁrmation by
responsible ofﬁcers and multiple logic checks. Information was obtained on existing EuroSCORE risk factors and additional factors
proven to inﬂuence risk from research conducted since the original model. The primary outcome was mortality at the base hospital.
Secondary outcomes were mortality at 30 and 90 days. The data set was divided into a developmental subset for logistic regression
modelling and a validation subset for model testing. A logistic risk model (EuroSCORE II) was then constructed and tested.
RESULTS: Compared with the original 1995 EuroSCORE database (in brackets), the mean age was up at 64.7 (62.5) with 31% females (28%).
More patients had New York Heart Association class IV, extracardiac arteriopathy, renal and pulmonary dysfunction. Overall mortality was
3.9% (4.6%). When applied to the current data, the old risk models overpredicted mortality (actual: 3.9%; additive predicted: 5.8%; logistic
predicted: 7.57%). EuroSCORE II was well calibrated on testing in the validation data subset of 5553 patients (actual mortality: 4.18%;
predicted: 3.95%). Very good discrimination was maintained with an area under the receiver operating characteristic curve of 0.8095.
CONCLUSIONS: Cardiac surgical mortality has signiﬁcantly reduced in the last 15 years despite older and sicker patients. EuroSCORE II isbetter calibrated than the original model yet preserves powerful discrimination. It is proposed for the future assessment of cardiac surgical risk.
Keywords: Risk assessment • EuroSCORE • Cardiac surgery • Mortality

INTRODUCTION
The European System for Cardiac Operative Risk Evaluation [1](EuroSCORE) is a cardiac risk model for predicting mortality after
cardiac surgery. It was published in 1999 and derived from an international European database [2] of patients who had undergone
cardiac surgery by the end of 1995. The system has been highly successful and used worldwide both for the measurement of risk and asa benchmark for the assessment of the quality of cardiac surgical services, with more than 1300 formal citations in the medical literature.
Over the last few years, several professionals from many partsof the world [3–9] have published evidence that the model now
overpredicts risk as the results of cardiac surgery have substantially improved with a sustained reduction of risk-adjusted mortality, so that the model may now be inappropriately calibrated
for current cardiac surgery.
†
Presented at the 25th Annual Meeting of the European Association for
Cardio-Thoracic Surgery, Lisbon, Portugal, 1–5 October 2011

Despite the calibration problem, both additive [1] and logistic[10] versions of the model have remained powerfully discriminatory with an area under the receiver operating characteristic(ROC) curve of around 0.75–0.8. Nevertheless, there is someevidence that discrimination may be improved further by reﬁning and modifying some of the risk factors and the way themodel handles them, such as renal dysfunction [11, 12].
The purpose of this study was to renew EuroSCORE in orderto maintain and optimize its usefulness in contemporary cardiac
surgical practice.

METHODS
Recruitment
Using journals, conferences, articles and presentations as well as
the existing www.EuroSCORE.org website, cardiac surgical units© The Author 2012. Published by Oxford University Press on behalf of the European Association for Cardio-Thoracic Surgery. All rights reserved.

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

* Corresponding author. Papworth Hospital, Cambridge CB23 3RE, UK. Tel: +44-1480-364299; e-mail: sam.nashef@papworth.nhs.uk (S.A.M. Nashef ).

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery

735

Argentina
Austria
BelarusBelgium
BosniaBrazilCanadaChina
Croatia

1
2
1
8
1
4
2
2
2

Denmark
FinlandFranceGermany
GreeceHolland
Hungary
India
Ireland

2
4
16
9
2
6
1
4
1

IsraelItaly
Japan
Lithuania
Montenegro
New Zealand
NorwayPolandPortugal

The data set
Review of the literature and of feedback received from manyusers of the logistic EuroSCORE identiﬁed the following areas forpotential improvement:• Creatinine clearance (CC) is a better predictor than absolute
serum creatinine.

• Hepatic function is not represented.
• Deﬁning unstable angina by the use of intravenous nitrates isout of date.

• Some continuous variables are treated as dichotomous(number of previous heart operations, serum creatinine,
pulmonary artery pressure).
• The model is not sufﬁciently sensitive to the ‘weight’ of theintervention.
A new set of risk factors was assembled to include the original
EuroSCORE variables modiﬁed or complemented to takeaccount of the above areas. The risk factor information collected
is given in Table 2.

RussiaSaudi Arabia
SerbiaSlovenia
South Africa
Spain
Sudan
SwedenTable 2:

Data set

3
2
4
1
1
19
1
5

Switzerland
Syria
TaiwanTurkeyUAE
UK
Uruguay
USA

2
1
1
1
1
12
1
3

Patient-related factors
Age and sex
Height and weight
Pulmonary disease
Diabetes status
Extracardiac arteriopathy
Neurological or musculoskeletal dysfunction
On dialysis
Last serum creatinine
Brain-natriuretic peptide
Serum albumin
Cardiac-related factors
Symptomatic status
NYHA
CCS
LV function
Recency and size of last myocardial infarct
Systolic PA pressure
Active endocarditis
Previous cardiac surgery
Operation-related factors
Urgency
Elective
UrgentEmergency
SalvageType of procedure(s) performed in detailTimes of
Bypass
Cross-clamp
Deep hypothermic arrest
Selective cerebral perfusion

Data collection
Data were collected from consecutive patients operated over a
12-week period (3 May–25 July 2010, inclusive) and entered into
the web database. Records could be opened at the beginning of
the data collection period and held as ‘pending’ while additional information on procedures and outcomes was obtained. Most units
submitted data on line, either contemporaneously or after completing a paper dataform. Three units sent data as spreadsheets, and theEuroSCORE project team entered these on their behalf. Ninety days
after the last eligible operation, units with pending records were
urged repeatedly to complete their data. The last did so by May
2011 and the data set was ﬁnally closed to data entry.
Once data collection was completed, the responsible personfor every unit was asked to conﬁrm on the website itself and byemail that the data provided were an accurate and complete
representation of the unit’s entire cardiac surgical activity during

the study period. Where there was a major discrepancy between
a unit’s projected activity on registration and actual data
received, the responsible ofﬁcer was asked to provide an explanation. Units which could not provide a satisfactory explanationor those with major ﬂaws in inadequate or missing data were
removed from the study.

Data preparation
The data set was downloaded directly from the database into a
spreadsheet and contained 24 385 records. Centres with few
observations, incomplete or duplicate data entry were dropped,leaving 23 451 cases. Cases from the same country with samedate of birth, sex, date of operation, height and weight wereassumed to be duplicate records and only one of the records was

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

worldwide were invited to participate in data collection for the
project. Expressions of interest were received on the EuroSCORE
website from 520 individuals, all of whom were given further
detailed information on participation and invited to register. This
resulted in 214 units registering to take part. Of these, 160 units from44 countries submitted data and 154 successfully completed datacollection. The distribution of the units by nation is given in Table 1.

1
15
3
1
1
1
1
1
4

ADULT CARDIAC

Table 1: Participating countries (43) and number of units (154)736

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery

Table 3: EuroSCORE II demographics and comorbidity(n = 22 381)
Variable

Frequencies (%) ormean (SD) [range]

64.6 (12.5) [18–95]
6919 (30.9%)
77.9 (15.9) [30–182]
168.5 (9.6) [100–213]
27.4 (4.8) [9.6–82.6]
1.87 (0.21) [1.04–2.90]
16 783 (75.0%)803 (3.6%)3103 (13.9%)
1705 (7.6%)
2384 (10.7%)713 (3.2%)
96.4 (57.1)
1.13 (0.92)562 (2.6%)
83.6 (50.9)
244 (1.1%)
31.6 (19.0)
497 (2.2%)
924 (4.1%)
137 (0.6%)
94 (0.4%)
251 (1.1%)475 (2.1%)
384 (1.7%)
108 (0.5%)

Figure 1: Downloaded data and ﬁnal data set for analysis.

retained in the ﬁnal data set. Duplicates were identiﬁed for 136
patients, resulting in 147 records being removed (129 wereentered twice, 4 were entered three times, 2 were entered fourtimes and 1 was entered ﬁve times). Two cases were known to betransplants and were removed. Of the 23 302 cases that remained,
295 (1.2%) did not have survival status at discharge or 90 days andwere dropped. Additionally, because we had pending records inthe data set there were 418 with at least one missing required
ﬁeld, mostly musculoskeletal dysfunction (n = 159, all one centre),
Canadian Cardiovascular Society (CCS) angina class (n = 204, allin two centres) and New York Heart Association (NYHA) class(n = 30, all in one centre). After checking that these were either ina small number of centres or related to a time period rather thansporadically missing, we concluded that they could be assumed‘missing completely at random’ and so excluded these cases fromthe analysis; this left 22 589 cases. Finally, we decided to exclude
208 transcatheter aortic valve implant (TAVI) procedures from theanalysis with a view to a separate study in the future. Thus, theﬁnal data set contained 22 381 cases (Fig. 1) from 154 centres in
43 countries (Table 1).
At the completion of data collection, logic checks were carriedout on all data. The data set prepared for the analysis was therefore
excellent in quality and completeness, rivalling if not exceeding thestandards set by the original EuroSCORE database of 1995.

AnalysisTables 3 and 4 summarize patient proﬁle and procedures performed. Body mass index (BMI) was calculated using the formula

VT: ventricular tachycardia; VF: ventricular fibrillation; IABP:
intra-aortic balloon pump.

BMI = weight (kg)/ height2 (m2). CC as an estimate of glomerular
ﬁltration rate was calculated using the Cockcroft–Gault formula:CC ðml=minÞ ¼

ð140  age ðyearsÞÞ  weight ðkgÞ  0:85 ðif femaleÞ
72  serum creatinine ðmg=dLÞ

Our aim was to construct a model that could be applied very
widely and that could be incorporated into local data collection
and management systems. Therefore, a parsimonious approach
to model ﬁtting was adopted. To this end, the data were divided
into a developmental data set (16 828 patients) and a validation
data set (5553 patients) using random sampling from a binomial
distribution with a probability 0.25. Initially, a series of singlevariable logistic regression models was ﬁtted to the developmental data set in order to identify variables that were associatedwith mortality. The variables are listed in Table 5. These models
were used to assess the nature of associations between predictors and risk of death, such as linear associations and thresholdmodels, as well as the optimal grouping of categorical variables.
Of these variables, 14 were considered compulsory for the ﬁnal
model [age, sex, extracardiac arteriopathy, chronic lung disease,poor mobility, previous cardiac surgery, CC, active endocarditis,
critical preoperative state, left ventricular (LV) function, systolicpulmonary artery pressure, urgency and weight of procedure].
Additional variables were included in the model using forward
selection based on likelihood ratio statistics comparing models

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

Patient-related factors
Age (years)
FemaleWeight (kg)Height (cm)BMI (calculated) (kg/m2)
Body surface area (calculated)
Diabetes—noDiet only
Oral therapy only
InsulinPulmonary disease
Neurological dysfunction
Serum creatinine (μmol/l)Serum creatinine (mg/dl)
Serum creatinine > 200 μmol/l
CC (calculated)
On dialysis
Serum albumin (g/l)
Active endocarditis
Critical preoperative statePre-op VT/VF or aborted sudden deathPre-op cardiac massage
Pre-op ventilation
Pre-op inotropes
Pre-op IABP
Pre-op acute renal failureTable 4: EuroSCORE II: types of procedure (n = 22 381)

17 165 (76.7%)4135 (18.5%)
972 (4.3%)
109 (0.5%)
10 448 (46.7%)
10 353 (46.3%)
269 (1.2%)
6753 (30.2%)
971 (13.8%)
1534 (21.7%)4545 (64.4%)
1935 (8.7%)2049 (9.2%)
534 (13.3%)
2901 (72.4%)
568 (14.2%)
1031 (4.6%)
79 (0.4%)
10 (0.04%)
46 (0.2%)
1636 (7.3%)
1100 (4.9%)
492 (2.2%)
141 (0.6%)
47 (0.2%)
42 (0.2%)
22 (0.1%)
106 (0.5%)64 (0.3%)
130 (0.6%)

with and without each variable and Akaike’s information criterion (AIC), until well-ﬁtting models were developed. A likelihood
ratio statistic with P < 0.05 or a change in AIC of at least 10 led toinclusion of a new variable. Due to the colinearity between
some covariates, there were several models that ﬁtted the dataand gave similar prediction. The ﬁnal model, named EuroSCORE
II, was chosen on the basis of clinical face validity (reﬂecting
current knowledge in the ﬁeld of cardiac surgery) and predictive
accuracy (maintaining the area under the ROC curve at 80% ormore). The logistic equation used wasPpredicted mortality ¼

e

ðb0 þ

bi Xi ÞP;
1 þ eðb0 þ bi Xi Þ

where β0 is the constant of the logistic regression equation =−5.324537, βi the coefﬁcient of the variable Xi, for age, Xi = 1 ifpatient age ≤60; Xi increases by one point per year thereafter(age 60 or less Xi = 1; age 61 if Xi = 2; age 62 if Xi = 3 and so on).
EuroSCORE II risk factors and their coefﬁcients are detailed inTable 6.
The model was then tested on the validation data set for calibration (by comparing the observed and predicted mortality)and for discrimination (using the area under the ROC curve).
Goodness of ﬁt of the ﬁnal model was tested using the Hosmer–
Lemeshow statistic. In addition, a range of model diagnostics
were employed to assess the validity of the model.

(i) In order to accommodate the hierarchical structure of thedata ( patients are clustered within hospitals), hospitals wereincorporated into the model as random intercepts. Although
there was some evidence of inter-hospital heterogeneity, it
was small and the random effects model did not improveprediction substantially. Therefore, we retained the ﬁxed
effects model for prediction.
(ii) Centres with missing outcome data were studied in depth toassess mechanisms and the effect on risk predictions. No
mechanism for missing outcomes was identiﬁed but 179 of
295 cases (61%) came from seven centres; no other centre
had more than nine cases with missing outcomes. Exclusionof these centres did not affect predictions.
(iii) We conducted 10-fold cross-validation by dividing the data
set into 10 equally sized samples at random, reﬁtting themodel to each of the 10 sets comprising 90% of the data,
calculating the area under the ROC curve for the unused
10% in each case and averaging over 10 areas under the
ROC curves. The resulting areas ranged from 0.77 to 0.83,
with an average of 0.80, very similar to that of the validation
set in our original analysis.
(iv) Graphical methods included examination of the effects ofleaving out cases with particular covariate patterns on themodel coefﬁcients, the model χ 2-statistics and the inﬂuence
statistics. On the basis of these plots, we were not able toidentify those variables that further discriminate between
cases with the same covariate pattern in our ﬁnal model.

RESULTS
Deﬁnition of mortality
For such a clear-cut binary outcome measure, the deﬁnition ofan early or operative death remains contentious. It could bedeﬁned as any of the following:• death in the same hospital as the operation took place, before
discharge from hospital;• death in the same hospital or at another hospital but before
discharge from hospital;• death within 30 days of surgery regardless of location;• death within 90 days of surgery regardless of location;• a compound of some or all of the above.
The selection of an appropriate and practical deﬁnition is an
important part of this project and that was the rationale forseeking the status of patients on discharge from the base hospital, at 30 days and at 90 days when this information is available. All units were able to supply data on status at discharge,
but not all units were able to provide data on 30-day and90-day status, so that we only have data on 56.6% of the patients
at 30 days and 44.4% of patients at 90 days.
An analysis of units which were able to provide 30-day status
data shows that of 12 673 patients, 12 164 patients weredischarged alive, of whom 79 patients had died by 30 days. Of
12 160 patients who are alive at 30 days, 75 went on to die
before hospital discharge. This means that in this group, hospitalmortality (4.015%) and 30-day mortality (4.048%) are virtually
identical although the overlap is incomplete. Combining the tworaises the mortality to 4.63% so that the additional post-discharge‘drop-off’ rate at 30 days is 0.615%.

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

Urgency of operation
Elective
UrgentEmergency
Salvage
CABG (isolated)
Valve procedures
Aortic valve
RepairReplacement
Regurgitation and stenosis
Mostly regurgitation
Mostly stenosis
Mitral valve
RepairReplacement
Regurgitation and stenosis
Mostly regurgitation
Mostly stenosis
Tricuspid valve
RepairReplacement
Pulmonary valve
RepairReplacement
Thoracic aortic surgery
Ascending aortic replacement
Root replacement with coronary reimplantation
Partial aortic arch replacement
Total aortic arch replacement
Descending aortic replacement
Thoracoabdominal aortic replacement
Other procedure on the thoracic aorta
Pericardiectomy
Other major heart procedure

737

ADULT CARDIAC

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery

738

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery

Table 5: Variables associated with mortality
Variable

AIC, P-valuea

0.0486477
0.3951562
0.7637420
0.4544856
0.7644773
1.2818960
1.5384690
0.0138048
0.2218056
0.7177771
1.2135250
1.8226770
1.6887740
0.6674962
1.2033870
1.4029890
2.1827250

5427.836, P < 0.0001
5498.874, P < 0.0001
5465.051, P < 0.0001
5506.220, P = 0.0001
5499.414, P < 0.0001
5402.522, P < 0.0001
5171.137, P < 0.0001
5095.174, P < 0.0001
5079.254, P < 0.0001

0.8217379
0.0777918
0.7037355
1.9128670
0.4626558
1.4371450
0.4626558
1.5041660
1.6481420
0.2863484
1.4105750
0.7201059
0.1647881
0.7566437
0.8295933
1.8999760
2.9450770
0.7193801
0.8267812
1.1779710
Insufficient cases
Baseline
1.1809770
0.6245959
Baseline
0.2216732
0.8473152
1.2831780

5015.454, P < 0.0001
5501.826, P < 0.0001
5465.037, P < 0.0001
5189.438, P < 0.0001
5470.678, P < 0.0001
5250.462, P < 0.0001

5382.789, P < 0.0001
5266.459, P < 0.0001

5458.959, P < 0.0001
5507.010, P = 0.0001
5506.203, P = 0.0002

5216.267, P < 0.0001

5447.145, P < 0.0001
5477.658, P < 0.0001
5481.659, P < 0.0001
5429.399, P < 0.0001

5327.226, P < 0.0001a
AIC is Akaike’s information criterion and assesses the fit of the model; lower values indicate better fit. AICs are only comparable if models use the samedata, i.e. the same cases. P-values are from likelihood ratio tests.
b
Age is less continuous in the logistic model and is set to 1 for ages ≤60, increasing by 1 thereafter; in the additive model score 1 point per 5 years or part
thereof above 60 years.

An analysis of units which were able to provide 90-day status
data shows that of 9939 patients, 9464 were discharged alive, ofwhom 155 had died by 90 days. Of 9309 patients who are alive at
90 days, 31 went on to die before hospital discharge. This meansthat 90-day mortality (6.023%) is higher than hospital mortality(4.779%). Combining the two raises the mortality to 6.34%, so thatthe additional post-discharge ‘drop-off’ rate at 90 days is 1.56%,
or a further 0.946% mortality over and above the 30-day rate.
In summary, therefore, when hospital mortality is around 4%,
adding 30-day mortality increases it by 0.6% and adding
90-day mortality increases it further by 0.9%.

It can be safely assumed that this level of data availability
from units participating in the EuroSCORE project is at least
representative and may even exceed that of cardiac surgicalunits globally so that, as things stand, we can expect only
about half of the units to have ready access to 30-day and90-day survival status data. Thus, despite the well-known
advantages of 30-day and 90-day criteria [13], the only practical outcome measure that can be used in the current statusof data availability to participating units must pragmatically bedeath in the hospital where the operation took place. This
will therefore be the outcome measure used for the

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

Patient-related factors
Ageb
FemaleExtracardiac arteriopathy
Pulmonary disease
Neurological or musculoskeletal dysfunction
Previous cardiac surgery
Serum creatinine > 200 (n = 16 201)
Serum creatinine (per μmol/l up to 200) (n = 16 201)
Serum creatinine > 90–110 μmol/lSerum creatinine > 110–130 μmol/lSerum creatinine > 130–200 μmol/lSerum creatinine > 200 μmol/l (n = 16 201)
CC ≤ 50
CC > 50–85 (n = 16 201)
On dialysis
Active endocarditis
Critical preoperative state
Cardiac-related factors
CCS angina class = 4
NYHA class II
NYHA class III
NYHA class IV
LVEF 30–50%
LVEF < 30%
LVEF 30–50%
LVEF 20–29%
LVEF < 20 (n = 16 614)
Myocardial infarct in previous 4–91 days
Myocardial infarct in previous 0–72 hSystolic pulmonary pressure > 60 mmHg
Systolic pulmonary pressure 20–60 mmHg
Systolic Pulmonary Pressure > 60 mmHg
Operation-related factors
Urgent operation
Emergency
Salvage
Other than isolated coronary surgery
Thoracic aortic surgery
Aortic arch surgery
Postinfarct ventricular septal rupture
Isolated CABG
Thoracic aortic surgery
Everything else
Isolated CABG
Single non-CABG procedure
Two procedures
Three or more procedures

Univariable logistic model coefficients

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery

739

Risk factor

Standard errorz

P ≥ |z|[95% confidence interval]

0.1070545
0.2958358
0.5597929
0.2226147
0.3542749
0.0285181
0.2196434
0.5360268
0.1886564
0.2407181
01.118599

0.1463849
0.141466
0.1697565
0.1462888
0.145863
0.0065954
0.0953505
0.1106046
0.1232126
0.1729494
0.1226272

0.73
2.09
3.30
1.52
2.43
4.32
2.30
4.85
1.53
1.39
9.12

0.465
0.037
0.001
0.128
0.015
0.000
0.021
0.000
0.126
0.164
0.000

[−0.1798547, 0.3939637][0.0185674, 0.5731042][0.2270763, 0.8925095][–0.0641061, 0.5093356][0.0683887, 0.6401611][0.0155914, 0.0414448][0.0327599, 0.4065269][0.3192458, 0.7528079][−0.0528358, 0.4301486][−0.0982564, 0.5796927][0.8782539, 1.3589440]

0.6421508
0.8592256
0.303553
0.6194522
1.086517

0.3083468
0.1446758
0.1240518
0.2046001
0.147657

2.08
5.94
2.45
3.03
7.36

0.037
0.000
0.014
0.002
0.000

[0.0378021, 1.2464990][0.5756663, 1.1427850][0.0604159, 0.5466901][0.2184433, 1.0204610][0.797115, 1.3759200]

0.3150652
0.8084096
0.9346919
0.1528943

0.1036182
0.1498233
0.2917754
0.136257

3.04
5.40
3.20
1.12

0.002
0.000
0.001
0.262

[0.1119773, 0.5181530][0.5147614, 1.1020580][0.3628227, 1.5065610][−0.1141646, 0.4199531]

0.1788899
0.3491475

0.1266713
0.1676641

1.41
2.08

0.158
0.037

[−0.0693812, 0.4271611][0.0205318, 0.6777632]

0.3174673
0.7039121
1.362947

0.1174178
0.1719835
0.33706

2.70
4.09
4.04

0.007
0.000
0.000

[0.0873326, 0.5476020][0.3668306, 1.0409940][0.7023221, 2.0235730]

0.0062118
0.5521478
0.9724533
0.6527205−5.324537

0.1463574
0.1268137
0.1463969
0.221183
0.1682446

0.04
4.35
6.64
2.95
−31.65

0.966
0.000
0.000
0.003
0.000

[−0.2806434, 0.2930670][0.3035975, 0.8006980][0.6855206, 1.2593860][0.2192097, 1.0862310][−5.65429, −4.9947830]

NYHA: New York Heart Association; CCS: Canadian Cardiovascular Society; IDDM: insulin-dependent diabetes mellitus; ECA: extracardiac arteriopathy;
CPD: chronic pulmonary dysfunction; N/M mob: neurological or musculoskeletal dysfunction severely affecting mobility; Redo: previous cardiac surgery;
CC: creatinine clearance; AE: active endocarditis; Critical: critical preoperative state; LV: left ventricle; MI: myocardial infarction; PA: pulmonary artery;
CABG: coronary artery bypass grafting. Weight of procedure ‘1 non-CABG’: single major cardiac procedure which is not isolated CABG; 2: two major
cardiac procedures; 3+: three or more major cardiac procedures. For age, Xi = 1 if patient age ≤60; Xi increases by one point per year thereafter (age 60 orless Xi = 1; age 61 if Xi = 2; age 62 if Xi = 3 and so on).

remainder of this article and for applications of the risk
model in the foreseeable future.

Risk-adjusted mortality
A total of 22 381 patients were included in the study from 154
units in 43 countries. In comparison with the original 1995
EuroSCORE database (in brackets), the mean age was up at 64.7
(62.5) with 31% females (28%) and more patients had NYHA
class IV, extracardiac arteriopathy and renal and pulmonary dysfunction. Overall mortality was 3.9% (4.6%), thus mortality is currently lower than in 1995 despite a worsening risk proﬁle. Whenapplied to the current data set, the original additive EuroSCORE
predicted a mortality of 5.8% and the logistic 7.57%. This meansthat the current risk-adjusted mortality ratio (RAMR = observed/
predicted) for the previous additive model is 0.67 and for theprevious logistic model is 0.53. This conﬁrms that the original
EuroSCORE is now no longer appropriately calibrated and shows

the substantial net improvement in cardiac surgical outcomes
since 1995 with risk-adjusted mortality falling by nearly half.
Despite this, both the old logistic and additive EuroSCORE
models retain very good discrimination, with an area under the
ROC curve of 0.7896 for the logistic model and 0.7894 for the
additive model (Fig. 2A and B).

Risk factors for the new model
Univariate regression analysis demonstrated that a number ofrisk factors are associated with increased mortality. These aredetailed in Table 5. The rationale for the ﬁnal selection of risk
factors for the model and some salient facts about these risk
factors are addressed below.
Age remains a signiﬁcant predictor of mortality from 60 years
onwards, but its impact has reduced when compared with 1995,
with the β-coefﬁcient dropping from over 0.06 to 0.0486 in univariate analysis. Multivariate analysis reduced this even further

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

NYHA
II
III
IV
CCS4
IDDM
Age
FemaleECA
CPD
N/M mob
Redo
Renal dysfunction
On dialysis
CC ≤ 50
CC 50−85
AE
Critical
LV function
Moderate
Poor
Very poor
Recent MI
PA systolic pressure
31–55 mmHg
≥55
UrgencyUrgentEmergency
Salvage
Weight of procedure
1 non-CABG
2
3+
Thoracic aorta
Constant

Coefficient

ADULT CARDIAC

Table 6: Final risk factors by multivariate regression for the model

740

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery(to 0.0286) when some of the other risk factors are taken into
account, especially the new measure of renal function which
relies on CC and that includes age in its calculation. It is noteworthy that of over 22 381 patients in the EuroSCORE database,only 21 patients (0.093%) were aged over 90. The oldest patientin the study was aged 95.
Females have a higher mortality than males. Most of the wellestablished EuroSCORE risk factors (extracardiac arteriopathy,
pulmonary disease, critical preoperative state, etc.) continue tohave an impact on mortality. After multivariate regression
analysis and comparison of the coefﬁcients between the originaland updated prediction algorithms, speciﬁc areas were identiﬁed
in which the new model differs substantially from the oldone. These areas and the reasons for these differences are
given below.
Symptomatic status is associated with increased risk. In thecase of angina, only CCS angina class 4 was associated with pooroutcome, whereas there was an increasing risk with an increasing
NYHA class. Thus, the ﬁnal decision was to incorporate NYHA
classes II, III and IV but only angina CCS class 4 into the model.
This has the advantage of both replacing the outdated deﬁnitionof unstable angina and taking into account congestive cardiac
failure, a signiﬁcant risk factor in the original model which wassacriﬁced due to collinearity with other risk factors.

BMI is weakly associated with mortality. Low BMI appears toincrease the risk of hospital death but high BMI does not. The
relationship between BMI and risk was very weak (P = 0.0845)
and this will not be considered for the ﬁnal model.
Diabetes, which was not a feature of the original model, wasrevisited. Insulin-dependent diabetes was associated with mortality; orally treated diabetes less so, and diet-controlled diabetics
actually had better outcomes than non-diabetics. Insulindependent diabetes features in the new model.
Reduced mobility has an effect whether due to neurologicaldysfunction or to musculoskeletal dysfunction.
CC, calculated using the Cockcroft–Gault formula, is a better
predictor of mortality than absolute serum creatinine. Renalfunction is thus deﬁned by calculated CC as follows:• normal (>85 ml/min)• moderately impaired (50–85 ml/min)• severely impaired (<50 ml/min)• virtually absent (on dialysis).
It is interesting to note that in renal dysfunction, the highest
risk of mortality is in patients with severely impaired renal
function who are not on established dialysis (Table 5).

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

Figure 2: Areas under the ROC curve for the previous additive and logistic models applied to current data, and the new logistic EuroSCORE II model applied to
the validation data set of 5553 patients.

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery• isolated CABG;• single major cardiac procedure other than isolated CABG;• two major cardiac procedures;• three or more major cardiac procedures.
Pulmonary artery (PA) systolic pressure was treated as adichotomous variable in the old model (>60 mmHg). We foundan increasing risk associated with rising PA pressure from 30 to
55 mmHg, followed by a plateau. Pulmonary hypertension istherefore subdivided into two categories:

Despite evidence that brain natriuretic peptide (BNP) is anindependent predictor of cardiac surgical outcomes [16], data on
BNP were only available for 1638 patients (7.3%). This factor,
though it may be useful in the future, is therefore not includedin the model due to poor availability of data.
Finally, surgery on the thoracic aorta remains associated withhigher mortality and therefore features in the risk model, but
post-infarction ventricular septal rupture only appeared twice inthe database with no deaths. This risk factor is therefore
removed. Surgeons are reassured that the high-risk nature of thisprocedure continues to be recognized through other risk factors(weight of intervention, urgency, recent MI, critical preoperative
state, PA pressure, etc).

Calibration and discrimination of the new model
Using the above risk factors, the ﬁnal logistic model was constructed from the developmental data set and applied to both
the developmental and validation data sets with very satisfactory
results.
Calibration was tested by applying the ﬁnal model to thevalidation data set which contained 5553 patients of whom 232
died in hospital (4.18%). The model-predicted mortality for thisdata set is 3.95%, a slight but acceptable underprediction.
Discrimination was tested by measuring the area under the
ROC curve (Fig. 2C). When applied to the validation data set,
the area under ROC curve was 0.8095 (95% conﬁdence interval 0.7820–0.8360), indicating very good discrimination and atrend towards slightly but not signiﬁcantly better discrimination than the old models. Goodness-of-ﬁt test results are in
Table 7.

Table 7:model

Goodness-of-fit data for EuroSCORE II logistic

Group

Prob

obs0

exp0

obs1

exp1

total

1
2
3
4
5
6
7
8
9
10

0.00690.00920.01170.01460.01870.02420.03230.04660.07980.8609

10
17
5
17
19
40
45
71
101
280

9.7
12.9
16.6
20.9
26.5
33.8
44.4
61.6
95.6
283.0

1607
1558
1590
1577
1585
1547
1551
1523
1494
1315

1607.31562.1
1578.41573.11577.51553.21551.6
1532.41499.41312.0

1617
1575
1595
1594
1604
1587
1596
1594
1595
1595

Number of observations = 15 952; Hosmer–Lemeshow χ 2(8) = 15.48;
Prob > χ 2 = 0.0505 (data collapsed into 10 quantiles of estimatedprobabilities (prob); obs: observed: exp: expected; 0: death; 1:
survival).

ADULT CARDIAC• PA pressure 30–55 mmHg and• PA pressure 56 mmHg and above.

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

It is known that liver failure increases the mortality of cardiac
surgery [14, 15] and yet this risk factor is not usually represented
in risk models. Of the various indicators of hepatic dysfunction,
serum albumin concentration was selected as the one that isleast affected by cardiac therapy and is the most objective andwidely available test. Disappointingly, the relationship between
serum albumin and risk was practically zero. There is somedoubt about the measurement of serum albumin concentration,and it is possible that different centres have used different units
and assay techniques.
Any previous cardiac surgery increases the risk, but the effectof multiple previous operations on outcomes is not signiﬁcantly
different from the effect of one previous operation. This risk
factor is therefore retained without modiﬁcation.
We explored the effect of the size and recency of myocardial
infarction (MI) by requesting data on troponin levels and the temporal separation between infarct and operation. Unfortunately,
the measuring of two types of troponin (I and T), the difﬁculty ofidentifying an easy and consistent conversion between the two,the multiplicity of assays available and the very wide variation
between the ‘normal’ ranges from many hospitals meant that
there is no practical, uniformly acceptable method of measuring
infarct size consistently across units. As for recency, the mostuseful categorization was a three-level factor; MI in previous 72 h,
MI 4 days to 3 months ago, no MI in last 3 months. However, thiscorrelated very closely with urgency of operation (see below),
and the effect was largely lost when urgency was appropriately
taken into account. This risk factor (MI within 90 days) therefore
remains unchanged.
Many units provided LV ejection fraction (LVEF) as a percentage in addition to subjective categorization of LV function. By
dividing those with poor LV function into ‘poor’ (LVEF 21–30%)
and ‘very poor’ (LVEF 20% or less), a slightly better ﬁt could beobtained. Because of this, and despite some missing data on
LVEF in the ‘poor’ LV group, we believe that adding the category‘very poor’ is clinically indicated and may help reduce risk-averse
behaviour. LV function is therefore divided into four categories:
good, moderate, poor and very poor.
The predictive value of urgency was improved by subclassiﬁcation. The previous model had ‘emergency’ as the only factor.
The new model recognizes elective, urgent, emergency andsalvage as urgency categories, and is therefore more predictive
and in harmony both with other risk models and the subjective
view of clinicians.
An important factor to receive attention in this study is theweight and nature of the intervention. The original model was
criticized for the same risk to an isolated aortic valve replacement (AVR) as to a double valve replacement with triple coronary artery bypass grafting (CABG). In harmony with the originalmodel, the lowest risk operation was found to be isolated
on-pump CABG (off-pump surgery was associated with higher
mortality, and this ﬁnding requires further study beyond thescope of this paper). We identiﬁed four classes of intervention‘weight’ associated with an incremental effect on mortality:

741

742

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery

Deﬁnitions and explanations of the risk factors• salvage: patients requiring cardiopulmonary resuscitation (ex-

NYHA class. NYHA classiﬁcation for dyspnoea:

ternal cardiac massage) en route to the operating theatre or
before induction of anaesthesia. This does not include cardiopulmonary resuscitation after induction of anaesthesia.

• I: no symptoms on moderate exertion;• II: symptoms on moderate exertion;• III: symptoms on light exertion;• IV: symptoms at rest.

Recent MI. Within 90 days before operation.

CCS class 4. CCS class 4 angina (inability to perform any
activity without angina or angina at rest).

intervention. The baseline is isolated CABG: operations ‘heavier’than the baseline are in three categories:

IDDM. Insulin-dependent diabetes mellitus.

• isolated non-CABG major procedure (e.g. single valve proced-

Extracardiac arteriopathy. One or more of the following:

Symptomatic Carotid Endarterectomy Trial criteria);• amputation for arterial disease;• previous or planned intervention on the abdominal aorta, limb
arteries or carotids.

Poor mobility. Severe impairment of mobility secondary to
musculoskeletal or neurological dysfunction.

Previous cardiac surgery. One or more previous major
cardiac operation involving opening the pericardium.

Renal dysfunction. This is assessed by CC as estimated using
the Cockcroft–Gault formula and falls into three categories:• CC 51–85• on dialysis (regardless of serum creatinine)• CC ≤ 50.
Active endocarditis. Patients still on antibiotic treatment for
endocarditis at the time of surgery.
Critical preoperative state. Any one or more of the following
occurring preoperatively in the same hospital admission as theoperation:• ventricular tachycardia or ﬁbrillation or aborted sudden death;• cardiac massage;• ventilation before arrival in the anaesthetic room;• inotropes;• intra-aortic balloon counterpulsation or ventricular-assist
device before arrival in the anaesthetic room;• acute renal failure (anuria or oliguria <10 ml/h).
LV function or LVEF.

• good (LVEF 51% or more);• moderate (LVEF 31–50%);• poor (LVEF 21–30%);• very poor (LVEF 20% or less).
Urgency of procedure.

• elective: routine admission for operation;• urgent: patients not electively admitted for operation but who
require surgery on the current admission for medical reasonsand cannot be discharged without a deﬁnitive procedure;• emergency: operation before the beginning of the next
working day after decision to operate;ure, replacement of ascending aorta, correction of septal
defect, etc.);• two major procedures (e.g. CABG + AVR), or CABG + mitralvalve repair (MVR), or AVR + replacement of ascending aorta,
or CABG + maze procedure, or AVR + MVR, etc.);• three major procedures or more (e.g. AVR + MVR + CABG, or
MVR + CABG + tricuspid annuloplasty, etc.), or aortic root
replacement when it includes AVR or repair + coronary reimplantation + root and ascending replacement).
Only major cardiac procedures count towards to the total.
Examples of procedures which do not qualify are: sternotomy,
closure of sternum, myocardial biopsy, insertion of intra-aortic
balloon, pacing wires, closure of aortotomy, closure of atriotomy;
removal of atrial appendage, coronary endarterectomy as part of
CABG, etc.

DISCUSSION
Limitations
EuroSCORE II was constructed from an international, contemporaneous and highly accurate, validated database and should
therefore be a robust risk model for use in cardiac surgery
worldwide. There are, of course, limitations to this study andthese are dictated by the restrictions imposed by the methodology and logistics of constructing the study.
With only 21 patients over the age of 90 in the data set, therisk model may not be accurate in these patients. The oldest
patient in the EuroSCORE database was 95, and the model istherefore not validated in patients over this age.
Participating units were volunteers, and this introduces anelement of bias in self-selection. However, there is no mechanism to force all units to participate or even to force a randomly
selected sample of units to do so. Even if that were possible, any
coercive element in such a study would have resulted in potentially greater bias introduced by reduced willingness, ability or
both to provide data. We believe that voluntary participation
improves the chances of obtaining high-quality data.
A further bias may be introduced by the simple possibility
that units with the ability and willingness to provide data may
have better outcomes than those without such facilities. To militate against such a bias, every effort was made to encourage
units of all types to participate. Multimedia promotional literature clariﬁed that we were not seeking only ‘centres of excellence’. Furthermore, we believe that many units recognized that,
if only centres of excellence participated in the study, the resultant model may set a standard that would be hard for many units
to meet. We believe, but cannot prove, that both the number

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

• claudication;• carotid occlusion or >50% stenosis (North American

Weight of procedure. This measures the extent or size of theThe new model
The old model is no longer appropriately calibrated.
Risk-adjusted mortality has fallen by around a half in comparison
with what could be expected in the 1990s. This is a powerful
testament to the substantial improvement in quality of care thathas been achieved in cardiac surgery. The new model is far
better calibrated. It continues to rely on a relatively small
number of risk factors, most of which featured in the originalmodel. The modiﬁcations to the risk factors in the new model
are modest but they are both evidence-based and intuitive. The
new risk factors and deﬁnitions should better reﬂect current
practice with improvement in discrimination as well as calibration, but that can only be gauged when the model is in use.
We set out to produce a global cardiac surgical risk model.
Others believe that procedure-speciﬁc models are superior, andthere have been and doubtlessly will be many models designed
to predict risk in very speciﬁc circumstances. Procedure-speciﬁc
risk models are better for narrowly deﬁned procedures, but
become problematic in complex double and triple procedures.
There are inherent difﬁculties in data collection and proceduredeﬁnition when developing such models for every conceivabletype of operation and combination of operations.

Using EuroSCORE II
Like a scalpel or a needle holder, a risk model is an instrumentin the cardiac surgeon’s toolkit. Used judiciously, it can enhance
the quality of cardiac surgical care by facilitating better decisionmaking and providing a benchmark for quality control.
Ill-conceived and misguided application can damage bothpatients and surgeons. We highlight those areas where the application of risk modelling calls for special caution and the exerciseof judgment.
(i) The intrinsic imperfection of modelling
No risk model predicts the outcome for an individual patient.
Any prediction of percentage mortality is for a population ofpatients: an individual patient will either survive or die from an

operation regardless of the predicted risk. However, for an individual patient, the knowledge of the predicted mortality for agroup of similar patients undergoing the same procedure is an
important part of decision-making and informed consent.
No risk model is perfect. The selection of risk factors in amodel is a necessary compromise between what is practical andwhat is feasible. All surgeons know that not all risk factors
appear in all models, nor would it be possible to devise a model
which includes every conceivable risk factor and every rare
medical syndrome. This is for valid reasons: such a model would
be too complex for clinical use, and the database from which it
can be derived is virtually impossible to assemble. The selectionof risk factors to include in the model is necessarily a compromise in which risk factors compete for inclusion on the basis offour features:• availability• objectivity• resistance to falsiﬁcation• credibility to users.
Where rare risk factors are present, the clinician must make avalue judgment about the absolute applicability of the model tothe patient in question.
It can be argued that a global risk model is unsuitable for theﬁne distinction between risk levels in different procedures, andthis is of course true. The Society of Thoracic Surgeons risk
model has a handful of broad categories, but risk modelling caneven further be reﬁned by additional subclassiﬁcation based onprocedure, pathology, different combinations of procedures andmultiple combinations of the all of the above. However, suchreﬁnement has its price in that the resultant modelling becomescomplex and there will be categories of patients and proceduresthat are not supported by the model. We deliberately set out toachieve a global risk model for cardiac surgery in adults andbelieve that we achieved this to a good standard, but that ofcourse does not preclude the creation of separate procedurespeciﬁc risk models by other workers if this is perceived as desirable. The global nature of EuroSCORE is undoubtedly one of its
limitations, but it has also been one of its great strengths inachieving user acceptability.
(ii) The variation in outcomes between centres and surgeons
A risk model sets a standard, and units and surgeons will
perform at a level equal, below or above that standard. Once themodel is in use, all units and surgeons should calculate their
RAMR by dividing their actual (observed) mortality by theirpredicted mortality according to the risk model. The RAMR canthen used in a number of risk assessment situations, two of
which are addressed below:• Informed consent and the evaluation of risk for an individual
patient: the most accurate and scientiﬁc predicted mortality toquote to a patient is the predicted mortality for the procedure
as calculated by EuroSCORE II multiplied by the unit’s or theindividual surgeon’s RAMR. The RAMR can only be stated withmeaningful conﬁdence when sufﬁcient observations have been
made of predicted versus observed mortality in a large enoughpatient population representing at least a year’s workload. It
should be derived from recent experience and updated atregular intervals.
• Determining the level of risk at which a new or experimental
procedure is justiﬁed: if it is decided that an experimental

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

and range of participating units are fairly representative of
current cardiac surgery.
Validation is necessarily limited by the resources available tothe project. We are unable directly to employ researchers
to gather and validate data in such a large number of units on
a global scale. Nevertheless, validation was facilitated by
web-based data collection where multiple mandatory ﬁelds witherror and range checks were employed throughout. Thesefeatures, when added to the subsequent logic checks onreceived data, resulted in an overall data set of high quality.
This is supported by the fact that the number of units which had
to be disregarded in the analysis due to faulty data was indeed
very small.
Finally, the model is based on logistic regression, taking
account of multiple risk factor interactions. Artiﬁcial neuralnetworks (ANNs) may surpass logistic regression and we are
currently working on developing ANN versions of the model todetermine what additional advantages may be obtained by thisapproach [17].

743

ADULT CARDIAC

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery

744

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery

procedure should be offered to patients whose risk from conventional cardiac surgery exceeds, for example, 20%, then theexperimental procedure in a particular unit should only beoffered to those whose predicted mortality is >20% after division by the unit’s RAMR. If, for example, a unit is outperforming EuroSCORE II and its RAMR is 0.5, then the experimental
procedure can be considered in patients whose score is 20%
divided by 0.5, i.e. 40%.
(iii) The use of risk models as performance indicators(iv) The future
No model is future-proof. Work has already begun on the
EuroSCORE III project, in which we plan to collect continuous,
prospective contemporaneous data from specially selected units
to determine when and how the model will require an update.
This will ensure that the model will remain relevant as cardiac
surgical results hopefully continue to improve.

CONCLUSIONEuroSCORE II, an update of the logistic EuroSCORE model, uses
similar methodology but is derived from a more current data set
and reﬁned to incorporate evidence-based improvements andto reﬂect better current cardiac surgical practice. It is recommended for assessing risk in general adult cardiac surgery.

ACKNOWLEDGEMENTS
We are grateful to the Journal of Heart Valve Disease for promoting the project and to Sandra Church for help in designingpromotional material. We thank all participating centres forinvesting time and effort into data collection, without which thisproject would not have been possible.

This study was supported by Edwards Laboratories, Karolinska
Hospital, Stockholm and Papworth Hospital, Cambridge.
Conﬂict of interest: none declared.

REFERENCES[1] Nashef SAM, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon
R; the EuroSCORE Study Group. European System for Cardiac Operative
Risk Evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999;16:9–13.
[2] Roques F, Nashef SAM, Michel P, Gauducheau E, de Vincentiis C, Baudet
E et al. Risk factors and outcome in European cardiac surgery: analysis of
the EuroSCORE multinational database of 19 030 patients. Eur J
Cardiothorac Surg 1999;15:816–23.
[3] Basraon J, Chandrashekhar YS, John R, Agnihotri A, Kelly R, Ward H et al.
Comparison of risk scores to estimate perioperative mortality in aortic
valve replacement surgery. Ann Thorac Surg 2011;92:535–40.
[4] Parolari A, Pesce LL, Trezzi M, Loardi C, Kassem S, Brambillasca C et al.
Performance of EuroSCORE in CABG and off-pump coronary artery
bypass grafting: single institution experience and meta-analysis. Eur
Heart J 2009;30:297–304.
[5] Qadir I, Perveen S, Furnaz S, Shahabuddin S, Sharif H. Risk stratiﬁcation
analysis of operative mortality in isolated coronary artery bypass graft
patients in Pakistan: comparison between additive and logistic
EuroSCORE models. Interact Cardiovasc Thorac Surg 2011;13:137–41.
[6] Lebreton G, Merle S, Inamo J, Hennequin JL, Sanchez B, Rilos Z et al.
Limitations in the inter-observer reliability of EuroSCORE: what shouldchange in EuroSCORE II? Eur J Cardiothorac Surg 2011;40:1304–8.
[7] Shih HH, Kang PL, Pan JY, Wu TH, Wu CT, Lin CY et al. Performance ofEuropean System for Cardiac Operative Risk Evaluation in Veterans
General Hospital Kaohsiung cardiac surgery. J Chin Med Assoc 2011;74:
115–20.
[8] Akar AR, Kurtcephe M, Sener E, Alhan C, Durdu S, Kunt AG et al.; The
working Group for the Turkish Society of Cardiovascular Surgery and Turkish
Ministry of Health. Validation of the EuroSCORE risk models in Turkish adultcardiac surgical population. Eur J Cardiothorac Surg 2011;40:730–5.
[9] Yap CH, Reid C, Yii M, Rowland MA, Mohajeri M, Skillington PD et al.
Validation of the EuroSCORE model in Australia. Eur J Cardiothorac Surg
2006;29:441–6.
[10] Roques F, Michel P, Goldstone A, Nashef SAM. The logistic EuroSCORE.
Eur Heart J 2003;24:881–2.
[11] Van Gameren M, Klieverik LM, Struijs A, Venema AC, Kappetein AP,
Bogers AJ et al. Impact of the deﬁnition of renal dysfunction on
EuroSCORE performance. J Cardiovasc Surg 2009;50:703–9.
[12] Walter J, Mortasawi A, Arnrich B, Albert A, Frerichs I, Rosendahl U et al.
Creatinine clearance versus serum creatinine as a risk factor in cardiac
surgery. BMC Surg 2003;3:4.
[13] Sergeant P, de Worm E, Meyns B, Wouters P. The challenge of departmental quality control in the reengineering towards off-pump coronary
artery bypass grafting. Eur J Cardiothorac Surg 2001;20:538–43.
[14] Shaheen AA, Kaplan GG, Hubbard JN, Myers RP. Morbidity and mortality
following coronary artery bypass graft surgery in patients with cirrhosis:
a population-based study. Liver Int 2009;29:1141–51.
[15] Suman A, Barnes DS, Zein NN, Levinthal GN, Connor JT, Carey WD.
Predicting outcome after cardiac surgery in patients with cirrhosis: acomparison of Child-Pugh and MELD scores. Clin Gastroenterol Hepatol
2004;2:719–23.
[16] Cuthbertson BH, Croal BL, Rae D, Gibson PH, McNeilly JD, Jeffrey RR
et al. N-terminal pro-B-type natriuretic peptide levels and earlyoutcome after cardiac surgery: a prospective cohort study. Br J Anaesth
2009;3:647–53.
[17] Nilsson J MD, Ohlsson M, Thulin L, Höglund P, Nashef SAM, Brandt J.
Risk factor identiﬁcation and mortality prediction in cardiac surgery
using artiﬁcial neural networks. J Thorac Cardiovasc Surg 2006;132:12–9.

APPENDIX. CONFERENCE DISCUSSION
Dr J. Takkenberg (Rotterdam, The Netherlands): The new EuroSCORE,
EuroSCORE II, has good calibration and excellent discrimination with an area

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

Some risk models, including EuroSCORE, have been used or
adapted for use as performance indicators to evaluate thequality of a clinical service. EuroSCORE has been developed todeal with a global cardiac surgical practice, and this approach isappreciated by many cardiac surgeons with a mixed practice
who require a single model for risk evaluation, but it is possiblethat any model designed for global cardiac surgical risk assessment may work better in some categories of patients thanothers. We plan to sub-analyse the data from the EuroSCORE
database to determine whether there are indeed areas wheresuch differences are important, and we have no doubt that otherworkers will carry out similar analyses in their units and databases. In the meantime, we advise that caution be exercised
before applying the model to sub-groups of isolated procedures
rather than to a global practice, and to surgeons whose practiceconsists overwhelmingly or exclusively of one type of heart operation or another. As ever, the determination of the performancethreshold should be the responsibility of bodies and individuals
who are able to assess the relevance of any risk model to theirdata and to select the appropriate threshold on the basis ofcareful and sound professional and clinical judgment.

Funding

90-day mortality. And when that information is available to all, then, ofcourse, it should be included.
Dr Takkenberg: So you will promise to do that for EuroSCORE III then?
Dr Nashef: Work on EuroSCORE III has started already.
Dr A. Badreldin ( Jena, Germany): We participated in the data collection. I
have just two quick questions. First of all, the beneﬁt of any preoperative
scoring system has been thoroughly discussed in the literature over the last
15 years so as to compare the performance of different centres on the
National Registry level. Moreover, in clinical practice for us, for surgeons mostimportantly, it is a basis for the preoperative consents and preoperative discussion with the patient. You included the operative procedure in the new
EuroSCORE. Should we drop this second rule of the scoring system ( preoperative consents based on the score value), especially since we know that
we would change our strategy, maybe often intraoperatively, according to
TOE diagnostics or any unexpected surprise?
The second question very quickly. A scoring system should not be used inany centre unless it has already been validated internally for its reliability. You
did not validate this score system, EuroSCORE II, yet. Should we add this notefor any end user on the website to avoid any drawbacks that we have alreadyexperienced with the original EuroSCORE due to lack of this validation?
Dr Nashef: In answer to your ﬁrst question, which I think was about incorporating data about the operative procedure, I would say that the amountof data that is requested in terms of the operative procedure itself is fairly
limited in EuroSCORE and it also tends to be very, very objective data. All
models are subject to gaming. And if somebody wants to be very clever anddevious in gaming, then you can upgrade a procedure relatively easily. We
hope that no surgeons do that. But I think we really have to have somemeasure of the weight of the intervention, because for every comment likeyours, I have received 10 complaining that in the old EuroSCORE, an AVRscores the same as an AVR with ﬁve grafts and that this is really unfair.
Now, in answer to your second question—you have to remind me what
your second question was.
Dr Badreldin: Validation, eventually the customization, ﬁrst degree or
second degree.
Dr Nashef: A model sets a standard. Units will perform as well as, better,or worse. And the data and the information you give to your patients shouldtake into account how your unit performs and how you yourself performagainst that model. In the next few months, an opportunity to calculate yourown risk-adjusted mortality ratio will appear on the website so that themodel can be tailor-made for your own institution. But, of course, until wehave data from you on that, we cannot really do it yet.
You have criticized the old model for causing problems in that respect,particularly in relation to the TAVI selection. But, of course, if you’re going touse a model, a model can discriminate very powerfully, but its calibrationmust be adjusted to what you yourself can achieve. Many units will hoveraround the average, some units will have a mortality that is much higher than
predicted, and some units will have a mortality that is much lower than
predicted. And clinicians should use their reasoning and their insights inorder to adjust things for their own patients. This is when it comes toinformed consent. In terms of the comparison of the quality of performance,
the standard is there and it can still be used.

Downloaded from https://academic.oup.com/ejcts/article/41/4/734/646622 by guest on 23 August 2021

under the curve of 0.81. That’s great. I have numerous questions, as you canimagine, but I was told to restrict myself to two. So my ﬁrst question is amore technical question related to the modelling process. An area under the
curve of 0.81 is excellent, but perhaps could be even better if the data set
would be utilized to the fullest. I would use imputation techniques for thosedata that can be assumed missing completely at random and which you have
now excluded. And I would actually apply bootstrapping techniques as amethod of validation to prevent overﬁtting. Additionally, one could take into
account potential interaction of variables. You state in your presentation thatyou looked at that. So, for example, the effect of female sex on mortality may
be stronger or weaker in older patient groups compared to younger patientgroups. So did you ﬁnd any interaction and consider interaction terms when
you were building EuroSCORE II? That would be my ﬁrst question.
My second question pertains to the use of hospital mortality as theprimary endpoint of your model. The hazard function of death after cardiac
surgery shows an early phase of rapidly falling risk in the ﬁrst three postoperative months. It underlines that surgical mortality is time-related rather
than related to the location of the patient, and particularly in western countries where patients are discharged really early to general hospitals. Realizing
the practical barriers to measure 30- and 90-day mortality, I nevertheless ﬁndthat we should aim to report time-related outcomes in our models as part ofour endeavour to continuously improve cardiac surgical care. So do you plan
to build a EuroSCORE II model with time-related outcomes as well?
Dr. Nashef: First, may I say I fully agree with everything that you have said.
In answer to your ﬁrst question, there is no doubt that logistic regression
modelling does offer you the opportunity to explore interactions between
variables, but this has to be initiated in that you have to look for them yourself. And we have done our best to look for as many interactions as possible.
In fact, one of the very interesting interactions we found is the interaction
between renal function and age, and some of you, when you try the newmodel, are going to be appalled to ﬁnd that when you enter a patient age
89, the predicted risk is very small, but that will increase as soon as you enter
the creatinine clearance. So we have looked as much as we could for clinicalinteractions of risk factors even beyond what is immediately apparent.
However, regression modelling by its nature is limited in its ability to do that.
And as you say, bootstrap techniques and the use of artiﬁcial neural networks
offer substantial possibilities in doing this, and I can assure you that we are
currently working on an artiﬁcial neural network model to see if it can actually improve on what the regression model can offer.
And in response to your second question, of course, there is mortality after
going from hospital and, of course, it doesn’t plateau until 90 days havepassed. However, this is a suboptimal situation. I would put it to you thatthose units that actually voluntarily participated are probably better at getting
data than those units that did not participate. And even if you participate in
the EuroSCORE project, and only half of you can give me 90-day mortality,
then it is simply not practical to use that in a model that will be used for
quality of performance measurement. Because the problem is those whohave the data will report higher mortalities; those who don’t have the data
will report lower mortalities. So we have to accept that this is not ideal. But inthe current status of data availability to units, this is what we are stuck with.
In the future, we hope units will start looking at their 30-day and especially

745

ADULT CARDIAC

S.A.M. Nashef et al. / European Journal of Cardio-Thoracic Surgery