State‐of‐the‐art machine learning improves predictive accuracy of 1‐year survival after heart transplantation

Heart transplantation (HT) remains the treatment of choice for patients with medically refractory end-stage heart failure given its improved long-term outcomes and quality of life. Despite the effectiveness of the treatment, only about 3000 HTs are performed annually in the USA, with a small rise attributed to the opioid epidemic and the expanded use of donors with hepatitis C. Optimization of outcomes and risk stratification is recognized as a critically important issue in HT today. Although several risk scores have been developed for the prediction of outcomes after HT, their accuracy remains modest. Machine learning (ML) algorithms have shown promising results in predicting outcomes and discovering phenotypes in patients with heart failure and have even been applied in patients undergoing HT. However, further work is needed to determine the role of ML in the prediction of outcomes and risk stratification of HT patients. Herein, we present our work on prediction of 1-year survival after HT using state-of-the-art ML algorithms. We queried the United Network for Organ Sharing (UNOS) database for patients that were enlisted and underwent heart transplant between 2010 and 2018. Extensive baseline clinical and laboratory data as well as follow-up survival were collected. We excluded patients with multiple transplants during the index hospitalization, non-adult patients, and patients with >10% missing values or less than 1-year follow-up. The study dataset was then randomly split into training and validation cohorts with a 3:1 ratio. A random oversampling technique was applied to artificially equalize the number of patients that were alive with the number of patients that did not survive at 1-year in the training cohort. This is common practice for many ML algorithms. Next, a feature selection method based on Support Vector Machines (SVM) and Fuzzy Logic, namely, SVM-FuzCoC, was used to (i) identify the most informative variables, (ii) reduce the complexity, and (iii) improve the performance of the ML models. The selected features were then used to train five state-of-the-art supervised ML algorithms: Adaptive boosting (Adaboost), SVM, Decision Trees, K-nearest neighbour, and Logistic Regression. Training of the models was controlled by hyperparameter optimization. After training, the validation cohort was used to internally test the ML algorithms. Their performance was evaluated by measuring the total area under the curve (AUC) of the receiver-operator curve as well as sensitivity, specificity, and positive and negative predictive value. We performed explainability analysis of the best ML model using the method of local interpretable model-agnostic explanations (LIME). This method creates a simple linear model that approximates any complex ML model. We separately applied the IMPACT score for the prediction of 1-year survival after HT. After preprocessing, the dataset composed of 18 625 recipients (mean age 53 ± 13 years, 73% male) with 134 pre-transplant variables. There were no significant differences between patients assigned to the development and validation cohort. There were a total of 2334 (12.5%) deaths at 1-year after HT. Feature selection identified 39 out of 134 variables that were highly predictive of 1-year survival and were used in the ML algorithms (Table 1). These included recipient, donor, and transplant characteristics. AUC for each of the five ML using the validation cohort is shown in Figure 1A. Although the best performance was achieved by Adaboost (AUC 0.689, 95% CI 0.665–0.715), all ML models achieved higher AUC compared with IMPACT score with the exception of the K-nearest neighbour (SVM AUC 0.637 95% CI 0.612–0.662, DT AUC 0.649 95% CI 0.622–0.676, LR AUC 0.642 95% CI 0.614–0.667, and KNN AUC 0.527 95% CI 0.502–0.550 in contrast to IMPACT AUC 0.569, 95% CI 0.545–0.592). With the exception of K-nearest neighbour, there was an exponential increase in AUC with the use of one to five pre-transplant variables. However, the increase in AUC gradually plateaued after the use of 10 variables. Sensitivity, specificity, and positive and negative predictive value of the ML models using the validation subcohort ranged from 55.6% to 68.5%, 42.6% to 63.4%, 88.7% to 93.6%, and 13.5% to 21.6%, respectively. The results of explainability analysis for the best ML model using LIME are shown in Figure 1B. Increased serum creatinine, shorter recipient height, and increased ischaemic time had the strongest association with 1-year mortality. Artificial intelligence and ML algorithms in particular offer some striking advantages compared with traditional statistical methods in regard to their ability to analyse large and LETTER TO THE EDITOR