# Enhancing Survival Analysis Model Selection through XAI(t) in Healthcare
## 3 Results  
We evaluated the models' performance on the test set according to the following metrics:  
* [PERSON]'s C-Index [44]--Also known as the concordance index, it measures the proportion of all pairs of observations for which the survival order predicted by the model matches the order in the data. A higher C-index (ideally equal to 1) indicates better concordance between the model prediction and the relative ground truth, whereas a C-index equal to 0.5 indicates a random prediction. * Integrated Cumulative-Dynamic Area Under the Curve (C/D AUC) [45]--This measures the area under the Receiver Operating Characteristic (ROC) curve at different time points during the observation. * Brier Score [46]--This measures the difference between the model prediction and the corresponding ground truth event; a lower Brier score suggests better model precision, while a high Brier score indicates performance degradation. Ideal Brier score values should be closer to 0 since values closer to 0.5 indicate random predictions. We placed greater emphasis on the C-index and the Brier score, as they are the most commonly used metrics in survival tasks and are generally more easily understandable than the C/D AUC. The evaluation metrics on the test set are shown in Table 2, with graphical comparisons shown in Figures 3 and 4 for the ML and DL models, respectively. Figure 2: Experimental pipeline. We exploited the compatibility with the SurvSHAP and mlr3 libraries by creating a survival task and wrapping unsupported models in a composite learner. For each trained model, we created the related explainer object and computed performance metrics and explanations. Models included Cox proportional hazards (CPH), survival random forest (SRF), survival SVM (SSVM), survival gradient boosted model (SGBM), and survival Deep Learning (SDL). Figure 4: Metrics of survival DL models computed on the test set. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Family**} & \multirow{2}{*}{**Model**} & \multirow{2}{*}{**C-Index**} & **Integrated C/D** & **Integrated Brier** \\  & & & **AUC** & **Score** \\ \hline \multirow{4}{*}{Machine Learning} & CPH & **0.81** & **0.72** & **0.10** \\  & SRF & 0.81 & 0.70 & 0.12 \\  & SSVM & 0.71 & 0.61 & 0.15 \\  & SGBM & 0.79 & 0.69 & 0.14 \\ \hline \multirow{4}{*}{Deep Learning} & Cox time & **0.78** & **0.73** & 0.12 \\  & DeepHit & 0.73 & 0.70 & 0.13 \\  & DeepSurv & 0.57 & 0.60 & 0.16 \\  & LogHazard & 0.77 & 0.70 & **0.11** \\  & PCHazard & **0.78** & 0.71 & 0.13 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Metrics of survival models computed on the test set, with best performance metrics highlighted. Cox proportional hazards (CPH); survival random forest (SRF); survival SVM (SSVM); and survival gradient boosted model (SGBM). Figure 3: Metrics of survival Machine Learning models computed on the test set. Cox proportional hazards (CPH), survival random forest (SRF), survival SVM (SSVM); and survival gradient boosted model (SGBM). ML Model Results As a general result, SSVM was the worst-performing model, while all other models achieved good results. The differences between Cox regression, SGBM, and SRF were minimal for the C-index and integrated C/D AUC, whereas Cox regression exhibited a lower Brier score. Thus, we chose CPH for the explainability step. Moreover, this model returned a hazard ratio for each feature that can be used for data explainability in addition to XAI techniques (Section 4). DL Model Results As shown in Figure 4, CT, PCH, and LH exhibited similar performance and were thus the best models. We selected LH for the explainability phase (see Section 4 for more details). The time-variant Brier scores and C/D AUC values for the CPH, CT, and LH models are depicted in Figure 5.  
Moreover, to assess the stability of the outcomes, we performed model training using 10-fold cross-validation. The results obtained (on the same test set used in the holdout approach) are depicted in Appendix A, Table A1, confirming the performance trends obtained with the holdout split.