# Using Explainable Machine Learning to Explore the Impact of Synoptic Reporting on Prostate Cancer
## 4 Discussion  
In this paper, we showed a practical application of XML in oncology. More specifically, we looked at the effect that SR had on the survival of a Dutch prostate cancer subpopulation of 14,878 patients. For this, we used two different approaches: classical statistical survival analysis using a CPH model and state-of-the-art ML in the shape of an XGB model. More importantly, we included an explainability layer in the form of SHAP values to generate an explicit knowledge representation of how the models generated their predictions. The comparison between the models (Figure 2) showed that the XGB approach performed significantly better (\(p<0.0001\)) with a \(c\)-index of 0.67 (which can practically be considered reasonable performance [61]), in contrast to the CPH approach, with a \(c\)-index of 0.58. This is in line with findings in the literature that performed similar comparisons between CPH and ML models [62; 63; 28; 64; 65]. These works found that ML-based techniques can perform at least as well as classical statistical methods (if not better) than classical statistical methods in predicting ranked occurrence of patient events. This highlights the potential of a more extensive use of ML-based methods when studying patient survival or recurrence. Afterwards, we opened the models' _black boxes_ by computing their SHAP values. This allowed us to generate valuable insights on many different aspects. First of all, we were able to generate an explicit representation (Figure 3) of what impact the different features had on the models' output (and thus on their performance). On one hand, we found that \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Feature** & **HR** & \multicolumn{2}{c}{**95\% CI**} & \multicolumn{2}{c}{**z-Value**} & \multicolumn{1}{c}{**\(p\)-Value**} \\ \hline age & 1.07 & 1.06 & 1.09 & 10.72 & \(<\)0.005 \\ \hline EAU & & & & & \\ Localized—low risk & 1.00 & - & - & - & - \\ Localized—intermediate risk & 2.39 & 1.28 & 4.46 & 2.73 & 0.01 \\ Localized—high risk & 3.27 & 1.73 & 6.18 & 3.65 & \(<\)0.005 \\ Locally advanced—high risk & 2.82 & 1.47 & 5.31 & 3.16 & 0.01 \\ \hline SR & & & & & \\ No & 1.00 & - & - & - & - \\ Yes & 0.78 & 0.63 & 0.97 & \(-\)2.27 & 0.02 \\ \hline academic hospital & & & & & \\ No & 1.00 & - & - & - & - \\ Yes & 0.90 & 0.76 & 1.06 & \(-\)1.23 & 0.22 \\ \hline incidence year & 0.99 & 0.94 & 1.05 & \(-\)0.25 & 0.80 \\ \hline HR: hazard ratio; CI: confidence interval. & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results of the CPH multiple regression  
Figure 5: Interaction effects between SR and the two most relevant features: (**A**) age and (**B**) EAU (encoded as 1—low risk/localised, 2—intermediate risk/localised, 3—high risk/localised, and 4—high risk/locally advanced). In both cases, artificial jitter was added along the \(x\)-axis to better show the point density. for the CPH model, the most important feature was age, followed by SR and EAU. On the other hand, for the XGB model the most important feature was also age, but was followed by EAU and _then_ by SR. We believe that the latter ranking makes more sense, given that the EAU feature has more information about the condition of a patient (since it is calculated from his TNM staging, Gleason score, and PSA levels, Section 2.2). In both cases, incidence year and academic hospital were the least important features. We hypothesize that this could be because there were no important changes in the care pathway of this particular patient subpopulation dependent on time (incidence year) or on the type of hospital where the patients were treated (academic hospital). These results are also consistent with the HRs calculated earlier (Table 4), where those of age, EAU, and SR were significant, while those of incidence year and academic hospital were not. Therefore, we will focus on the top three (significant) features (age, EAU, and SR) for the rest of this discussion. Dependence plots (Figure 4) can provide a few insights on the feature distributions.