## 4 Results
### RQ2: What Aspects/Methods of the Use of XAI in CDSS Have Been the Focus of the Literature?  
We classified the CDSS according to three main categories of XAI: algorithmic transparency, explainer generalizability, and explanation granularity. More specifically, we examined whether they implemented a _post-hoc_ or _ante-hoc_ explainability method, a model-specific or model-agnostic technique, and whether they provided global or local explanations. The classifications of the CDSS in these categories are presented in Table 2.  
Almost all studies aimed for the provision of local explanations, for a specific prediction. The work by [PERSON] et al. [120] was the only one that focused on global explanations, using a model-agnostic _post-hoc_ technique. Most studies implemented a model-specific _ante-hoc_ technique that provided local explanations [122, 124, 125, 126, 128]. One of the studies used a variety of model-agnostic _post-hoc_ methods to provide local explanations in the form of feature attribution, counterfactual rules, and sensitivity analysis [118]. The remaining CDSS implemented model-agnostic _post-hoc_ methods for local explanations [119, 123].  
In terms of _post-hoc_ explainability, perturbation-based models that use model-agnostic explanations are common in the literature [98, 99, 118, 120, 127] and have been used in two of the proposed CDSS [118, 120]. An additional system that incorporated _post-hoc_ explainability was designed by [PERSON] et al. [123] and provided local explanations of the preference model using rainbow boxes [56]. Deep Neural Networks, generally a black-box technique, were explained visually with regional abnormality maps in the system proposed by [PERSON] et al. [119].  
The remaining CDSS were created in an _ante-hoc_ explainable manner. Rule-based systems were developed in two studies to provide local explanations, as they were considered closer to human reasoning, and thus more preferable by clinicians [124, 125]. Case-Based Reasoning is an intrinsically explainable method that was used by [PERSON] et al. [126] for their CDSS for breast cancer, which was also supported by visual explanations in the form of rainbow boxes and a polar multidimensional scaling scatter plot. In the study by [PERSON] et al. [122], the CDSS was developed using a bidirectional gated recurrent unit (BiGRU) with attention mechanism, which allowed for the exploration of how much each fragment of the text contributed to a prediction, thus providing local explanations. [PERSON] et al. [128] built a CDSS using Relational Functional Gradient Boosting (RFGB), a statistical relational learning method which attributes its explainability to the usage of tree models and the provision of explanations in terms of features of interest.  
Most studies used visualization as a key aspect to enhance explainability, either with SHAP plots [118; 120], regional abnormality maps [119], rainbow boxes [123; 126], or the attention mechanism that highlighted the important words that lead to a prediction [122].  
[PERSON] et al. [121] focused on the use of a user-centered GUI that functions with a semi-automatic strategy, requiring input from the clinicians, and allows for safe interaction. Considering that the focus of this work was on the interface, we did not include this study in Table 2. [PERSON] et al. [117] proposed a CDSS that used a Random Forest to predict the outcome. The authors stated that the use of time-domain features leads to a transparent and explainable CDSS, but there is not sufficient information towards this claim. For this reason, this study is not included in Table 2.