# XAI-Based Clinical Decision Support Systems: A Systematic Review
## 2 Related Work
### XAI-Based CDSS
Other applications include the diagnosis of Parkinson disease [74], hip disease [75], Alzheimer's disease, and mild cognitive impairment [76]. By contrast, SHAP uses the explanations on Shapley values that measure the contribution of each feature to the model and is a global explainer method that provides a theoretical interpretation of any dataset using cooperative game theory concepts [77] to calculate the contributions of biomarkers or clinical features (players) for a specific disease outcome (reward) [72]. To predict postoperative malnutrition in children with congenital heart disease, the XGBoost and SHAP algorithms were used to calculate the average of five risk factors (weight one month after surgery, weight at discharge, etc.) for all patients [78]. PHysiologicAl Signal Embeddings (PHASE), a method for transforming time series signals into input features, was first applied to embed body signals with an LSTM model to features extracted using SHAP from EMR/EHR data [79]. In addition, a multi-layer XAI framework utilizing multimodal data, such as MRI and PET images, cognitive scores, and medical histories, have been proposed [80]. SHAP is applied to all layers of the framework, where the first layer performs multiple classification for the early diagnosis of AD. In the second layer, the binary classification score is used to determine the transition from cognitive impairment to AD [80]. Similarly, SHAP has been widely used in various diseases and clinical domains such as predicting readmission [81; 82], COVID-19 [83; 84; 85; 86], liver cancer [87], influenza [88], and malignant cerebral edema [89]. More recently, researchers have utilized LIME and SHAP simultaneously to ensure a convincing description of the system. A hybrid approach combining vision transformer (ViT) and a gated recurrent unit was used to generate LIME heat maps using the top three features from the brain MRI images, and SHAP was used to visualize the model's predictions to demonstrate the validity of data patterns [90]. In addition, the Department of Chronic Kidney Disease also used LIME and SHAP algorithms simultaneously to represent the importance of features in the best model trained by five machine learning methods (random forest, decision tree, naive Bayes, XGBoost, and logistic regression) [91]. Model-based techniques can be classified into model-specific and model-agnostic methods. Model-specific methods utilize the unique features of a model to make decisions, indicating that they can only be applied to the internal operations of a specific model. An  
Figure 3: Trade-off between performance and interpretability [66; 67]. example is Score-CAM [92], which is based on CNNs and compares output for the given input features, thereby indicating their importance. A previous study proposed a system for classifying images from a clock drawing test as a tool for diagnosing dementia that was trained on API-Net [93] and visualized using Score-CAM to provide explainability and transparency [94]. However, model-agnostic methods are model independent and can be applied to any model or algorithm. As a CDSS tool that reduces the model dependency, a COVID-19 symptom severity classifier that utilizes different machine learning models to identify high-risk patients for COVID-19 has been proposed [95]. Complexity-based techniques make machine learning or deep learning models fully interpretable. Interpretability can be categorized into intrinsic interpretability [96] and post hoc interpretability [72] depending on the viewpoint. In general, intrinsic interpretability indicates that a model with a simple architecture can be explained by the trained model itself, whereas post hoc interpretability means that the trained model has a complex architecture and must be retrained to explain this phenomenon. In a study on brain tumor detection based on MRI images, three pre-trained CNNs, DarkNet53 [92], EfficientNet-B0 [97], and DenseNet201 [98], were used to extract features using a hybrid methodology to explain post-interpretability [99].