## 4 Discussion
### Implications and Future Research  
The findings in this review have several implications for the current research, development, and clinical implementation of XAI. XAI has shown the potential to enhance clinician confidence and act as a second reader, improving clinical accuracy and decision making. However, while explainability can improve transparency, it can lead to overreliance on AI systems. This is problematic when models are uncertain or incorrect, potentially reducing the decision performance [12]. Therefore, it is necessary to integrate clinical end-users in the development and evaluation processes, focusing on the quality and utility of the explanations provided. Future research should prioritize this by creating \"explainability for end-users\". Interactive explanations, where end-users can refine and adjust explanations,should be explored. Additionally. the traditional XAI methods like LIME or SHAP focus on feature attributions, which may not align with human conceptual reasoning. Future research should explore concept-based XAI methods, as proposed by [PERSON] et al. (2024) [21]. These explanations embed human-understandable concepts into AI outputs, facilitating the integration of expert clinical knowledge. These methods should be developed for tasks beyond computer vision applications, such as classification and regression in oncological contexts. Initial examples can already be seen in the BoF [38] and BI-RADS-Net [52] methods.  
XAI methods combining functions and scopes can provide rich explanations that align more closely with human cognitive processes [21]. Such hybrid methods ensure technical accuracy whilst enhancing transparency and user trust, particularly important in critical care areas such as oncology. However, many XAI methods identified present generalization issues across different datasets or conditions, indicating the need for further refinement of XAI to ensure robustness and sophistication. Policymakers should consider the balance between the benefits of explainability and the potential monetary and safety costs [15].  
Future research should address identified shortcomings to advance the application and effectiveness of XAI in cancer-focused ultrasound image analysis. These include the need for standardized evaluation metrics and the development of an evaluation framework that addresses the clinically relevant impact of the explanations provided. Such a framework should ensure that XAI methods are consistently and quantitatively evaluated for increased reliability and comparability.  
Subsequently, various other studies should be performed to really grasp the impact of XAI in a clinical setting for US imaging. It is of interest to understand how XAI aids clinical decision making or changes treatment plans compared to conventional methods. The cost of implementing XAI in clinical practice is not well documented in current studies. A cost-benefit analysis would be valuable for hospitals considering adoption, weighing potential improvements in diagnosis accuracy against financial costs. Another important topic that requires further research is the user-friendliness of the output of XAI models, especially focusing on the interobserver variability of the end-users and whether the XAI output is rational, instead of derived from biases. Lastly, given the gaps in demographic data, particularly concerning sex and ethnicity, the generalizability of these studies to the broader population may be limited. Future research should prioritize comprehensive reporting of patient demographics to enhance the applicability and relevance of study findings.