# Breast Cancer Diagnosis With Explainable Artificial Intelligence (XAI): Uncovering Strengths and Biases

Samita Bai, Sidra Nasir, Rizwan Ahmed Khan, Alexandre Meyer, Hubert Konik

► To cite this version:

Samita Bai, Sidra Nasir, Rizwan Ahmed Khan, Alexandre Meyer, Hubert Konik. Breast Cancer Diagnosis With Explainable Artificial Intelligence (XAI): Uncovering Strengths and Biases. IEEE Access, 2025, 13, pp.205197-205223. 10.1109/ACCESS.2025.3639184. hal-05418268

HAL Id: hal-05418268

https://hal.science/hal-05418268v1

Submitted on 16 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Received 13 November 2025, accepted 24 November 2025, date of publication 1 December 2025, date of current version 9 December 2025.

Digital Object Identifier 10.1109/ACCESS.2025.3639184

TOPICAL REVIEW

# Breast Cancer Diagnosis With Explainable Artificial Intelligence (XAI): Uncovering Strengths and Biases

SAMITA BAI<sup>1</sup>, SIDRA NASIR<sup>2</sup>, RIZWAN AHMED KHAN<sup>3</sup>, ALEXANDRE MEYER<sup>4</sup>, AND HUBERT KONIK<sup>5</sup>

Corresponding author: Rizwan Ahmed Khan (RizwanKhan@iba.edu.pk)

**ABSTRACT** Breast cancer is one of the most common malignancies afflicting women globally, necessitating the use of cutting-edge AI approaches in diagnostic procedures to improve patient outcomes drastically. However, one key difficulty with the most recent AI models is a lack of transparency, making it difficult for medical practitioners to implement these technologies to increase diagnostic accuracy. Many explainable AI (XAI) solutions have been developed to overcome this issue. This survey focuses on applying XAI approaches in breast cancer detection and diagnosis, particularly emphasizing their role in increasing model transparency and clinical decision-making. The article also provides insights into the inherent biases in the most recent machine learning models towards specific XAI approaches, such as the compatibility of Convolutional Neural Networks (CNNs) with visual explanation methods and tree-based models with feature significance evaluations. Finally, the article covers the obstacles to using XAI technologies in clinical practice and the significance of defining standard measures for assessing their performance.

**INDEX TERMS** Explainable artificial intelligence (XAI), breast cancer detection, interpretability, transparency, medical imaging, clinical decision-making.

## I. INTRODUCTION

Breast cancer (BC) ranks as the predominant form of cancer in adults worldwide, with an alarming rate of over 2.3 million new cases each year, as reported by the World Health Organization (WHO) 2022 [1]. Breast cancer survival rates vary significantly across the globe, with a majority of deaths occurring in low- and middle-income countries. Early detection is crucial as it leads to a clinical cure rate of over 90%, but this decreases significantly as the cancer progresses [2]. Integrating artificial intelligence (AI) techniques into breast cancer diagnosis can improve early diagnosis and patient outcomes. AI systems' ability to analyze large datasets, recognize patterns, and perform predictive analysis can

enhance decision-making and personalized care, ultimately improving the precision and effectiveness of diagnosis and treatment strategies [3].

Breast cancer is categorized into three primary groups that reflect the stage of the disease and the complexity of treatment. Primary breast cancer, which makes up 80% of cases, is found initially in the breast without having spread beyond the breast or the lymph nodes under the arms. Secondary breast cancer, constituting 14% of cases, involves cancer spreading to other parts of the body, such as the lungs, bones, or liver. The smallest group, which accounts for 6% of cases, includes those with both primary and secondary characteristics due to recurrence, indicating that the cancer has returned after treatment, either at the original site or elsewhere. The studies reviewed in this article are distributed according to the abovementioned categories.

The associate editor coordinating the review of this manuscript and approving it for publication was Juan Wang<sup>1</sup>.

<footnote n="1">¹Canadian Institute of Cybersecurity (CIC), University of New Brunswick, Fredericton, NB E3B 5A3, Canada</footnote>

<footnote n="2">²Dipartimento di Informatica, Università di Verona, 37134 Verona, Italy</footnote>

<footnote n="3">³Department of Computer Science, School of Mathematics and Computer Science, Institute of Business Administration, Karachi 75270, Pakistan</footnote>

<footnote n="4">⁴Université Claude Bernard Lyon 1, 69100 Villeurbanne, France</footnote>

<footnote n="5">⁵Laboratoire Hubert Curien (LHC), UMR5516, Université de Saint-Étienne, 42000 Saint-Etienne, France</footnote>
AI optimizes diagnostics and treatments by analyzing vast medical data, including Electronic Health Records (EHRs) and image data, to identify patterns and accurately predict outcomes. It aids in detecting diseases like cancer in medical images, enhancing diagnoses and patient outcomes. By analyzing medical history and genetic data, AI enables early disease detection and suggests preventive measures, improving patient care. Moreover, by automating routine tasks, AI reduces healthcare costs and streamlines workflows, allowing providers to focus on complex cases and improving patient access to care [4].

However, some limitations must be considered. One of the limitations is the inadequate diversity of the training data utilized. For instance, if the training data consists of images of breasts from only one ethnic group, it may not accurately identify cancer in breasts from other ethnic groups [5]. Moreover, the systems might lack clarity, posing a challenge for doctors to comprehend the reasoning guiding the system's decision-making process. This lack of transparency could be troublesome in medical contexts, where human expertise and intuition are necessary for accurate diagnosis [6], [7].

To address these limitations, Explainable Artificial Intelligence (XAI) could be a promising approach that offers clear explanations of AI systems' decision-making processes. This clarity increases trust in recommendations for both medical professionals and patients, aiding informed decision-making. Additionally, interpretability helps reduce false positives and negatives, enhancing accuracy and minimizing risks in diagnosis and treatment [8], [9], [10]. This article provides a comprehensive overview of XAI techniques leveraged for various state-of-the-art breast cancer detection applications. Examples of these XAI techniques include case-based reasoning (CBR), visualization, and feature importance analysis. Additionally, the emphasis is on the main XAI methods discussed in the paper - LIME, SHAP, and GRAD CAM- rather than focusing on others.

This article presents a systematic literature review (SLR) examining the use of Explainable Artificial Intelligence (XAI) strategies in breast-cancer detection and diagnosis. We evaluate how XAI methods enhance model transparency and trust in automated systems, identify prevailing gaps across existing studies, highlight promising future paths for XAI in medical imaging, and investigate the influence of XAI on clinical decision-making and patient outcomes.

While prior reviews have focused on the broader application of XAI in medical imaging, this paper narrows the scope to breast-cancer diagnosis only. In doing so, our contributions are three-fold: (a) We present a novel mapping between explanation-methods, model types and bias sources (i.e., a bias/explanation-method/model-type matrix). (b) We conduct a quantitative meta-analysis of 44 selected studies, assessing patterns of explanation-method usage and model-explanation alignment. (c) We discussed about evaluation frameworks that brings together bias detection and fairness metrics, clinical workflow integration and usability considerations for XAI in the breast-cancer context. These

elements provide a focused and actionable overview that goes beyond general XAI surveys and directly supports the translation of XAI technologies into breast-cancer diagnostic workflows.

The structure of the article is as follows: Section II outlines the review methodology and protocol. Section III presents the meta-analysis of selected studies, including publication sources, citation trends, geographic distribution and dataset modalities. Section IV provides an overview of XAI techniques and analyzes their frequency of application across various machine learning (ML) and deep learning (DL) models. Section V discusses the specific XAI methods utilized for explaining ML/DL models applied to primary and secondary breast-cancer detection. Section VI offers a comprehensive analysis of the most frequently employed XAI techniques and their implications, while Section VII concludes with a discussion of key implications and future research directions.

## II. REVIEW METHOD

This paper uses the PRISMA (Preferred Reporting Items on Systematic Reviews and Meta-analysis) [11], [12] model. As mentioned above, this SLR aims to identify and present literature on XAI in breast cancer detection and diagnosis, with a particular emphasis on their role in increasing model transparency, thus enhancing trust in automated detection systems among healthcare professionals and patients. This study will examine different XAI methods, including model-agnostic and model-specific approaches, to assess their ability to enhance breast cancer detection accuracy. In summary, the objectives of this review are:

1) To identify, collect, and synthesize existing research on the application of XAI techniques in the detection and diagnosis of breast cancer.

2) To assess various XAI approaches, including both model-agnostic and model-specific techniques, in terms of their effectiveness in improving the transparency and diagnostic accuracy of ML and DL models used for breast cancer detection.

3) To explore the obstacles faced in integrating XAI technologies into clinical workflows for breast cancer diagnosis and to emphasize the importance of developing standardized metrics for evaluating the performance and effectiveness of XAI models.

The study will also investigate how XAI techniques can enhance trust in automated detection systems among healthcare professionals and patients.

## A. REVIEW PROTOCOL

This study was initiated by developing a thorough review protocol aligned with established methodologies for comprehensive literature surveys [13]. The protocol serves as a guide for the survey process by outlining key components such as the study's background, the literature search strategy,
<figure><img src="image_2.png" /><figcaption>FIGURE 1. Complete overview of studies selection process.</figcaption></figure>

methods for data extraction, and criteria for assessing the
quality of included studies.

The review protocol separates SLRs from traditional or
narrative literature reviews. It improves the uniformity of the
review process. It reduces researcher bias by demanding the
provision of an explicit search strategy as well as established
criteria for the inclusion and removal of studies.

B. CRITERIA FOR INCLUSION AND EXCLUSION

Setting precise inclusion and exclusion criteria guarantees that only pertinent articles on breast cancer diagnosis using XAI are included in this study. We focused on studies published in reputable journals from January 2019 to June 2025. Our initial search based on the listed keywords resulted in 6302 research articles (refer to Figure 1 for a complete overview of the selection process). After a thorough review of the articles, we excluded articles that were unrelated to an XAI but appeared in the search because of keyword match. Additionally, articles were also excluded based on duplicity and non-availability of full text.

C. SEARCH STRATEGY

To conduct this survey, our search strategy aimed to gather
relevant research articles addressing the application of XAI
in breast cancer detection. Both automatic and manual search
methods were employed to ensure comprehensive coverage
of the available literature.

The automatic search was conducted using reputable
databases known for hosting scholarly works related to XAI
and healthcare, refer to Figure 1. These databases were
selected for their extensive collection of relevant research
articles in the field. While other sources such as magazines,
working papers, newspapers, books, and blogs contain
valuable information, they were not included in this survey
due to the lack of rigorous peer-review processes, which
could potentially compromise the reliability and quality of
the findings.

A variety of general keywords exploring different aspects
of XAI in breast cancer diagnosis were used for the search.

These keywords were derived from the research questions
and study titles to ensure relevance. The selected keywords
include the following:

* "Explainable AI" AND "Breast Cancer"

* "Interpretable Machine Learning" AND "Breast Cancer Diagnosis"

* "XAI" AND "Oncology"

* "Explainable Deep Learning" AND "Breast Cancer Detection"

* "Transparent AI Models" AND "Breast Cancer Prognosis"

* "Interpretable AI" AND "Mammography Analysis"

* "Explainable Artificial Intelligence" AND "Medical Imaging"

* "XAI" AND "Breast Cancer Risk Prediction"

* "Interpretable Neural Networks" AND "Breast Cancer Screening"

* "Explainable Machine Learning" AND "Cancer Research"

These keywords were designed to capture a broad range of
relevant articles, ensuring a comprehensive understanding of
the topic.

D. QUALITY ASSESSMENT CRITERIA

The quality assessment criteria (QAC) aim to determine the
overall quality of a selected group of studies. The following
criteria were applied to evaluate the quality of the chosen
studies.

1) Are the subjects covered in the research study related to the aims of this review article?

2) Does the research publication describe the approach and methods clearly?

3) Is the data used in the study standardized? If not, is ethical approval mentioned in the study?

4) Is data analysis explained with appropriate examples?

To evaluate each study's trustworthiness, we assessed
62 chosen papers using the previously defined quality rating
criteria. The four QA criteria have been adapted from [13].
The quality of each study was evaluated using the score
received for each QA question, with each question worth a maximum of two points. Studies scoring six or higher out of eight were considered qualified for inclusion. As a result, papers with scores below six were omitted from the study. Based on this criterion, 44 papers were eventually chosen for inclusion in this review. Figure 1 provides a thorough summary of the selection process.

### III. META ANALYSIS

This section presents a comprehensive meta-analysis of the selected research studies. The analysis focuses on several essential factors, including the sources of publication, citation counts, and temporal trends. Additionally, a demographic overview is provided to highlight the geographic and institutional distribution of the studies. Lastly, we examine the various dataset modalities used in the selected research. This extensive review aims to provide insights into the effects, relevance, and spread of research on this topic.

#### A. SELECTED STUDIES

Table 1 summarizes the metadata fields extracted from the selected 44 research studies. Meanwhile, Table 2 presents the distribution of articles across different quartiles and publishing platforms related to breast cancer detection and classification using XAI. It categorizes publications into Quartile 1 (Q1) and Quartile 2 (Q2) according to Scimago rankings.

The citation data, obtained from Google Scholar, provided a detailed view of the scholarly impact of the included research articles. This combination of quartile categorization by Scimago Journal Rank (SJR) and citation analysis from Google Scholar offers valuable perspectives on the research's distribution and influence, aiding in a thorough understanding and evaluation of the field within the academic community.

#### B. TEMPORAL ANALYSIS

The temporal analysis of research reveals distinct patterns in annual publications, indicative of evolving scholarly interest and engagement with the subject matter. The minimal publication activity in 2019 was followed by a significant increase in research from 2020 to 2021 and a slight decrease in 2022, with a resurgence in 2023 as shown in Figure 2, suggests that factors such as funding, academic priorities, and technological advancements might have influenced these trends. Thus, highlighting the growing importance and potential for further exploration and innovation in integrating XAI into breast cancer diagnosis.

#### C. GEOGRAPHIC DISTRIBUTION

The geographic distribution of research publications reflects the global commitment to advancing the field of breast cancer detection and classification using XAI. The map illustrated in Figure 3, highlights the number of publications from each country based on author affiliation in this domain. China is at the forefront with six publications, followed by Italy with four and Turkey with three publications, respectively. This

widespread participation from various countries highlights the global collaboration and concerted efforts to push forward research in this vital healthcare sector.

#### D. DATASET AND MODALITIES FOR BREAST CANCER

Early detection and accurate classification of breast tumors as benign or malignant are crucial in preventing the progression of breast cancer and reducing associated mortality rates. The correct diagnosis enables timely and effective treatment, significantly improving patient outcomes. Various imaging modalities play essential roles in these early detection efforts, with each technology chosen based on specific diagnostic needs.

In this section, we examine the various dataset modalities used by the selected research studies. The analysis in graphically presented in Figure 4. In this figure, each bar represents a specific imaging modality (name written at the bottom), with the accompanying dataset names given at the top. The height of each bar represents the total number of studies that used that modality, giving a clear visual depiction of how frequently each dataset modality was used. This analysis illustrates the variability in data utilization among studies, providing insights into potential patterns and gaps.

Among the most commonly employed imaging modalities for detecting and diagnosing breast cancer are mammograms, CT Scans, ultrasounds, and Histopathological (Whole Side Images (WSI)). These modalities offer unique benefits that are instrumental in identifying cancerous changes at an early stage. For instance, mammograms are a primary tool in breast cancer screening programs due to their effectiveness in detecting early signs of cancer through low-dose X-ray images. These images are easy to capture and are often used as the first test in breast cancer detection. In addition to traditional imaging techniques, integrating advanced modalities like Digital Breast Tomosynthesis (DBT) and various genomic datasets enhances the depth and accuracy of breast cancer analysis. Additionally, the features dataset refers to the variables or attributes that describe each instance in a dataset. These features serve as inputs to the model, helping it to identify patterns and make predictions. They can be represented in a structured, tabular format, typically comprising numerical values (e.g., age, income) and categorical values (e.g., gender, product type), or even pixel-based values from images that capture visual information. The visual representation of each modality is depicted in Figure 5.

The literature review has indicated that XAI methods have been significantly employed across various modalities, with the features dataset emerging as a prominently utilized resource for enhancing the transparency and interpretability of models in breast cancer research, as exhibited in Figure 4. A feature dataset in breast cancer research encompasses specific attributes gathered from various diagnostic tools, including imaging, genetic tests, and biochemical analysis. Imaging features from mammograms detail tumor characteristics like shape and texture [72]. Histopathological features
<table><caption>TABLE 1. Selected studies and citations, ordered by their year of publication (Last accessed on November 2025).</caption><thead><tr><td>Year</td><td>Article Title</td><td>Authors</td><td>Cit.</td><td>Ref.</td></tr></thead><tbody><tr><td>2019</td><td>Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach<sup>Q1</sup></td><td>Lamy et al.</td><td>403</td><td>[14]</td></tr><tr><td>2020</td><td>Explainable AI (XAI) for anatomic pathology<sup>Q1</sup></td><td>Tosun et al.</td><td>103</td><td>[15]</td></tr><tr><td>2020</td><td>A case-based ensemble learning system for explainable breast cancer recurrence prediction<sup>Q1</sup></td><td>Gu et al.</td><td>105</td><td>[16]</td></tr><tr><td>2020</td><td>Convolutional neural network-based models for diagnosis of breast cancer<sup>Q1</sup></td><td>Masud et al.</td><td>205</td><td>[17]</td></tr><tr><td>2020</td><td>Automated BC detection in digital mammographs of various densities via deep learning<sup>Q2</sup></td><td>Suh et al.</td><td>152</td><td>[18]</td></tr><tr><td>2021</td><td>XAI and susceptibility to adversarial attacks: a case study in the classification of breast ultrasound images<sup>Q1,4,14,15</sup></td><td>Rasaee et al.</td><td>26</td><td>[19]</td></tr><tr><td>2021</td><td>Explainable autoencoder-based representation learning for gene expression data<sup>B10,B14,14,15</sup></td><td>Yu et al.</td><td>6</td><td>[20]</td></tr><tr><td>2021</td><td>Explainings of graph CNN: patient-specific mol. subnetworks responsible for metastasis prediction in BC<sup>Q1</sup></td><td>Cherada et al.</td><td>127</td><td>[21]</td></tr><tr><td>2021</td><td>ML Approaches to Classify Primary and Metastatic Cancers Using Tissue of Origin-Based DNA Methylation Profiles<sup>Q1</sup></td><td>Modhukur et al.</td><td>40</td><td>[22]</td></tr><tr><td>2021</td><td>Axillary lymph node metastasis status prediction of early-stage breast cancer using CNN<sup>Q1</sup></td><td>Lee et al.</td><td>90</td><td>[23]</td></tr><tr><td>2021</td><td>One step further into the blackbox: a pilot study of how to build more confidence around an AI-based decision system of breast nodule assessment in 2D ultrasound<sup>Q1</sup></td><td>Dong et al.</td><td>46</td><td>[24]</td></tr><tr><td>2021</td><td>XAI Reveals Novel Insight into Tumor Microenvironment Conditions Linked with Better Prognosis in Patients with BC<sup>Q1</sup></td><td>Chakrbortyet et al.</td><td>49</td><td>[25]</td></tr><tr><td>2021</td><td>Automated scoring of CerbB2/HER2 receptors using histogram-based analysis of immunohistochemistry BC tissue images<sup>Q1</sup></td><td>Kabakcci et al.</td><td>25</td><td>[26]</td></tr><tr><td>2021</td><td>Morphological and molecular breast cancer profiling through explainable machine learning<sup>Q1</sup></td><td>Binder et al.</td><td>194</td><td>[27]</td></tr><tr><td>2021</td><td>A deep learning classifier for digital breast tomosynthesis<sup>Q1</sup></td><td>Ricciardi et al.</td><td>49</td><td>[28]</td></tr><tr><td>2021</td><td>MGBN: Convolutional neural networks for automated benign and malignant breast masses classification<sup>Q1</sup></td><td>Lou et al.</td><td>23</td><td>[29]</td></tr><tr><td>2021</td><td>A roadmap towards breast cancer therapies supported by explainable artificial intelligence<sup>Q2</sup></td><td>Amorose et al.</td><td>64</td><td>[30]</td></tr><tr><td>2021</td><td>Comparison of feature importance measures as explanations for classification models<sup>Q2</sup></td><td>Saarela et al.</td><td>490</td><td>[31]</td></tr><tr><td>2021</td><td>Explainable ML can outperform Cox regression predictions and provide insights into breast cancer survival<sup>Q2</sup></td><td>Moncada et al.</td><td>345</td><td>[32]</td></tr><tr><td>2021</td><td>Cancer Cell Profiling Using Image Moments and Neural Networks with Model Agnostic Explainability: A Case Study of Breast Cancer Histopathological (BreakHis) Database<sup>Q2</sup></td><td>Kaplun et al.</td><td>27</td><td>[33]</td></tr><tr><td>2022</td><td>Benchmarking Counterfactual Algorithms for XAI: From White Box to Black Box<sup>Q1,4,14,15</sup></td><td>Moreira et al.</td><td>9</td><td>[34]</td></tr><tr><td>2022</td><td>Explainable machine learning of the breast cancer staging for designing smart biomarker sensors<sup>Q1</sup></td><td>Idrees et al.</td><td>22</td><td>[35]</td></tr><tr><td>2022</td><td>Potential of Non-Contrast-Enhanced Chest CT Radiomics to Distinguish Mol. Subtypes of BC: A Retrospective Study<sup>Q2</sup></td><td>Wang et al.</td><td>19</td><td>[36]</td></tr><tr><td>2022</td><td>Applications of Explainable Artificial Intelligence in Diagnosis and Surgery<sup>Q2</sup></td><td>Zhang et al.</td><td>384</td><td>[37]</td></tr><tr><td>2022</td><td>Shape-Based Breast Lesion Classification Using Digital Tomosynthesis Images: The Role of XAI<sup>Q2</sup></td><td>Hussain et al.</td><td>49</td><td>[38]</td></tr><tr><td>2022</td><td>Over-the-Counter Breast Cancer Classification Using Machine Learning and Patient Registration Records<sup>Q2</sup></td><td>Hanis et al.</td><td>6</td><td>[39]</td></tr><tr><td>2022</td><td>Explainable Multi-Class Classification Based on Integrative Feature Selection for Breast Cancer Subtyping<sup>Q2</sup></td><td>Meshoul et al.</td><td>13</td><td>[40]</td></tr><tr><td>2023</td><td>A Hybrid Algorithm of ML and XAI to Prevent Breast Cancer: A Strategy to Support Decision Making<sup>Q1</sup></td><td>Silva et al.</td><td>46</td><td>[41]</td></tr><tr><td>2023</td><td>Explainable quantum clustering method to model medical data<sup>Q1</sup></td><td>Deshmukh et al.</td><td>34</td><td>[42]</td></tr><tr><td>2023</td><td>Analyzing breast cancer invasive disease event classification through explainable artificial intelligence<sup>Q1</sup></td><td>Massafra et al.</td><td>31</td><td>[43]</td></tr><tr><td>2023</td><td>Information bottleneck-based interpretable multitask network for breast cancer classification and segmentation<sup>Q1</sup></td><td>Wang et al.</td><td>85</td><td>[44]</td></tr><tr><td>2023</td><td>Applying Ex. ML Models for Detection of BC Lymph Node Metastasis in Patients Eligible for Neoadjuvant Treatment<sup>Q1</sup></td><td>Vrdoljak et al.</td><td>28</td><td>[45]</td></tr><tr><td>2023</td><td>Peripheral blood mononuclear cell-derived biomarker detection using explainable Artificial Intelligence (XAI) provides the better diagnosis of breast cancer<sup>Q2</sup></td><td>Kumar et al.</td><td>38</td><td>[46]</td></tr><tr><td>2023</td><td>Applying Deep Learning Methods for Mammography Analysis and Breast Cancer Detection<sup>Q2</sup></td><td>Prodan et al.</td><td>66</td><td>[47]</td></tr></tbody></table>
<table><caption>TABLE 1. (Continued.) Selected studies and citations, ordered by their year of publication (Last accessed on November 2025).</caption><thead><tr><td>Year</td><td>Article Title (continued)</td><td>Authors</td><td>Cit.</td><td>Ref.</td></tr></thead><tbody><tr><td>2023</td><td>An Explainable Artificial Intelligence Model for the Classification of Breast Cancer<sup>Q1</sup></td><td>Khater et al.</td><td>35</td><td>[48]</td></tr><tr><td>2023</td><td>Cancer Metastasis Prediction and Genomic Biomarker Identification through Machine Learning and eXplainable Artificial Intelligence in Breast Cancer Research<sup>Q2</sup></td><td>Yagin et al.</td><td>4</td><td>[49]</td></tr><tr><td>2023</td><td>Attention guided Grad-CAM: an improved explainable artificial intelligence model for infrared breast cancer detection<sup>Q1</sup></td><td>Raghvan et al.</td><td>56</td><td>[50]</td></tr><tr><td>2024</td><td>Explainable Artificial Intelligence in Quantifying Breast Cancer Factors: Saudi Arabia Context<sup>Q2</sup></td><td>Alelyani et al.</td><td>11</td><td>[51]</td></tr><tr><td>2024</td><td>A novel approach for breast cancer detection using optimized ensemble learning framework and XAI<sup>Q2</sup></td><td>Munshi et al.</td><td>53</td><td>[52]</td></tr><tr><td>2024</td><td>Quantitative evaluation of Saliency-Based Explainable artificial intelligence (XAI) methods in Deep Learning-Based mammogram analysis<sup>Q1</sup></td><td>Cerekci et al.</td><td>31</td><td>[53]</td></tr><tr><td>2024</td><td>Efficient breast cancer detection using neural networks and explainable artificial intelligence<sup>Q1</sup></td><td>Murugan et al.</td><td>9</td><td>[54]</td></tr><tr><td>2025</td><td>An improved XAI-based DenseNet model for breast cancer detection using reconstruction and fine-tuning<sup>Q1</sup></td><td>Talukder</td><td>4</td><td>[55]</td></tr><tr><td>2025</td><td>XAI-RACapsNet: Relevance aware capsule network-based breast cancer detection using mammography images via explainability O-net ROI segmentation<sup>Q1</sup></td><td>Alhussen et al.</td><td>57</td><td>[56]</td></tr><tr><td>2025</td><td>An explainable AI-driven deep neural network for accurate breast cancer detection from histopathological and ultrasound images<sup>Q1</sup></td><td>Alom et al.</td><td>12</td><td>[57]</td></tr></tbody></table>
<table><caption>TABLE 2. The Quartile information and number of citations for XAI in breast cancer domain.</caption><thead><tr><td>Quartile</td><td>Number of Articles</td><td>Total Citations</td></tr></thead><tbody><tr><td>Q1</td><td>27</td><td>1909</td></tr><tr><td>Q2</td><td>14</td><td>1766</td></tr><tr><td>Arxiv</td><td>2</td><td>35</td></tr><tr><td>BioArxiv</td><td>1</td><td>6</td></tr></tbody></table>

involve the microscopic analysis of tissue samples [33], and genomic features highlight gene expression and mutations [20], [25], [40], [49]. Biochemical features such as levels of enzymes, including lactate dehydrogenase, alkaline phosphatase, and aromatase, as well as hormone assays, provide insights into biological activities associated with cancer [32], [43]. Clinical features integrate personal data such as age and family history [41], [45].

These comprehensive modalities encompass not just the screening and detection but also the segmentation and classification of breast cancer, utilizing a broad range of datasets and imaging technologies. Each modality contributes uniquely to the complex process of cancer diagnosis, from the detailed anatomical insights provided by the genetic information revealed through genomic databases. This multi-modal strategy is critical for developing a nuanced understanding of breast cancer, leading to more personalized and effective treatment strategies. Thus, the blend of traditional and advanced imaging modalities, enriched by diverse datasets, highlights the multidisciplinary nature of contemporary breast cancer research and treatment.

## IV. TAXONOMY AND FOUNDATIONS OF EXPLAINABLE ARTIFICIAL INTELLIGENCE (XAI) TECHNIQUES

ML and DL models have been extensively used in the healthcare domain to predict and diagnose different diseases, such as Autism Spectrum Disorder (ASD) [73], and other kinds of cancer, for instance, lung cancer [74], breast cancer [75], etc. However, the black-box aspect of these models complicates interpretability, which is crucial for clinical decision-making [76]. XAI approaches have been developed to increase transparency and trust in model predictions as explained in Figure 6.

This section presents a taxonomy of XAI methodologies, focusing on the three most often used approaches: SHapley Additive Explanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME), and Gradient-weighted Class Activation Mapping (Grad-CAM). The strengths and weaknesses have been discussed in Section VI.

As this review article is a comprehensive examination of individual studies, aiming to pinpoint the algorithms and methodologies proposed, this section showcases the relationship between ML/DL models and XAI techniques, as outlined in Table 3.

Table 5 (in Section VI) presents a per-study view and statistical analysis of the utilized XAI techniques. When

combined with the Table 3, these insights reveal biases of different families of machine learning algorithms towards specific XAI techniques.

A comprehensive taxonomy of XAI techniques is presented in Figure 7. In general, XAI models can be categorized into two types, including **intrinsic** and **post-hoc models**. It is possible to explain and comprehend intrinsic models, also called transparent models. Typical examples of transparent models include linear regression, logistic regression, and decision trees [112]. On the other hand, post-hoc is categorized into **model-specific** and **model-agnostic**. XAI explanations can also be categorized into two main types based on their scope, which includes **global** and **local**. A global explanation offers a comprehensive overview of the entire model, while a local explanation focuses on a single prediction or output. The specifics of various XAI models are outlined in the following subsections, based on their prevalence in ML/DL applications within the breast cancer domain. The following subsections provide a detailed discussion of widely used XAI models.

### A. EVALUATION FRAMEWORKS AND METRICS FOR EXPLAINABLE AI

XAI seeks to bridge the gap between the opaque decision-making of AI models and the need for human interpretability, fostering trust and reliability in AI-driven systems. However, measuring the effectiveness of XAI requires a multifaceted approach that addresses both technical rigor and user-centric considerations. A robust evaluation framework ensures explanations are not only accurate and reliable but also actionable and comprehensible to diverse stakeholders.

Measuring Explainable Artificial Intelligence (XAI) involves a multifaceted evaluation that balances technical accuracy with user-centric effectiveness, ensuring explanations are both reliable and actionable. Fidelity is a critical metric, assessing how well explanations mirror the model's decision-making process. Methods such as Correlation Metrics and Fidelity Scores, like Mean Squared Error (MSE), compare explanations with model predictions to validate their accuracy. Stability and Consistency are equally important, as they measure whether explanations remain reliable under slight input changes. This can be evaluated through Variance Analysis and Consistency Scores, ensuring trust in the system.

Another vital consideration is Computational Efficiency, which focuses on the time and resources needed to generate explanations. Tools like Runtime Analysis and Resource Utilization are used to assess whether the system is suitable for real-time or resource-constrained applications. Beyond technical metrics, Comprehensibility and Usability are central to the user experience. These are often measured using User Surveys, Think-Aloud Protocols, and the System Usability Scale (SUS) to gather feedback on the clarity, coherence, and satisfaction users derive from explanations.
<figure><img src="image_3.png" /><figcaption>FIGURE 2. The distribution of studies about breast cancer diagnosis using XAI spanning from 2019-2025.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>FIGURE 3. Investigating breast cancer diagnosis with XAI: Analyzing first author affiliation belonging to different countries.</figcaption></figure>

Task-specific effectiveness is another key area, particularly how explanations influence Decision-Making Impact and support Debugging and Model Improvement. Metrics such as Task Performance and Decision Confidence evaluate whether explanations help users make faster, more accurate decisions. At the same time, Error Detection Rates and Bias Identification measure the ability of explanations to uncover model errors and biases, enhancing both trust and model refinement.

Subjective metrics like Trust and Transparency focus on user perceptions, assessed through Trust Scales and surveys on Perceived Transparency, ensuring that users feel confident in the model's reliability. Satisfaction and User

Confidence are equally significant, gauged via Satisfaction Surveys and Confidence Ratings, which reflect the overall success of the explanation system in engaging users effectively.

To support these evaluations, various frameworks and toolkits are available. Tools like LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations) offer built-in metrics for assessing fidelity and consistency. Additionally, Explainable AI Evaluation Frameworks (XAI-EF) provide structured methodologies for user-centric evaluation, while Benchmark Datasets ensure objective comparison of explanation quality across different models and applications. These comprehensive measures
<figure><img src="image_5.png" /><figcaption>**FIGURE 4.** Graph illustrating the distribution of selected studies based on the use of specific imaging modalities or datasets. Features Wisconsin Breast Diagonstic Dataset (WBDC) [58] and others), Mammograms (Breast Cancer Dataset (BCD) [59], Mammographic Mass (MM) [60], MIAS, Digital Database for Screening Mammography (DDSM) [61], INbreast [62], Wisconsin Breast Cancer (WBC) [58], and Radiological Society of North America (RSNA) [63] and others), Genes (The Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO) (TCG-GEO) [64], [65], and National Center for Biotechnology Information (NCBI) and Gene Expression Omnibus (NCBI-GEO) [66], [67]), Ultrasound (Breast Ultrasound Image (PlumX) [68], Breast Ultrasound Image (BUSI) [69], and others), Histopathological (BreakHis [70], BACH, ITU-MED-1, ITU-MED-2 [26], and others), Digital Breast Tomosynthesis (DBT) (Two in House (TiH) [28] and others), Computed Tomography (CT)(others), and Infrared (Mastology DRM (MDRM) [71]). Others: Proprietary datasets.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**FIGURE 5.** Visual representation of the modalities.</figcaption></figure>

ensure that XAI systems are not only technically sound but also effective in real-world applications. The explanation

provided in this section is based on/conferred from the article by Sokol and Flach [113].
<figure><img src="image_7.png" /><figcaption>FIGURE 6. Concept of black box models focusing the essential aspect of explainability in machine learning [76].</figcaption></figure>

<table><caption>TABLE 3. Association of machine learning/deep learning models with utilized XAI methods in reviewed studies (holistic view) CBR= Case-based Reasoning, (G)LRP= (Graph) Layer-wise Relevance Propagation, PIMP= Permutation Importance, PDP= Partial Dependence Plot, ADR= Adaptive Dimension Reduction, RF= Random Forest, LightGBM= Light Gradient Boosting Machine, SVM= Support Vector Machine, MIB-Net= Multitask Information Bottleneck Network, MaxVit= Multi-axis vision transformer, LSTM= Long Short-term Memory.</caption><thead><tr><th></th><th>SHAP [77]</th><th>LIME [78]</th><th>GRAD-CAM [79]</th><th>CAM [80]</th><th>AGG-CAM [49]</th><th>CBR [81]</th><th>GLRP [21]</th><th>ADR [30]</th><th>LRP [82]</th><th>PIMP [83]</th><th>PDP [84]</th></tr></thead><tbody><tr><td>RF [85]</td><td>√</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>√</td><td>√</td></tr><tr><td>RF (Survival) [86]</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Gradient Boost [87]</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>AdaBoost [88]</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>GoogLeNet [89]</td><td></td><td></td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>AlexNet [90]</td><td></td><td></td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>VGG [91]</td><td></td><td></td><td>√</td><td></td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Xception [92]</td><td></td><td></td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>ResNet [93]</td><td>√</td><td>√</td><td>√</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>DenseNet [94]</td><td></td><td>√</td><td>√</td><td>√</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mask R-CNN [95]</td><td></td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Graph CNN [96]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>√</td><td></td><td></td><td></td></tr><tr><td>MobileNet [97]</td><td></td><td>√</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SqueezeNet [98]</td><td></td><td>√</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>ResNeXt [99]</td><td></td><td>√</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MIB-Net [44]</td><td></td><td></td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>DarkNet [100]</td><td></td><td></td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>EfficientNet [101]</td><td></td><td></td><td>√</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>XGBoost [102]</td><td>√</td><td>√</td><td></td><td></td><td>√</td><td>√</td><td></td><td></td><td></td><td></td><td>√</td></tr><tr><td>SVM [103]</td><td>√</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>LightGBM [104]</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>CatBoost [105]</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Autoencoder [106]</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td>√</td><td></td><td></td><td></td></tr><tr><td>Clustering [107]</td><td></td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MaXViT [108]</td><td></td><td></td><td></td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Naïve Bayes [109]</td><td>√</td><td>√</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Bag-of-Words [110]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>√</td><td></td><td></td></tr><tr><td>LSTM [111]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

## B. SHapley ADDITIVE ExPlanations (SHAP): GAME-THEORETIC FRAMEWORK FOR FEATURE ATTRIBUTION AND MODEL EXPLAINABILITY

Lundberg and Lee [77] presented a unified framework, SHAP for predicting interpretations. Each feature

is assigned an essential value for a particular prediction.

SHAP is a post-hoc and model-agnostic, explainable model in machine learning. It can explain the output of any black box model by computing the contribution of each
<figure><img src="image_8.png" /><figcaption>FIGURE 7. XAI taxonomy; A detailed depiction.</figcaption></figure>
feature to the prediction. SHAP values provide a measure for the impact of each feature in a given data instance on the model's output, offering both a global and local understanding of the model.

The mathematical foundation of SHAP is rooted in game theory [114], specifically in the concept of Shapley values. Its design ensures equitable distribution of the “payout” (prediction) among the “players” (features).

## 1) SHAP EQUATION

The primary SHAP equation is formulated as follows:

$$ \phi(f, x) = \sum_{z' \subseteq x'} \frac{|z'|! (M - |z'| - 1)!}{M!} [f_x(z') - f_x(z' \setminus i)] \quad (1) $$

where, $\phi(f, x)$ represents the SHAP value for a prediction $f$ given the input $x$, $z'$ is a subset of the features, $M$ is the total number of features, $|z'|$ is the number of non-zero entries in $z'$ and $f_x(z')$ is the prediction for the subset $z'$, and $f_x(z' \setminus i)$ is the prediction without a particular feature.

For the given equation 1 the underlying properties taken into consideration are as follows;

1) **Local Accuracy:** This ensures that the explanation model's output $g(x')$ matches the output of the original model $f(x)$ for any input $x$. The relationship is given by:

$$ f(x) = g(x') = \phi_0 + \sum_{i=1}^{M} \phi_i x'_i \quad (2) $$

* $\phi_0$ is the base value (model's output for the average of all inputs).

* $\phi_i$ are the SHAP values attributing the difference from the base value to each feature.

2) **Missingness:** This principle asserts that if a feature is absent (or has a value of zero) in the input, it should not influence the prediction. It's mathematically represented as:

$$ x'_{i} = 0 \Rightarrow \phi_{i} = 0 \quad (3) $$

This ensures that features not present do not impact the model's output.

3) **Consistency:** The SHAP values should remain consistent when the influence of a feature on the model output increases or remains unchanged. If increasing or keeping a feature constant increases the output, the SHAP value for that feature should not decrease. This is crucial to ensuring that more influential features are always given higher importance.

Let $f_x(z') = f(h_x(z'))$ and $z'_i$ denote setting $z'_i = 0$. For any two models, $f$ and $f'$, if

$$ f'_{x}(z') - f'_{x}(z' \setminus i) \geq f_{x}(z') - f_{x}(z' \setminus i) \quad (4) $$

$$ \forall z' \in \{0, 1\}^M, \text{ then } \phi_i(f', x) \geq \phi_i(f, x) \quad (5) $$

SHAP values provide a mathematically rigorous, fair, and consistent method for interpreting machine learning

model predictions, making them an invaluable tool in explainable AI.

## C. CLASS ACTIVATION MAP AND GRADIENT-WEIGHTED CLASS ACTIVATION MAP (GRAD-CAM): VISUAL EXPLAINABILITY FOR DEEP LEARNING MODELS

Zhou et al. [80] introduced class activation maps (CAM) using Global Average Pooling (GAP) in CNNs after the final convolutional layer to decrease the size of the image and reduce the parameters to avoid overfitting. CAM helps to identify the discriminative regions of an image that affect the output of CNNs by performing global average pooling on the convolutional feature maps and using those as features for a fully connected layer that produces the desired output. Class activation weights are then computed based on the gradients of the predicted class score relying on the feature maps. These weights indicate the importance of each feature map for the target class.

CAM generates heatmaps employing $f_k(x, y)$, the pixel-wise activation of unit $k$ across spatial coordinates $(x, y)$ in the last convolutional layers, weighted by $w_k^c$, the coefficient corresponding to unit $k$ for class $c$ at pixel $(x, y)$ is formulated as follows:

$$ M_c(x, y) = \sum_k w_k^c f_k(x, y) \quad (6) $$

## 1) GRAD-CAM

Grad-CAM (Gradient-weighted Class Activation Mapping) [79] is a post-hoc technique that provides visual explanations for the decisions made by convolutional neural networks (CNNs). It achieves this by utilizing the gradients and feature maps specific to a given CNN to highlight regions of an input image that are most influential in the model's prediction. This reliance on the internal structure of CNNs, particularly the gradients flowing into the final convolutional layers, makes Grad-CAM a model-specific method tailored to convolutional architectures. Consequently, it is not directly applicable to non-convolutional models, such as those handling tabular data. However, Grad-CAM is also model-agnostic in the sense that it can be applied across various CNN architectures without requiring modifications to each model's specific design. This flexibility allows it to be used with a wide range of CNN-based models, making it a versatile tool for interpreting and visualizing CNN decisions [115].

The mathematical foundation of Grad-CAM is based on the gradients of the target class with respect to the feature maps. These gradients indicate the importance of each feature map for the target prediction. Grad-CAM is particularly useful for understanding and debugging CNNs in tasks such as image classification, image captioning, and visual question answering.

**Grad-CAM Equation:** The primary Grad-CAM equation is formulated as follows:

$$ L_{\text{Grad-CAM}}^{c} = \text{ReLU}\left(\sum_{k} \alpha_{k}^{c} A^{k}\right) \quad (7) $$
where $L_{Grad-CAM}^c$ represents the Grad-CAM localization map for class c, $A^k$ is the $k$-th feature map of the convolutional layer and $\alpha_k^c$ is the importance weight of the $k$-th feature map for class c.

For the given equation 7, the underlying properties taken into consideration are as follows:

1) **Localization:** Grad-CAM localizes the critical regions in the image that contribute to the prediction of a specific class. This is achieved by combining the feature maps $A^k$ with the importance weights $\alpha_k^c$ to produce a class-specific localization map. The ReLU activation function ensures that only the positive contributions are highlighted in the localization map. The ReLU activation function is applied to the weighted sum of the feature maps to ensure that only the positive contributions are included in the localization map. This step helps in highlighting the regions that positively influence the prediction of the target class.

2) **Class Discrimination:** The importance weights $\alpha_k^c$ are computed using the gradients of the target class with respect to the feature maps. This ensures that the localization map is specific to the target class and highlights the regions that are important for predicting that class.

3) **Global Average Pooling:** The importance weights $\alpha_k^c$ are obtained by performing global average pooling on the gradients of the target class with respect to the feature maps:

$$ \alpha_k^c = \frac{1}{Z} \sum_i \sum_j \frac{\partial y^c}{\partial A_{ij}^k} \quad (8) $$

where Z is the number of pixels in the feature map, $y^c$ is the score for class c, and $\frac{\partial y^c}{\partial A_{ij}^k}$ is the gradient of the score for class c with respect to the activation $A_{ij}^k$ at spatial location (i, j) in the feature map $A^k$. This aggregation step ensures that the importance of each feature map is considered across the entire spatial dimensions of the feature map.

## D. LOCAL INTERPRETABLE MODEL-AGNOSTIC EXPLANATIONS (LIME): MODEL-AGNOSTIC FRAMEWORK FOR LOCAL EXPLAINABILITY

Ribeiro et al. [78] introduced LIME to explain predictions of complex models like deep neural networks. LIME is a post-hoc, model-agnostic technique that approximates the decision boundary of a complex model as linear in the local vicinity of the instance being explained. It generates perturbed samples around the instance, obtains predictions from the complex model, assigns weights based on proximity to the instance, and builds an interpretable model on these weighted samples. The goal is to provide a locally accurate and understandable explanation for the instance in question, enhancing model transparency.

### 1) LIME EQUATION

The explanation is derived as:

$$ \xi(x) = \arg \min_{g \in G} L(f, g, \pi_x) + \Omega(g) \quad (9) $$

where, G different explanation families, fidelity functions L, and complexity measures $\Omega$, focusing on sparse linear models.

1) **Interpretable Data Representations:** Explanations require a human-comprehensible representation, independent of the model's specific features. For text classification, an interpretable representation might be a binary vector indicating word presence or absence. For image classification, it might indicate the presence or absence of a super-pixel. The variable $x \in \{R\}^d$ represents the original instance, while $x_0 \in \{0, 1\}^{d_0}$ is its interpretable form.

2) **Fidelity-Interpretability Trade-off:** The explanation model $g \in G$ should be simple to interpret. The complexity of $g$ is denoted by $\Omega(g)$. The model $f: R^d \to R$ represents the original classifier, with $f(x)$ indicating class probability. Proximity $\pi_x(z)$ defines the locality around $x$. Discrepancy between $g$ and $f$ within this locality is quantified by $L(f, g, \pi_x)$. The goal is to minimize $L(f, g, \pi_x)$ while keeping $\Omega(g)$ low for interoperability.

3) **Sampling for Local Exploration:** To minimize locality-aware loss $L(f, g, \pi_x)$ without assumptions about $f$, a sample is drawn weighted by $\pi_x$. Perturbed samples are generated by randomly selecting non-zero elements of $x_0$, converted back to the original representation, and labeled by $f(z)$. The dataset Z comprising perturbed samples and labels is used to optimize Eq. (9). LIME samples instances near and far from $x$, providing resilience to sampling noise due to $\pi_x$ weighting.

4) **Sparse Linear Explanations:** Consider G as linear models, $g(z_0) = w_g \cdot z_0$, and locally weighted square loss L as:

$$ L(f, g, \pi_x) = \sum_{z, z_0 \in Z} \pi_x(z)(f(z) - g(z_0))^2 \quad (10) $$

For text classification, the interpretable representation is a bag of words, with a maximum word constraint K. For image classification, “super-pixels” are used instead of words. The direct solution of Eq. (9) is impractical, so K-LASSO is used.

$$ \Omega(g) = \infty \mathbb{1}[||w_g||_0 > K] \quad (11) $$

## V. EMPIRICAL EVALUATION OF XAI TECHNIQUES ACROSS SELECTED STUDIES

In the field of breast cancer research, XAI techniques have proven instrumental in improving both model interpretability and predictive accuracy. This section provides a brief review of how these techniques have been applied across 44 selected studies and various contexts.
A comprehensive overview of current practices, highlighting the alignment between specific models and their corresponding explainability methods is presented in the Table 3. This comparison provides a clearer understanding of the field's trends and preferences and the role these techniques play in enhancing model transparency and interoperability.

In the subsequent subsections, the details of XAI models in the breast cancer domain are discussed.

## A. SHAP-BASED INTERPRETABILITY AND FEATURE ATTRIBUTION IN BREAST CANCER RESEARCH

In survival analysis and predictive modeling, SHAP has been instrumental in clarifying the impact of key features, such as age, tumor characteristics, receptor statuses, TNM (Tumor, Node, Metastasis) staging, and lymph node involvement. Moncada et al. [32] applied SHAP with the Cox Proportional Hazards (CPH) model and machine learning algorithms like Extreme Gradient Boosting (XGBoost) to improve the interpretability of survival predictions. Similarly, Vrdoljak et al. [43] and Massafra et al. [45] employed SHAP to interpret models predicting lymph node metastases, demonstrating its effectiveness in making complex models more comprehensible for clinical applications. Moreover, SHAP has been used to compare traditional rule-based models with advanced techniques, offering valuable insights into diagnostic accuracy and model performance.

In survival analysis and predictive modeling, SHAP has been instrumental in clarifying the impact of key features such as age, tumor characteristics, receptor statuses, TNM (Tumor, Node, Metastasis) staging, and lymph node involvement. Moncada et al. [32] employed SHAP with the Cox Proportional Hazards (CPH) model and machine learning algorithms like XGBoost to enhance the interpretability of survival predictions. Similarly, Vrdoljak et al. [43], [45] utilized SHAP to interpret models predicting lymph node metastases, showcasing its effectiveness in making complex models more comprehensible for clinical applications. Additionally, SHAP has been applied to compare traditional rule-based models with advanced techniques, providing insights into diagnostic accuracy and model performance.

Zhang et al. [37] utilized SHAP to demonstrate how features such as “worst parameter,” “worst area,” “worst radius,” “worst concave points,” “worst texture,” and “mean texture” influence the performance of Random Forest and XGBoost models. Their findings revealed significant improvements in accuracy and precision compared to simpler models. Similarly, Wang et al. [36] applied SHAP to interpret radiomics models based on chest CT scans, enabling more precise differentiation between luminal and non-luminal breast cancer types.

SHAP has been instrumental in identifying key features and biomarkers, serving as a valuable tool for pinpointing significant predictors and potential biomarkers. Studies such as [36], [40], [46], [48], and [49] employed SHAP to analyze multi-omics data, highlighting critical features such as

Deoxyribonucleic Acid (DNA), Messenger Ribonucleic Acid (mRNA), and Copy Number Variation (CNV). Similarly, Yu et al. [20] and Chakraborty et al. [25] utilized SHAP for gene expression data, uncovering valuable insights into gene contributions and pathways associated with cancer.

In clinical and preventive contexts, SHAP has offered valuable insights into the influence of lifestyle factors, such as diet and breastfeeding, on cancer risk. Silva et al. [41] applied SHAP to evaluate the impact of these factors on cancer classification, contributing to the development of personalized risk assessments and preventive strategies. While, Alelyani et al. [51] also employed SHAP to derive global and local importance scores for clinical and pathological features across six classifiers, revealing model-dependent SHAP-based rankings of key breast cancer risk factors.

For down-sampled histopathology images, Murugan et al. [54] used SHAP to decompose VGG-19 predictions into feature-level contributions, helping to characterise how the network weights different tissue patterns in malignant vs. benign classification.

Table 4 provides a concise overview of the diverse features deemed crucial in various studies that employ SHAP for model explainability in medical and cancer-related research. These features encompass demographic and clinical attributes, specific biomarkers, and lifestyle factors, reflecting a comprehensive approach to understanding and interpreting model predictions in healthcare.

## B. GRAD-CAM-BASED VISUAL EXPLAINABILITY IN BREAST CANCER RESEARCH

Studies focusing on breast cancer imaging using convolutional neural networks (CNNs) have found visualization techniques, such as class activation maps (CAM) and Grad-CAM, to be pivotal in enhancing model interpretability and diagnostic accuracy.

In studies by Prodan et al. [47] and Masud et al. [17], Grad-CAM was used to visualize the regions of mammography and ultrasound images that most influenced model predictions. Prodan et al. [47] applied CAM across CNN models such as ResNet, EfficientNet, and MaxVit, providing intuitive insights into the classification process by highlighting key image areas. Similarly, Masud et al. [17] and Suh et al. [18] employed Grad-CAM to visualize the focus of various CNN models, demonstrating its effectiveness in distinguishing between benign and malignant lesions.

Additionally, Dong et al. [24] introduced a novel Region of Evidence (ROE) framework to enhance interpretability and facilitate Grad-CAM explanations, achieving high accuracy and AUC scores. Similarly, Lou et al. [29] proposed the Multi-level Global-Guided Branch-Attention Network (MGBN) for mass classification in mammography, employing Grad-CAM to visualize localization maps. This approach significantly improved the accuracy of mass region identification compared to traditional methods. In contrast, Wang et al. [44] used Grad-CAM to explain their proposed Multitask Information Bottleneck Network (MIB-NET).
<table><caption>**TABLE 4.** Key features considered in studies using SHAP for explainability
Ki67= Kiel 67 (Protein), PgR= Progesterone Receptor, TNBC= Triple-Negative Breast Cancer, HER2= Human Epidermal Growth Factor Receptor 2, ERBB2= Erythroblastic Oncogene B2, Luminal A & Luminal B= Subtype classification of breast cancer.</caption><thead><tr><td>Ref.</td><td>Features</td></tr></thead><tbody><tr><td>[32]</td><td>Age, Tumor characteristics, Hormonal receptor statuses, Clinical and pathological TNM (Tumour, Node, Metastasis) staging, and the presence of removed and positive lymph nodes</td></tr><tr><td>[37]</td><td>Numerical features extracted from digitized images</td></tr><tr><td>[36]</td><td>Ki67, Age, CT scheme, Grading, PgR, Multiple</td></tr><tr><td>[40]</td><td>Luminal A, luminal B, ERBB2 (HER2(+)), TNBC, and Unclear</td></tr><tr><td>[45]</td><td>Patient age at the time of diagnosis, tumor size (in cm), pathohistological type, immunophenotype, pathohistological grade, estrogen (ER) and progesterone (PR) receptor quantities (0–100), HER-2 levels (0–3), Ki-67 index (0–100), and lymph node metastasis status (0/1).</td></tr><tr><td>[41]</td><td>Age at menarche, the first pregnancy, age at menopause, and others. In addition, a high-fat diet and determinants of body mass index (BMI), parity, breastfeeding</td></tr></tbody></table>

Ricciardi et al. [28] employed Grad-CAM to enhance model interpretability in digital breast tomosynthesis (DBT), evaluating the behavior of CNN models in classifying mass lesions. Their findings highlighted that central lesion regions were the most influential in model predictions. Similarly, Hussain et al. [38] utilized Grad-CAM alongside other interpretability techniques, such as LIME, t-SNE [116], and UMAP [117], achieving high classification performance. They provided both visual and mathematical explanations to quantify model interpretability and gain peer confidence.

More recently, Alom et al. [57] proposed the Deep Neural Breast Cancer Detection (DNBCD) model based on DenseNet121 for histopathological and ultrasound images, using Grad-CAM to highlight the most relevant regions guiding benign, malignant, and normal classifications on the BreakHis-400x and BUSI datasets. Similarly, Talukder et al. [55] modified DenseNet169 for multi-magnification histopathology and employed CAM and saliency maps to visualize the discriminative tissue regions supporting their decisions.

A complementary study quantitatively evaluated Grad-CAM, Grad-CAM++, and Eigen-CAM on mammograms using radiologist-annotated bounding boxes and the Pointing Game metric [53], assessing how reliably these saliency methods point to true lesion locations. In another work on down-sampled histopathology images, several CNN architectures (including ResNet, VGG-16, and VGG-19) were compared, and saliency maps were applied to the best-performing model to visualize the image regions most influencing its predictions [54].

## C. LOCAL EXPLAINABILITY USING LIME IN BREAST CANCER RESEARCH

Similar to SHAP and Grad-CAM, researchers have adapted LIME to explain breast cancer classification and detection models. Lee et al. [23] used LIME to classify lymph nodes in ultrasound images, employing the Simple Linear

Iterative Clustering (SLIC) superpixel segmentation method to segment images and identify critical regions influencing predictions. Deshmukh et al. [42] integrated LIME with an enhanced quantum k-means clustering algorithm to explain clustering outcomes for breast cancer data, achieving high clustering accuracy. Kaplun et al. [33] utilized LIME to interpret automated breast cancer cell image analysis, highlighting crucial regions in cell images that influenced classification results.

In other studies, LIME has been applied to genomic datasets and compared with explanation methods such as SHAP and PDP. For instance, Modhukur et al. [22] utilized LIME to evaluate the significance of methylation biomarkers in cancer-type classification. Similarly, Zhang et al. [37] used SHAP to identify influential features, such as “worst area” and “worst radius,” in breast cancer predictions, with findings that closely aligned with LIME results.

Alelyani et al. [51] applied LIME to clinical and pathological breast cancer data from Saudi patients to locally explain model predictions, showing that the Random Forest classifier combined reasonable accuracy with clinically meaningful LIME-based feature attributions. Murugan et al. [54] also used LIME on their best-performing VGG-19 histopathology classifier to generate local explanations for individual cases, which medical experts judged as the most accurate and clinically relevant among the evaluated explanations.

## D. COMPLEMENTARY XAI METHODS IN BREAST CANCER RESEARCH

Beyond the core XAI techniques previously discussed, other methods also offer valuable insights into making complex machine-learning models more interpretable. These methods include counterfactual explanations, Permutation Importance, and various others, each contributing uniquely to the landscape of explainable AI (XAI). For instance, Althoff et al. [81] and Lamy et al. [14] introduced a visual case-based reasoning (CBR) method that integrates visual
algorithms to illustrate case similarities, enhancing user interpretation of recommendations. In the context of CBR, a case refers to a problem situation. Generally, CBR consists of a (*p* + 1)-dimensional space case database represented as *X*, where *x*ᵢ represents each case. Each *x*ᵢ has components in different dimension spaces *A*₁, ..., *A*ₖ, ..., *A*ₚ and *Y* is the solution space associated with each case. The query case can be represented as *q* ∈ *A*₁ × ... × *A*ₖ × ... × *A*ₚ This hierarchical process involves four key steps: retrieval, reuse, revision, and retention. Such a structured approach facilitates decision-making, as demonstrated in breast cancer therapy, where CBR helps simplify complex decisions and validate results against established guidelines. However, challenges persist, including accuracy limitations and sensitivity to high-dimensional data. Gu et al. [16] addressed some of these challenges by combining CBR with ensemble learning, utilizing XGBoost for predictions while leveraging CBR for explanations. While this approach enhances XAI, it remains reliant on existing techniques and could benefit from incorporating more comprehensive attribute considerations.

Permutation Importance (PIMP), introduced by Altmann et al. [83], assesses feature significance by measuring performance degradation after shuffling feature values. Khater et al. [48] integrated PIMP with SHAP and Partial Dependence Plots (PDP) to identify key features in breast cancer detection, such as bare nuclei and clump thickness. PDP extends this analysis by illustrating how individual features influence predictions, offering a contrast to methods that solely highlight the most significant features [84]. The use of PDP revealed the correlation between changes in tissue architecture and breast cancer progression, with particular emphasis on the relationship between tissue thickness and cell stickiness, as shown in Khater et al. [48].

Similarly, Tosun et al. [15] developed HistoMapr-Breast, an XAI-enabled application designed to visualize and identify regions of interest in breast core biopsies. This tool supports pathologists' decision-making by incorporating XAI into pathology workflows, effectively addressing concerns related to transparency and safety.

Layerwise Relevance Propagation (LRP) [82] interprets deep neural networks by tracing information from the output neurons back to the input layers. Binder et al. [72] applied LRP to identify critical image regions and molecular biomarkers relevant to breast cancer prognosis. Cherada et al. [21] expanded on this approach by employing Graph LRP to explain predictions from Graph-CNNs, revealing significant genes associated with cancer.

Furthermore, Amoroso et al. [30] proposed an XAI method based on Adaptive Dimension Reduction (ADR), specifically designed for breast cancer therapies. Their framework, which integrates clustering and dimensionality reduction, effectively identified critical clinical features and contributed to the development of oncological treatment strategies.

Counterfactual explanations have recently emerged as a significant post-hoc method, offering users valuable insights

into the internal workings of AI models [118], [119], [120], [121], [122]. Hanis et al. [39] investigated the impact of various machine learning models on counterfactual generation, focusing on predicting breast cancer likelihood and accuracy based on medical history and risk factors. Their study, which employed multiple ML classifiers, highlighted that KNN achieved notable accuracy and F-score. The authors subsequently adopted a model-agnostic approach to further enhance explainability.

Finally, Feature Importance Measures are essential for understanding ML models by identifying significant features. Studies by Bhatt et al. [123] and Saarela et al. [31] compared classification explanation models, including feature importance metrics, highlighting their critical role in interpreting model behavior.

## E. SPECIALIZED XAI APPROACHES FOR BREAST CANCER ANALYSIS

Beyond conventional explainable AI techniques, several specialized approaches have been proposed to enhance interpretability in breast cancer research, focusing on biomarker analysis, automated tissue scoring, and hybrid image-based models.

Idrees et al. [35] introduced XAI algorithms to analyze potential biomarkers for breast cancer. Various classification models were applied to anthropometric data from three studies [124], [125], [126], augmented with XAI models. The results of these analyses revealed that serum levels of HOMA, leptin, adiponectin, and resistin could serve as novel biomarkers for breast cancer. The findings demonstrated sensitivity ranging from 83% to 87%, specificity ranging from 81% to 88%, and AUC values between 0.82 and 0.89 with a 95% confidence interval. These promising results suggest that integrating the characteristics of HOMA, leptin, adiponectin, and resistin may facilitate the development of a cost-effective breast cancer biomarker.

Kabakcci et al. [26] proposed an automated method for scoring CerbB2/HER2 receptors in breast cancer tissue images, designed to replace the manual, error-prone scoring traditionally performed by pathologists. This method employs histogram-based analysis to separate tissue images into color channels, segment cell nuclei and boundaries, and extract features related to membrane staining intensity and completeness. These extracted features are used to compute cell-based HER2 scores, which are then aggregated to generate a final tissue score in accordance with ASCO/CAP guidelines. The approach was validated using two publicly available datasets, demonstrating high efficacy in scoring HER2 tissue for both balanced and unbalanced datasets. A comparison of classifiers revealed that the Ensemble Boosted Tree achieved the highest accuracy for cell-based and tissue-based scoring on the ITU-MED datasets, while the Cosine kNN classifier performed best on the contest dataset, securing 6th place out of 18 competitors.
<figure><img src="image_9.png" /><figcaption>FIGURE 8. Comparative visualization of XAI techniques for breast cancer detection [127]. (a) The input image (b) SHAP (c) Grad-CAM (d) LIME.</figcaption></figure>

AlHussen et al. [56] introduced the XAI-RACapsNet framework, which integrates explainable ROI segmentation through the XAI O-Net architecture combining transformer and ResNet modules. The model generates relevance-based heat maps from segmented mammograms, providing interpretable localization of malignant regions alongside high diagnostic accuracy.

Similarly, Munshi et al. [52] developed a hybrid XAI approach combining U-Net for image-based feature extraction with an ensemble of CNN, Random Forest, and SVM classifiers. Their method employed ROI segmentation as an explainability mechanism to highlight diagnostically significant breast regions and achieved notable accuracy on the Wisconsin dataset.

Section VI provides a detailed discussion on the integration of XAI techniques with various machine learning and deep learning models.

## VI. MAPPING EXPLAINABLE AI TO MACHINE LEARNING AND DEEP LEARNING APPROACHES

This section explores the integration of XAI techniques with various machine learning and deep learning models, highlighting both their strengths and weaknesses. Additionally, Table 5 provides valuable insights into the frequency with which different machine learning and deep learning models are paired with various explainability techniques. The table also identifies potential biases in certain deep learning models, revealing a tendency to favor specific explainability methods over others. This analysis aids in understanding the limitations and advantages of different XAI techniques across various contexts.

To begin with, SHAP is model-agnostic, meaning it is not tied to any specific machine learning (ML) or deep learning (DL) model. Refer to Figure 8 for a visualization created by SHAP. This characteristic is advantageous because it allows SHAP values to be applied to any machine learning model, regardless of its architecture or underlying algorithms [128].

This flexibility is particularly valuable for tasks involving model comparison, as it provides a consistent method for interpreting predictions and understanding model behavior. Furthermore, because SHAP values are independent of the specific model being used, they can be easily adapted when the model is updated or replaced. In a machine learning pipeline, where models are often refined or replaced over time, SHAP's flexibility ensures that the explanations

remain consistent and informative, requiring minimal effort to integrate with updated models [129].

The Shapley values also offer three valuable properties not found in other XAI methods [77], [131].

1) SHAP values provide local accuracy, meaning they accurately capture the difference between the expected output of the model and the output for a specific instance of input data. This means they provide insights into how individual features contribute to each prediction.

2) If a feature is missing (i.e. not present in the input data), its attribution value in the SHAP analysis is set to zero, ensuring that the absence of a feature is appropriately accounted for in the explanation.

3) Changes in the model should not result in contradictory changes to the SHAP values assigned to individual features. If increasing the importance of a feature model's prediction also increases its corresponding SHAP value, then consistency is maintained.

These properties make the SHAP model an ideal choice for different rstudies'studies' objectives. It offers a robust and reliable method for explaining model predictions and comparing various models, regardless of their type or complexity. For example, SHAP performed well with the XGBoost model, as witnessed in [32], [41], [43], [45], and [48].

Despite its benefits, the SHAP method may have limitations.

1) The computational burden of computing SHAP values can be significant, potentially slowing down the analysis process.

2) SHAP values are efficient for tree-based models, for example, XGBoost, but can be less effective for complex architectures like deep learning models. The same can be observed in Table 5.

In scenarios with limited computational resources or when working with complex architectures such as deep learning models, the application of SHAP is generally not recommended.

Grad-CAM is one of the most widely used XAI methods for medical imaging [132], as reflected in Table 5. It generates heatmaps that highlight the regions of an image contributing significantly to the model's decisions, offering insights into the features or areas the model focuses on for classification (refer to Figure 8). This interpretability helps radiologists and clinicians understand and validate the model's predictions.

Grad-CAM's ability to localize regions of interest in medical images can assist in pinpointing abnormalities within breast tissue, potentially improving diagnostic accuracy. Studies such as [17], [18], [24], and [29] have demonstrated that Grad-CAM is a valuable tool for corroborating the findings of deep learning models by providing visual evidence of the features driving their predictions.

However, like any other method, Grad-CAM also has certain limitations:
<table><caption>**TABLE 5.** Mapping machine learning and deep learning models to the applied explainability techniques in the reviewed/selected 44 articles
RBIA= Rainbow boxes- Inspired Algorithm, CBR= Case-Based Reasoning, ROI= Region of Interest, WSI= Whole Slide Images, XGBoost= Extreme Gradient Boosting, RF= Random Forest, MTL= MultiTask Learning, GLRP= Graph Layer-wise Relevance Propagation, SLIC= Simple Linear Iterative Clustering, LRP= Layerwise Relevance Propagation, ADR= Adaptive Dimension Reduction, PI= Permutation importance, PDP= Partial dependence plots, DBT= Digital Breast Tomosynthesis, MGBN= Multi-level Global-guided Branch-attention Network, MIB-Net= Multitask information bottleneck network, LE= low-energy, DES= Dual-Energy Subtraction, CAM= Class Activation Maps, RSF= Random Survival Forest, QDA= Quadratic Discriminant Analysis, t-SNE= T-distributed Stochastic Neighbor Embedding and UMAP= Uniform Manifold Approximation and Projection, LSTM= long short-term memory.</caption><thead><tr><td>Ref.</td><td>ML/DL Model</td><td>XAI Model</td><td>Modality</td><td>Explanation Type</td></tr></thead><tbody><tr><td>[130]</td><td>KNN, WKNN, RBIA</td><td>Visual CBR</td><td>Features</td><td>Visual Explanations</td></tr><tr><td>[15]</td><td>HistoMapr</td><td>Customized</td><td>WSI</td><td>ROI</td></tr><tr><td>[16]</td><td>XGBoost</td><td>CBR and SHAP</td><td>Attributes</td><td>Shapley Values</td></tr><tr><td>[17]</td><td>AlexNet, DarkNet19, GoogleNet, MobileNet-v2, ResNet18, ResNet50, VGG16, Xception</td><td>GRAD-CAM</td><td>Ultrasound Images</td><td>Heatmaps</td></tr><tr><td>[18]</td><td>Dense-Net-169, Efficient-Net-B5</td><td>GRAD-CAM</td><td>Mammographs</td><td>Heatmaps</td></tr><tr><td>[19]</td><td>ResNet-50, MTL</td><td>GRAD-CAM</td><td>Ultrasound Images</td><td>Feature map</td></tr><tr><td>[20]</td><td>Autoencoders</td><td>SHAP</td><td>Genes</td><td>SHAP values</td></tr><tr><td>[21]</td><td>Graph CNN</td><td>GLRP</td><td>Graphs</td><td>Heatmaps</td></tr><tr><td>[22]</td><td>SVM, Naive Bayes, XGBoost, RF</td><td>LIME</td><td>Genes</td><td>LIME Explanations</td></tr><tr><td>[23]</td><td>Mask R-CNN, DenseNet-121, Logistic Regression, SVM, XGBoost</td><td>SLIC and LIME</td><td>Images</td><td>LIME Explanations</td></tr><tr><td>[24]</td><td>Dense-Net-121</td><td>GRAD-CAM</td><td>Images</td><td>Heatmaps</td></tr><tr><td>[25]</td><td>XGBoost</td><td>SHAP</td><td>Feature values</td><td>Shapley Values</td></tr><tr><td>[26]</td><td>KNN, Decision Trees, LSTM</td><td>Customized</td><td>WSI histological images</td><td>Feature-based</td></tr><tr><td>[27]</td><td>Bag-of-words</td><td>LRP</td><td>WSI histological images</td><td>Heatmaps</td></tr><tr><td>[28]</td><td>AlexNet, VGG19</td><td>GRAD-CAM</td><td>DBT</td><td>Heatmaps</td></tr><tr><td>[29]</td><td>MGBN</td><td>GRAD-CAM</td><td>Mammogram images</td><td>Heatmaps</td></tr><tr><td>[30]</td><td>Cluster Analysis</td><td>ADR</td><td>WSI histological images</td><td>Statistical analysis</td></tr><tr><td>[31]</td><td>Logistic Regression, Random Forest</td><td>Feature Importance</td><td>Features</td><td>Statistical analysis</td></tr><tr><td>[32]</td><td>RSF, Survival SVM, XGBoost</td><td>SHAP</td><td>Features</td><td>SHAP score</td></tr><tr><td>[33]</td><td>ANN</td><td>LIME</td><td>Instrinsic feature</td><td>Heatmaps</td></tr><tr><td>[34]</td><td>Decision Trees, RF &amp; Neural Networks</td><td>Counterfactual Algorithms</td><td>Numeric</td><td></td></tr><tr><td>[35]</td><td>SVM, RUSBoosted Ensemble Trees, KNN, QDA, &amp; Logistic Regression</td><td>GRAD-CAM</td><td>Features</td><td>Heatmaps</td></tr><tr><td>[36]</td><td>SVM, RF, XGBoost, Adaboost, LightGBM, GaussianNB, MLP</td><td>SHAP</td><td>Radiomics</td><td>Shapley Values</td></tr><tr><td>[37]</td><td>Rule-based, RF</td><td>SHAP, LIME &amp; PDP</td><td>Radiomics</td><td>SHAP score, HeatMaps</td></tr><tr><td>[38]</td><td>VGG, ResNet, ResNeXt, DenseNet, SqueezeNet, &amp; MobileNet-V2</td><td>GRAD-CAM, LIME, t-SNE &amp; UMAP</td><td>DBT Images</td><td>Heatmaps</td></tr><tr><td>[40]</td><td>SVM, RF, Extra Trees, XGBoost</td><td>SHAP</td><td>Multi-omics</td><td>Shapley Values</td></tr></tbody></table>

1) It only works with convolutional neural networks (CNNs) as it relies on the gradient information computed during the backpropagation step.

2) It provides coarse, class-discriminative visualizations, making it difficult to understand specific pixel-level contributions. In other words, it highlights regions of
<table><caption>**TABLE 5. (Continued.)** Mapping machine learning and deep learning models to the applied explainability techniques in the reviewed/selected 44 articles
RBIA= Rainbow boxes- Inspired Algorithm, CBR= Case-Based Reasoning, ROI= Region of Interest, WSI= Whole Slide Images, XGBoost= Extreme Gradient Boosting, RF= Random Forest, MTL= MultiTask Learning, GLRP= Graph Layer-wise Relevance Propagation, SLIC= Simple Linear Iterative Clustering, LRP= Layerwise Relevance Propagation, ADR= Adaptive Dimension Reduction, PI= Permutation importance, PDP= Partial dependence plots, DBT= Digital Breast Tomosynthesis, MGBN= Multi-level Global-guided Branch-attention Network, MIB-Net= Multitask information bottleneck network, LE= low-energy, DES= Dual-Energy Subtraction, CAM= Class Activation Maps, RSF= Random Survival Forest, QDA= Quadratic Discriminant Analysis, t-SNE= T-distributed Stochastic Neighbor Embedding and UMAP= Uniform Manifold Approximation and Projection, LSTM= long short-term memory.</caption><tr><td>[41]</td><td>XGBoost</td><td>SHAP</td><td>Features</td><td>SHAP values &amp; SHAP plots</td></tr><tr><td>[42]</td><td>SVM, RF, KNN, K-means, C-means</td><td>LIME</td><td>Attributes</td><td>LIME explanations</td></tr><tr><td>[43]</td><td>K-Fold CV, Boruta, RF, SVM, XGBoost, NB</td><td>SHAP</td><td>Features</td><td>Shapley Values</td></tr><tr><td>[44]</td><td>MIB-Net</td><td>GRAD-CAM</td><td>LE &amp; DES mammograms &amp; Ultrasound images</td><td>Highlighted regions</td></tr><tr><td>[45]</td><td>Logistic Regression, RF, XGBoost</td><td>SHAP</td><td>Histopathological features</td><td>Shapley Values</td></tr><tr><td>[46]</td><td>XGBoost</td><td>SHAP</td><td>Genes</td><td>SHAP Values</td></tr><tr><td>[47]</td><td>ResNet, EfficientNet, MaxViT</td><td>CAM &amp; centered bounding boxes</td><td>Mammogram images</td><td>Highlighted regions</td></tr><tr><td>[48]</td><td>KNN, SVM, XGBoost, RF, ANN</td><td>PI, PDP, and SHAP</td><td>Mammogram images</td><td>PI Plots, Interactive PDP plots &amp; SHAP Summary plots</td></tr><tr><td>[49]</td><td>LightGBM, CatBoost, XGBoost, GBT, AdaBoost</td><td>SHAP</td><td>Genomic Biomarkers</td><td>Shapley Values</td></tr><tr><td>[50]</td><td>DenseNet, VGG, EfficientNet</td><td>Attention Guided Grad-CAM</td><td>Infrared breast images</td><td>Heatmaps</td></tr><tr><td>[39]</td><td>KNN and other ML classifiers</td><td>Counterfactual explanations</td><td>Clinical/risk-factor features</td><td>Counterfactual examples</td></tr><tr><td>[53]</td><td>Pre-trained CNN</td><td>GRAD-CAM, Grad-CAM++, Eigen-CAM</td><td>Mammograms</td><td>Saliency maps (Pointing Game)</td></tr><tr><td>[51]</td><td>RF, SVM and other ML classifiers</td><td>LIME, SHAP</td><td>Clinical and pathological features</td><td>LIME explanations, SHAP values</td></tr><tr><td>[54]</td><td>CNN, ResNet, VGG-16, VGG-19</td><td>LIME, SHAP, Saliency maps</td><td>Histopathology images</td><td>LIME/SHAP explanations, saliency heatmaps</td></tr><tr><td>[52]</td><td>U-Net, CNN, RF, SVM</td><td>ROI-based XAI</td><td>Mammograms &amp; Features</td><td>ROI maps</td></tr><tr><td>[57]</td><td>DenseNet-121 (DNBCD)</td><td>GRAD-CAM</td><td>Histopathology &amp; Ultrasound images</td><td>Heatmaps</td></tr><tr><td>[55]</td><td>DenseNet-169</td><td>CAM, Saliency maps</td><td>Histopathology images</td><td>Heatmaps</td></tr><tr><td>[56]</td><td>XAI O-Net, RACapsNet (Capsule Network)</td><td>ROI-based XAI, Relevance heatmaps</td><td>Mammograms</td><td>ROI maps &amp; relevance heatmaps</td></tr></table>

high importance for classification, but it may prioritize features that are not clinically relevant or meaningful for diagnosis.

The CAM technique focuses solely on broad areas within an image, often overlooking subtle nuances. In contrast, the LIME technique (discussed below) creates perturbations to emphasize key features, while the SHAP model precisely measures the contribution of specific regions,

showing significant potential for inclusion in future research endeavors.

To address the limitations of conventional CAM, Raghavan et al. [50] proposed Attention-Guided Grad-CAM (AGG-CAM). This method generates enhanced visual representations by considering not only the gradient importance of individual pixels but also the significance of various input components identified through an attention
mechanism. As a result, AGG-CAM produces heatmaps that are more interpretable, resilient, and targeted compared to traditional Grad-CAM approaches. Applying this attention-guided Grad-CAM method in clinical settings holds the potential to revolutionize breast cancer detection and significantly improve patient outcomes.

Similar to SHAP, LIME is also model-agnostic, meaning it can be applied to any machine-learning model. It provides explanations at the individual prediction level, highlighting the features that contribute most to a specific prediction. This local interpretability is particularly valuable in medical diagnosis, as it enables clinicians to understand why a particular prediction was made and assess its reliability. However, while LIME explanations are locally faithful, they may not always align with global model behavior, as discussed in [42]. When applying LIME, researchers need to account for the following limitations:

LIME's explanations can be inconsistent across repeated runs due to its reliance on a random sampling of perturbed data [133]. Refer to Section IV-D to see a discussion on perturbations.

1) The choice of how data is perturbed can significantly impact the explanations, leading to potential misinterpretation.

2) LIME lacks robustness as it provides a local approximation of the model, which may not accurately reflect the global behavior of the model.

The integration of Explainable AI (XAI) techniques into clinical workflows presents a promising opportunity to enhance the transparency and reliability of ML and DL models in breast cancer diagnosis. While specific explainability techniques provide valuable insights, they are not universally applicable to all scenarios. The choice of an explainability method depends on the specific context and application. Emerging methods may offer more tailored or complementary insights, making it essential to evaluate the unique requirements and constraints of each scenario when selecting an approach [14], [134].

The following insights, drawn from the reviewed studies, further enhance the understanding presented in Table 5.

1) **Pre-trained deep learning models favor GRAD-CAM:** Papers utilizing pre-trained CNN models commonly apply GRAD-CAM as the XAI tool ([17], [28], [47]). This reveals a pattern of reliance on visualization techniques for deep learning models that handle image data.

2) **Graph-based ML models use graph-specific XAI:** Cherada et al. [21] demonstrates a unique application of Graph Layer-wise Relevance Propagation (GLRP) for Graph CNN models. This highlights the need for specialized XAI techniques when working with graph-structured data.

3) **Sparse usage of ensemble models:** Although ensemble models (e.g., Random Forest, XGBoost) are frequently mentioned ([22], [36], [37], [49]), they are paired

with a limited set of XAI tools like SHAP and LIME. Other potentially valuable methods like Integrated Gradients [135] are notably absent, showing an under-explored space for explainability in ensemble learning.

4) **Use of unconventional models with SHAP:** Deshmukh et al. [42] apply SHAP to non-traditional models like Quantum Clustering, showcasing SHAP's adaptability beyond mainstream ML models.

5) **Specialized XAI for dimensionality reduction models:** Amoroso et al. apply [30], Adaptive Dimension Reduction (ADR) to cluster analysis, indicating that some studies focus on niche interpretability techniques tailored for dimensionality reduction, a less common focus compared to classification models.

6) **Limited ML models without specific XAI techniques:** Tosun et al. [15] and Moreira et al. [34] did not explicitly define specific machine learning models, focusing instead on techniques such as HistoMapr for the region of interest (ROI)-based diagnosis and Counterfactual Algorithms. This highlights that not all studies prioritize the use of a defined ML model when integrating XAI into clinical workflows.

7) **Combination of traditional ML and DL in the same study:** Lee et al. [23] employed both classical machine learning models (e.g., Logistic Regression, SVM, XGBoost) and deep learning architectures (e.g., Mask R-CNN, DenseNet), integrating them with XAI techniques such as LIME and SLIC. This approach highlights a balanced strategy for achieving explainability across diverse types of models.

8) **XAI for multitask learning models is underexplored:** The pairing of multitask models ([19], [44]) with advanced XAI techniques, such as GRAD-CAM, Information Bottleneck, and Attention Mechanisms, remains relatively uncommon, indicating a growing and underexplored area in model interpretability.

## A. EXPLAINABILITY FOR TRANSFORMER- AND VISION-LANGUAGE CLINICAL AI

As clinical AI increasingly adopts Vision Transformers (ViTs) and vision-language models (VLMs), evaluating explainability requires more than visually plausible outputs. Explanations must be decision-faithful, region-grounded, robust, and fully auditable, ensuring that model rationales can be rigorously verified and trusted. For ViTs, we advocate transformer-aware pipelines that pair LRP-style relevance propagation (to trace token→patch evidence) with attention rollout/flow (to characterize cross-layer information flow), and we insist that visualizations remain *testable hypotheses*. Concretely: validate saliency with deletion/insertion curves and sanity checks; run counterfactual/perturbation audits (blur, mask, out-paint, or clinically plausible replacements) to demonstrate necessity and sufficiency; and add concept-level probes to estimate causal effects of named findings. These steps directly address evidence that raw attention can mislead
without faithfulness tests [136] and leverage established transformer interpretability methods [137], [138]. For CLIP (Contrastive Language-Image Pretraining) - style clinical models, make text/image alignment auditable by design: expose region concept grounding during zero/few-shot inference, enforce prompt governance (template libraries, ensembles, sensitivity/ablation tests), calibrate uncertainty *before* thresholding, and run stratified subgroup/shift audits (site, device, demographics) to surface shortcutting and bias prior to deployment building on medical VLM work such as MedCLIP and CheXzero [139], [140].

Additionally in order to align with governance frameworks, we can employ documenting method cards, model/data cards, immutable decision logs, and a post-market monitoring plan consistent with the EU AI Act's transparency and monitoring duties [141], the FDA/Health Canada/MHRA GMLP and Transparency principles [142], [143], ISO/IEC 23894 risk management [144], and IMDRF SaMD clinical evaluation [145]; report prospective studies under SPIRIT-AI/CONSORT-AI [146], [147]. The result is not prettier heatmaps, but a verifiable chain from evidence to decision that regulators, clinicians, and patients can inspect and trust.

## B. OBSTACLES TO CLINICAL ADOPTION AND STANDARDISED EVALUATION OF XAI IN BREAST CANCER DIAGNOSIS

Despite the substantial promise of explainable artificial intelligence (XAI) in breast cancer diagnostics, significant challenges persist on the path from research prototypes to routine clinical implementation. A key challenge is the lack of standardised evaluation frameworks: while many studies report model accuracy, far fewer assess how well explanations support human decision-making, inter-operator consistency, or workflow integration. Reviews of XAI evaluation methods in healthcare highlight that many papers rely on anecdotal evidence or bespoke user studies, and only a minority adopt shared frameworks, thereby limiting cross study comparability and trust in explanation outputs [148], [149], [150]. In the specific domain of breast-cancer diagnosis, this evaluation gap undermines study comparability, reduces clinician confidence in explanation tools, and limits replicability of results.

Another major barrier is the limited real-world validation and generalisability of current XAI systems. Many breast-cancer detection studies rely on retrospective or single centre datasets with homogeneous populations or imaging devices, raising questions about external validity across breast densities, ethnic groups, or device vendors [151], [152]. Without thorough multi centre validation and diverse cohorts, even technically advanced XAI tools may fail when transferred to real-world clinical settings.

Third, there is an inherent trade-off between explainability, accuracy and bias. Complex deep-learning models (such as convolutional neural networks) may yield high diagnostic performance but require post-hoc explanation tools whose fidelity is not always guaranteed; conversely, simpler models

may be interpretable but less robust. Many tree-based models are paired with SHAP/LIME, while CNN-based image models often rely on saliency maps reflecting a bias in model-explanation matching that our survey also revealed. Furthermore, if training data lacks diversity, explanation methods may systematically emphasise features pertinent in one domain but not in others, thereby embedding fairness risks [153].

Even when explanations are generated, the interpretability and usability of those explanations for end-users (clinicians, pathologists) remain under-explored. For example, a heatmap produced by Grad-CAM may appear intuitive, yet unless validated to correspond to a clinically-relevant pathological landmark, the clinician may ignore or mistrust it. Human-centred evaluation studies show wide variation in clinician comprehension and trust of different explanation formats, emphasising the need for user centric design and training [149].

Finally, workflow integration, ethical and regulatory issues present additional hurdles. Deploying an XAI enabled system in radiology or pathology services requires alignment with existing PACS/EHR systems, audit trails, clinician accountability and clear responsibility when AI recommendations (and explanations) are delivered. Reviews of AI in screening note that regulatory, ethical and governance frameworks are still emergent and must evolve to support XAI-enabled clinical care [154], [155]. The deployment of such AI systems raises questions about liability, informed consent, transparency, and data-fairness especially in high-stakes domains such as breast-cancer diagnosis.

In light of these gaps, recent literature suggests several strategic directions:

* Develop and adopt clinically-oriented, standardised evaluation metrics and benchmark frameworks for XAI in breast-cancer diagnostics. The literature strongly emphasises that without such metrics—covering fidelity, stability, usability and domain relevance XAI explanations cannot be reliably compared or trusted across studies [148].

* Expand multi centre, multi-modality and population-diverse validation studies to establish generalisability and fairness. Many XAI systems to date rely on single centre or homogeneous datasets; broad validation is required to ensure robustness across devices, modalities and patient populations [153].

* Design human-in-the-loop, workflow-embedded evaluations, where clinicians interact with the AI + XAI system and their decision-making, trust and outcomes are formally measured. Human AI collaboration is now identified as a key element of trustworthy implementation [156].

* Tailor explanation methods to match the user group (radiologist, pathologist, oncologist) and the model type (tree-based, CNN, multimodal). Explanation formats that are technically accurate but misaligned to the
<figure><img src="image_10.png" /><figcaption>FIGURE 9. Comprehensive evaluation framework for XAI.</figcaption></figure>

clinician's context or model architecture risk being ignored or mis-interpreted [154].

* Address bias, fairness, ethics and regulatory readiness early in the design and deployment of XAI systems. Without careful attention to fairness of explanations, liability, informed consent and auditability, XAI-enabled tools may fail regulatory or ethical acceptability even if technically accurate [157].

### C. BRIDGING BIAS DISCOVERY TO CLINICAL IMPACT AND MITIGATION

Beyond identifying methodological and data-centric biases, it is crucial to understand how these biases manifest in clinical workflows. For instance, spurious correlations in imaging datasets such as scanner artefacts, skin markers or acquisition metadata inadvertently correlated with malignancy labels can lead XAI models to focus on non-pathological cues. Clinically, such biases may distort malignancy estimation and influence biopsy decisions, thereby impacting patient safety and diagnostic confidence. Recent work emphasises that bias is a systemic concern at every stage of AI for medical imaging, from data sampling to deployment, making proactive detection and correction essential [158]. Moreover, frameworks like the “Reveal to Revise” life-cycle explicitly combine explanation-based bias identification with model revision and re-evaluation, demonstrating how counterfactual testing, feature sensitivity analysis or domain-adversarial methods can attenuate spurious dependencies [159]. Embedding such mitigation mechanisms into model development and explanation evaluation therefore closes the translational loop from bias discovery to clinical practice, improving diagnostic robustness and enhancing clinician trust in XAI systems. Future work should extend these strategies through clinician-in-the-loop bias audits and post-deployment monitoring to ensure sustained fairness and reliability across diverse patient populations.

To support the translation of XAI into breast-cancer diagnostics, we propose an evaluation framework (shown in Figure 9 composed of four inter-linked domains:

* **Technical & Fidelity:** Assess how well explanations mirror the model's internal reasoning (e.g., fidelity, stability, runtime) and whether explanation methods reveal model errors or biases.

* **Fairness & Bias-Detection:** Quantify and mitigate bias sources (e.g., dataset imbalance, device shift, spurious correlations), evaluate cross-population robustness, and measure fairness of explanations across sub-groups.

* **Usability & Clinician Trust:** Capture human centred metrics of how easily clinicians understand and act on explanations (e.g., usability scales, think-aloud protocols, decision-error reduction, trust/confidence metrics).

* **Workflow & Clinical Integration:** Evaluate how the XAI system embeds into real-world clinical workflows: interoperability with PACS/EHR, audit-trail readiness, clinician AI collaboration, and regulatory/ethical readiness.

If these pathways are followed, the transition of XAI from promising research prototypes into trustworthy clinical tools for breast cancer diagnosis becomes far more likely.

### VII. CONCLUSION

XAI plays a crucial role in evaluating model performance, which is essential for optimizing accuracy and reliability in practical applications. Beyond performance, it is imperative to assess the impact of XAI on user trust and comprehension. As AI technologies become increasingly integrated into healthcare, ensuring their transparency and interpretability
is vital for fostering trust among medical practitioners and patients. Integrating various XAI approaches, however, presents significant challenges, particularly in comparative analysis and generalization across diverse scenarios. This complexity highlights the need for a robust framework to evaluate these models comprehensively. Establishing standardized evaluation metrics is critical, as they provide consistent and reliable measures of effectiveness in clinical settings. These metrics should not only assess the accuracy and efficiency of XAI applications but also evaluate the quality of the explanations in terms of relevance, completeness, and consistency. Moreover, the integration of XAI into clinical workflows can advance personalized medicine. By offering tailored explanations, XAI can help clinicians understand the reasoning behind specific diagnostic or treatment decisions, which is particularly important in complex cases like breast cancer, where patient-specific factors significantly influence outcomes. In addition to technical advancements, addressing the ethical implications of using AI in breast cancer diagnosis is imperative. The deployment of AI models in clinical settings must be guided by rigorous ethical standards to ensure patient privacy, data security, and equitable access to healthcare benefits. Transparency in AI decision-making, achieved through XAI techniques, not only enhances trust among healthcare professionals and patients but also mitigates biases that may stem from non-representative training data.

## AUTHOR CONTRIBUTIONS STATEMENT

**Conceptualization:** Samita Bai and Rizwan Ahmed Khan.
**Methodology:** Samita Bai and Sidra Nasir. **Software:** Not applicable. **Validation:** Sidra Nasir. **Formal Analysis:** Sidra Nasir, Rizwan Ahmed Khan, and Hubert Konik.
**Investigation:** Samita Bai, Sidra Nasir, and Rizwan Ahmed Khan. **Resources:** Samita Bai. **Data Curation:** Sidra Nasir.
**Writing—Original Draft:** Samita Bai, Sidra Nasir, and Rizwan Ahmed Khan. **Writing—Review and Editing:** Alexandre Meyer. **Supervision:** Rizwan Ahmed Khan, Alexandre Meyer, and Hubert Konik. **Project Administration:** Rizwan Ahmed Khan, Alexandre Meyer, and Hubert Konik. **Funding Acquisition:** Not applicable.

## CONFLICT OF INTEREST STATEMENT

The authors certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participaspeakers' peakers' bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.

## REFERENCES

[1] World Health Organization. *Breast Cancer: Prevention and Control*. Accessed: Mar. 16, 2024. [Online]. Available: https://www.who.int/newsroom/fact-sheets/detail/breast-cancer

[2] H. Sung, J. Ferlay, R. L. Siegel, M. Laversanne, I. Soerjomataram, A. Jemal, and F. Bray, "Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries," *CA: Cancer J. Clinicians*, vol. 71, no. 3, pp. 209–249, May 2021.

[3] G. Dileep and S. G. Gianchandani Gyani, "Artificial intelligence in breast cancer screening and diagnosis," *Cureus*, vol. 14, no. 10, pp. 1–6, 2022.

[4] A. Esteva, A. Robicquet, B. Ramsundar, V. Kuleshov, M. A. DePristo, K. Chou, C. Cui, G. S. Corrado, S. Thrun, and J. Dean, "A guide to deep learning in healthcare," *Nature Med.*, vol. 25, no. 1, pp. 24–29, 2018.

[5] M. Huti, T. Lee, E. Sawyer, and A. P. King, "An investigation into race bias in random forest models based on breast DCE-MRI derived radiomics features," in *Proc. Workshop Clin. Image-Based Procedures. Cham, Switzerland: Springer, 2023*, pp. 225–234.

[6] E. J. Topol, "High-performance medicine: The convergence of human and artificial intelligence," *Nature Med.*, vol. 25, no. 1, pp. 44–56, Jan. 2019.

[7] Y. Abas Mohamed, B. Ee Khoo, M. Shahrimie Mohd Asaari, M. Ezane Aziz, and F. Rahiman Ghazali, "Decoding the black box: Explainable AI (XAI) for cancer diagnosis, prognosis, and treatment planning—A state-of—The art systematic review," *Int. J. Med. Informat.*, vol. 193, Jan. 2025, Art. no. 105689.

[8] M. Ghassemi, L. Oakden-Rayner, and A. L. Beam, "The false hope of current approaches to explainable artificial intelligence in health care," *Lancet Digit. Health*, vol. 3, no. 11, pp. e745–e750, Nov. 2021.

[9] B. S. Caffo, F. A. D'Asaro, A. Garcez, and E. Raffinetti, "Explainable artificial intelligence models and methods in finance and healthcare," *Frontiers Artif. Intell.*, vol. 5, Aug. 2022, Art. no. 970246.

[10] G. Stiglic, P. Kocbek, N. Fijacko, M. Zitnik, K. Verbert, and L. Cilar, "Interpretability of machine learning-based prediction models in healthcare," *Wiley Interdiscipl. Rev., Data Mining Knowl. Discovery*, vol. 10, no. 5, p. e1379, 2020.

[11] D. Moher, A. Liberati, J. Tetzlaff, and D. G. Altman, "Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement," *PLoS Med.*, vol. 6, no. 7, Jul. 2009, Art. no. e1000097.

[12] A. Liberati, D. G. Altman, J. Tetzlaff, C. Mulrow, P. C. Götzsche, J. P. A. Ioannidis, M. Clarke, P. J. Devereaux, J. Kleijnen, and D. Moher, "The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration," *PLoS Med.*, vol. 6, no. 7, Jul. 2009, Art. no. e1000100.

[13] B. Kitchenham, R. Pretorius, D. Budgen, O. P. Brereton, M. Turner, M. Niazi, and S. Linkman, "Systematic literature reviews in software engineering—A tertiary study," *Inf. Softw. Technol.*, vol. 52, no. 8, pp. 792–805, 2010.

[14] J.-B. Lamy, B. Sekar, G. Guezennec, J. Bouaud, and B. Séroussi, "Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach," *Artif. Intell. Med.*, vol. 94, pp. 42–53, Mar. 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0933365718304846

[15] A. B. Tosun, F. Pullara, M. J. Becich, D. L. Taylor, J. L. Fine, and S. C. Chennubhotla, "Explainable AI (xAI) for anatomic pathology," *Adv. Anatomic Pathol.*, vol. 27, no. 4, pp. 241–250, 2020.

[16] D. Gu, K. Su, and H. Zhao, "A case-based ensemble learning system for explainable breast cancer recurrence prediction," *Artif. Intell. Med.*, vol. 107, Jul. 2020, Art. no. 101858.

[17] M. Masud, A. E. Eldin Rashed, and M. S. Hossain, "Convolutional neural network-based models for diagnosis of breast cancer," *Neural Comput. Appl.*, vol. 34, no. 14, pp. 1–12, Jul. 2022.

[18] Y. J. Suh, J. Jung, and B.-J. Cho, "Automated breast cancer detection in digital mammograms of various densities via deep learning," *J. Personalized Med.*, vol. 10, no. 4, p. 211, Nov. 2020.

[19] H. Rasaee and H. Rivaz, "Explainable AI and susceptibility to adversarial attacks: A case study in classification of breast ultrasound images," in *Proc. IEEE Int. Ultrason. Symp. (IUS)*, Sep. 2021, pp. 1–4.

[20] Y. Yu, P. Kossinna, Q. Li, W. Liao, and Q. Zhang, "Explainable autoencoder-based representation learning for gene expression data," *bioRxiv*, 2021.

[21] H. Chereda, A. Bleckmann, K. Menck, J. Perera-Bel, P. Stegmaier, F. Auer, F. Kramer, A. Leha, and T. Beißbarth, "Explaining decisions of graph convolutional neural networks: Patient-specific molecular subnetworks responsible for metastasis prediction in breast cancer," *Genome Med.*, vol. 13, no. 1, p. 42, Mar. 2021.
[22] V. Modhukur, S. Sharma, M. Mondal, A. Lawarde, K. Kask, R. Sharma, and A. Salumets, “Machine learning approaches to classify primary and metastatic cancers using tissue of origin-based DNA methylation profiles,” *Cancers*, vol. 13, no. 15, p. 3768, Jul. 2021.

[23] Y.-W. Lee, C.-S. Huang, C.-C. Shih, and R.-F. Chang, “Axillary lymph node metastasis status prediction of early-stage breast cancer using convolutional neural networks,” *Comput. Biol. Med.*, vol. 130, Mar. 2021, Art. no. 104206.

[24] F. Dong, R. She, C. Cui, S. Shi, X. Hu, J. Zeng, H. Wu, J. Xu, and Y. Zhang, “One step further into the blackbox: A pilot study of how to build more confidence around an AI-based decision system of breast nodule assessment in 2D ultrasound,” *Eur. Radiol.*, vol. 31, no. 7, pp. 4991–5000, Jul. 2021.

[25] D. Chakraborty, C. Ivan, P. Amero, M. Khan, C. Rodriguez-Aguayo, H. Başağaoğlu, and G. Lopez-Berestein, “Explainable artificial intelligence reveals novel insight into tumor microenvironment conditions linked with better prognosis in patients with breast cancer,” *Cancers*, vol. 13, no. 14, p. 3450, Jul. 2021.

[26] K. A. Kabakçi, A. Çakir, I. Türkmen, B. U. Töreyin, and A. Çapar, “Automated scoring of CerbB2/HER2 receptors using histogram based analysis of immunohistochemistry breast cancer tissue images,” *Biomed. Signal Process. Control*, vol. 69, Aug. 2021, Art. no. 102924.

[27] A. Binder, G. Montavon, S. Lapuschkin, K.-R. Müller, and W. Samek, “Layer-wise relevance propagation for neural networks with local renormalization layers,” in *Proc. Artif. Neural Netw. Mach. Learn.-ICANN: 25th Int. Conf. Artif. Neural Netw.*, Barcelona, Spain, Cham, Switzerland: Springer, 2016, pp. 63–71.

[28] R. Ricciardi, G. Mettivier, M. Staffa, A. Sarno, G. Acampora, S. Minelli, A. Santoro, E. Antignani, A. Orientale, I. A. M. Pilotti, V. Santangelo, P. D'Andria, and P. Russo, “A deep learning classifier for digital breast tomosynthesis,” *Phys. Medica*, vol. 83, pp. 184–193, Mar. 2021.

[29] M. Lou, R. Wang, Y. Qi, W. Zhao, C. Xu, J. Meng, X. Deng, and Y. Ma, “MGBN: Convolutional neural networks for automated benign and malignant breast masses classification,” *Multimedia Tools Appl.*, vol. 80, no. 17, pp. 26731–26750, Jul. 2021.

[30] N. Amoroso, D. Pomarico, A. Fanizzi, V. Didonna, F. Giotta, D. La Forgia, A. Latorre, A. Monaco, E. Pantaleo, N. Petruzzellis, P. Tamborra, A. Zito, V. Lorusso, R. Bellotti, and R. Massafra, “A roadmap towards breast cancer therapies supported by explainable artificial intelligence,” *Appl. Sci.*, vol. 11, no. 11, p. 4881, May 2021.

[31] M. Saarela and S. Jauhiainen, “Comparison of feature importance measures as explanations for classification models,” *Social Netw. Appl. Sci.*, vol. 3, no. 2, pp. 1–12, Feb. 2021.

[32] A. Moncada-Torres, M. C. van Maaren, M. P. Hendriks, S. Siesling, and G. Geleijnse, “Explainable machine learning can outperform cox regression predictions and provide insights in breast cancer survival,” *Sci. Rep.*, vol. 11, no. 1, p. 6968, Mar. 2021.

[33] D. Kaplun, A. Krasichkov, P. Chetyrbok, N. Oleinikov, A. Garg, and H. S. Pannu, “Cancer cell profiling using image moments and neural networks with model agnostic explainability: A case study of breast cancer histopathological (BreakHis) database,” *Mathematics*, vol. 9, no. 20, p. 2616, Oct. 2021.

[34] C. Moreira, Y. Chou, C. Hsieh, C. Ouyang, J. Jorge, and J. Pereira, “Benchmarking counterfactual algorithms for XAI: From white box to black box,” 2022, *arXiv:2203.02399*.

[35] M. Idrees and A. Sohail, “Explainable machine learning of the breast cancer staging for designing smart biomarker sensors,” *Sensors Int.*, vol. 3, Jan. 2022, Art. no. 100202.

[36] F. Wang, D. Wang, Y. Xu, H. Jiang, Y. Liu, and J. Zhang, “Potential of the non-contrast-enhanced chest CT radiomics to distinguish molecular subtypes of breast cancer: A retrospective study,” *Frontiers Oncol.*, vol. 12, Mar. 2022, Art. no. 848726.

[37] Y. Zhang, Y. Weng, and J. Lund, “Applications of explainable artificial intelligence in diagnosis and surgery,” *Diagnostics*, vol. 12, no. 2, p. 237, Jan. 2022.

[38] S. M. Hussain, D. Buongiorno, N. Altini, F. Berloco, B. Prencipe, M. Moschetta, V. Bevilacqua, and A. Brunetti, “Shape-based breast lesion classification using digital tomosynthesis images: The role of explainable artificial intelligence,” *Appl. Sci.*, vol. 12, no. 12, p. 6230, Jun. 2022.

[39] T. M. Hanis, N. I. R. Ruhaiyem, W. N. Arifin, J. Haron, W. F. Wan Abdul Rahman, R. Abdullah, and K. I. Musa, “Over-the-counter breast cancer classification using machine learning and patient registration records,” *Diagnostics*, vol. 12, no. 11, p. 2826, Nov. 2022.

[40] S. Meshoul, A. Batouche, H. Shaiba, and S. AlBinali, “Explainable multi-class classification based on integrative feature selection for breast cancer subtyping,” *Mathematics*, vol. 10, no. 22, p. 4271, Nov. 2022.

[41] F. Silva-Aravena, H. N. Delafuente, J. H. Gutiérrez-Bahamondes, and J. Morales, “A hybrid algorithm of ML and XAI to prevent breast cancer: A strategy to support decision making,” *Cancers*, vol. 15, no. 9, p. 2443, Apr. 2023.

[42] S. Deshmukh, B. K. Behera, P. Mulay, E. A. Ahmed, S. Al-Kuwari, P. Tiwari, and A. Farouk, “Explainable quantum clustering method to model medical data,” *Knowl.-Based Syst.*, vol. 267, May 2023, Art. no. 110413.

[43] R. Massafra et al., “Analyzing breast cancer invasive disease event classification through explainable artificial intelligence,” *Frontiers Med.*, vol. 10, Feb. 2023, Art. no. 1116354.

[44] J. Wang, Y. Zheng, J. Ma, X. Li, C. Wang, J. Gee, H. Wang, and W. Huang, “Information bottleneck-based interpretable multitask network for breast cancer classification and segmentation,” *Med. Image Anal.*, vol. 83, Jan. 2023, Art. no. 102687.

[45] J. Vrdoljak, Z. Boban, D. Baric, D. Šegvic, M. Kumric, M. Avirovic, M. P. Balja, M. M. Periša, C. Tomasovic, and S. Tomić, “Applying explainable machine learning models for detection of breast cancer lymph node metastasis in patients eligible for neoadjuvant treatment,” *Cancers*, vol. 15, no. 3, p. 634, Jan. 2023.

[46] S. Kumar and A. Das, “Peripheral blood mononuclear cell derived biomarker detection using eXplainable artificial intelligence (XAI) provides better diagnosis of breast cancer,” *Comput. Biol. Chem.*, vol. 104, Jun. 2023, Art. no. 107867.

[47] M. Prodan, E. Paraschiv, and A. Stanciu, “Applying deep learning methods for mammography analysis and breast cancer detection,” *Appl. Sci.*, vol. 13, no. 7, p. 4272, Mar. 2023.

[48] T. Khater, A. Hussain, R. Bendardaf, I. M. Talaat, H. Tawfik, S. Ansari, and S. Mahmoud, “An explainable artificial intelligence model for the classification of breast cancer,” *IEEE Access*, vol. 13, pp. 5618–5633, 2025.

[49] B. Yagin, F. Yagin, C. Colak, F. Inceoglu, S. Kadry, and J. Kim, “Cancer metastasis prediction and genomic biomarker identification through machine learning and eXplainable artificial intelligence in breast cancer research,” *Diagnostics*, vol. 13, no. 21, p. 3314, Oct. 2023.

[50] K. Raghavan, B. Sivaselvan, and V. Kamakoti, “Attention guided grad-CAM: An improved explainable artificial intelligence model for infrared breast cancer detection,” *Multimedia Tools Appl.*, vol. 83, pp. 57551–57578, Dec. 2023.

[51] T. Aleyliani, M. M. Alshammari, A. Almuhanna, and O. Asan, “Explainable artificial intelligence in quantifying breast cancer factors: Saudi Arabia context,” *Healthcare*, vol. 12, no. 10, p. 1025, May 2024. [Online]. Available: https://www.mdpi.com/2227-9032/12/10/1025

[52] R. M. Munshi, L. Cascone, N. Alturki, O. Saidani, A. Alshardan, and M. Umer, “A novel approach for breast cancer detection using optimized ensemble learning framework and XAI,” *Image Vis. Comput.*, vol. 142, Feb. 2024, Art. no. 104910.

[53] E. Cerekcı, D. Alis, N. Denizoglu, O. Camurdan, M. Ege Seker, C. Ozer, M. Y. Hansu, T. Tanyel, I. Oksuz, and E. Karaarslan, “Quantitative evaluation of saliency-based explainable artificial intelligence (XAI) methods in deep learning-based mammogram analysis,” *Eur. J. Radiol.*, vol. 173, Apr. 2024, Art. no. 111356.

[54] T. K. Murugan, P. Karthikeyan, and P. Sekar, “Efficient breast cancer detection using neural networks and explainable artificial intelligence,” *Neural Comput. Appl.*, vol. 37, no. 5, pp. 3759–3776, Feb. 2025.

[55] M. A. Talukder, “An improved XAI-based DenseNet model for breast cancer detection using reconstruction and fine-tuning,” *Results Eng.*, vol. 26, Jun. 2025, Art. no. 104802.

[56] A. Alhussen, M. Anul Haq, A. Ahmad Khan, R. K. Mahendran, and S. Kadry, “XAI-RACapsNet: Relevance aware capsule network-based breast cancer detection using mammogram images via explainability O-Net ROI segmentation,” *Expert Syst. Appl.*, vol. 261, Feb. 2025, Art. no. 125461.
[57] M. R. Alom, F. A. Farid, M. A. Rahaman, A. Rahman, T. Debnath, A. S. M. Miah, and S. Mansor, "An explainable AI-driven deep neural network for accurate breast cancer detection from histopathological and ultrasound images," *Sci. Rep.*, vol. 15, no. 1, p. 17531, May 2025.

[58] W. H. Wolberg, W. N. Street, and O. L. Mangasarian. (1992). *Breast Cancer Wisconsin (Diagnostic) Data Set*. [Online]. Available: http://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+ (Diagnostic)

[59] Z. Matjaz and S. Milan, “Breast cancer,” in *UCI Machine Learning Repository*. Irvine, CA, USA: UCI Machine Learning Repository, University of California, Irvine, 1988, doi: 10.24432/C51P4M.

[60] (2021). *Mammographic Mass Data Set*. Kaggle Dataset. [Online]. Available: https://www.kaggle.com/datasets/overratedgman/mammographic-mass-data-set/data

[61] R. S. Lee, F. Gimenez, A. Hoogi, K. K. Miyake, M. Gorovoy, and D. L. Rubin, “A curated mammography data set for use in computer-aided detection and diagnosis research,” *Sci. Data*, vol. 4, no. 1, pp. 1–9, Dec. 2017.

[62] M.-L. Huang and T.-Y. Lin, “Dataset of breast mammography images with masses,” *Data Brief*, vol. 31, Aug. 2020, Art. no. 105928.

[63] M. D. Halling-Brown, L. M. Warren, D. Ward, E. Lewis, A. Mackenzie, M. G. Wallis, L. S. Wilkinson, R. M. Given-Wilson, R. McAvinchey, and K. C. Young, “OPTIMAM mammography image database: A large-scale resource of mammography images and clinical data,” *Radiol., Artif. Intell.*, vol. 3, no. 1, Jan. 2021, Art. no. e200103.

[64] *The Cancer Genome Atlas (TCGA)*. Accessed: Nov. 3, 2025. [Online]. Available: https://www.cancer.gov/ccg/research/genome-sequencing/tcga

[65] *Gene Expression Omnibus (Geo)*. Accessed: Nov. 3, 2025. [Online]. Available: https://www.ncbi.nlm.nih.gov/geo/

[66] H. G. LaBreche, J. R. Nevins, and E. Huang, “Integrating factor analysis and a transgenic mouse model to reveal a peripheral blood predictor of breast tumors,” *BMC Med. Genomics*, vol. 4, no. 1, pp. 1–14, Dec. 2011.

[67] S. R. Piccolo, L. M. Hoffman, T. Conner, G. Shrestha, A. L. Cohen, J. R. Marks, L. A. Neumayer, C. A. Agarwal, M. C. Beckerle, I. L. Andrulis, A. E. Spira, P. J. Moos, S. S. Buys, W. E. Johnson, and A. H. Bild, “Integrative analyses reveal signaling pathways underlying familial breast cancer susceptibility,” *Mol. Syst. Biol.*, vol. 12, no. 3, p. 860, Mar. 2016.

[68] W. Al-Dhabyani, M. Gomaa, H. Khaled, and A. Fahmy, “Dataset of breast ultrasound images,” *Data Brief*, vol. 28, Feb. 2020, Art. no. 104863.

[69] P. S. Rodrigues, “Breast ultrasound image,” Mendeley Data, Amsterdam, The Netherlands, Dec. 2017. Accessed: Nov. 3, 2025. [Online]. Available: https://data.mendeley.com/datasets/wmy84gngw1 and http://dx.doi.org/10.17632/wmy84gngw1

[70] F. A. Spanhol, L. S. Oliveira, C. Petitjean, and L. Heutte, “A dataset for breast cancer histopathological image classification,” *IEEE Trans. Biomed. Eng.*, vol. 63, no. 7, pp. 1455–1462, Jul. 2016.

[71] L. F. Silva, D. C. M. Saade, G. O. Sequeiros, A. C. Silva, A. C. Paiva, R. S. Bravo, and A. Conci, “A new database for breast research with infrared image,” *J. Med. Imag. Health Informat.*, vol. 4, no. 1, pp. 92–100, Mar. 2014.

[72] A. Binder, M. Bockmayr, M. Hägele, S. Wienert, D. Heim, K. Hellweg, M. Ishii, A. Stenzinger, A. Hocke, C. Denkert, K.-R. Müller, and F. Klauschen, “Morphological and molecular breast cancer profiling through explainable machine learning,” *Nature Mach. Intell.*, vol. 3, no. 4, pp. 355–366, Mar. 2021.

[73] H. Sharif and R. A. Khan, “A novel machine learning based framework for detection of autism spectrum disorder (ASD),” *Appl. Artif. Intell.*, vol. 36, no. 1, Dec. 2022, Art. no. 2004655.

[74] Y. Li, X. Wu, P. Yang, G. Jiang, and Y. Luo, “Machine learning for lung cancer diagnosis, treatment, and prognosis,” *Genomics, Proteomics Bioinf.*, vol. 20, no. 5, pp. 850–866, Oct. 2022.

[75] S. M. Shah, R. A. Khan, S. Arif, and U. Sajid, “Artificial intelligence for breast cancer analysis: Trends & directions,” *Comput. Biol. Med.*, vol. 142, Mar. 2022, Art. no. 105221.

[76] S. Nasir, R. Ahmed Khan, and S. Bai, “Ethical framework for harnessing the power of AI in healthcare and beyond,” 2023, arXiv:2309.00064.

[77] S. Lundberg and S. Lee, “A unified approach to interpreting model predictions,” in *Proc. Adv. Neural Inf. Process. Syst.*, 2017, pp. 1–10.

[78] M. T. Ribeiro, S. Singh, and C. Guestrin, “Why should I trust you?: Explaining the predictions of any classifier,” in *Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining*, 2016, pp. 1135–1144.

[79] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” in *Proc. IEEE Int. Conf. Comput. Vis. (ICCV)*, Oct. 2017, pp. 618–626.

[80] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR)*, Jun. 2016, pp. 2921–2929.

[81] K.-D. Althoff, “Case-based reasoning,” in *Handbook of Software Engineering and Knowledge Engineering: Volume 1: Fundamentals*. Singapore: World Scientific, 2001, pp. 549–587.

[82] S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek, “On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,” *PLoS ONE*, vol. 10, no. 7, Jul. 2015, Art. no. e0130140.

[83] A. Altmann, L. Tolosi, O. Sander, and T. Lengauer, “Permutation importance: A corrected feature importance measure,” *Bioinformatics*, vol. 26, no. 10, pp. 1340–1347, May 2010.

[84] L. Gianfagna and A. Di Cecco, *Explainable AI With Python*. Cham, Switzerland: Springer, 2021.

[85] L. Breiman, “Random forests,” *Mach. Learn.*, vol. 45, no. 1, pp. 5–32, Oct. 2001.

[86] H. Ishwaran, U. B. Kogalur, E. H. Blackstone, and M. S. Lauer, *Random Survival Forests*. Waite Hill, OH, USA: Institute of Mathematical Statistics, 2008.

[87] J. H. Friedman, “Greedy function approximation: A gradient boosting machine,” *Ann. Statist.*, vol. 29, no. 5, pp. 1189–1232, Oct. 2001.

[88] Y. Freund and R. E. Schapire, “A decision-theoretic generalization of on-line learning and an application to boosting,” *J. Comput. Syst. Sci.*, vol. 55, no. 1, pp. 119–139, Aug. 1997.

[89] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR)*, Jun. 2015, pp. 1–9.

[90] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in *Proc. Adv. Neural Inf. Process. Syst.*, vol. 60, 2017, pp. 84–90.

[91] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014, arXiv:1409.1556.

[92] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR)*, Jul. 2017, pp. 1800–1807.

[93] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR)*, Jun. 2016, pp. 770–778.

[94] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR)*, Jul. 2017, pp. 2261–2269.

[95] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” in *Proc. IEEE Int. Conf. Comput. Vis. (ICCV)*, Oct. 2017, pp. 2980–2988.

[96] M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” in *Proc. Adv. Neural Inf. Process. Syst.*, vol. 29, 2016, pp. 1–9.

[97] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient convolutional neural networks for mobile vision applications,” 2017, arXiv:1704.04861.

[98] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size,” 2016, arXiv:1602.07360.

[99] S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR)*, Jul. 2017, pp. 5987–5995.

[100] J. Redmon and A. Farhadi, “YOLO9000: Better, faster, stronger,” in *Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR)*, Jul. 2017, pp. 6517–6525.

[101] B. Koonce, “EfficientNet,” *Convolutional Neural Networks With Swift for Tensorflow: Image Recognition and Dataset Categorization*. Berkeley, CA, USA: Apress, 2021, pp. 109–123.
[102] T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Aug. 2016, pp. 785–794.

[103] B. E. Boser, I. M. Guyon, and V. N. Vapnik, “A training algorithm for optimal margin classifiers,” in Proc. 5th Annu. Workshop Comput. Learn. Theory, Jul. 1992, pp. 144–152.

[104] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T. Liu, “LightGBM: A highly efficient gradient boosting decision tree,” in Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 1–9.

[105] A. Veronika Dorogush, V. Ershov, and A. Gulin, “CatBoost: Gradient boosting with categorical features support,” 2018, arXiv:1810.11363.

[106] G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, vol. 313, no. 5786, pp. 504–507, Jul. 2006.

[107] P. Hansen and B. Jaumard, “Cluster analysis and mathematical programming,” Math. Program., vol. 79, nos. 1–3, pp. 191–215, Oct. 1997.

[108] Z. Tu, H. Talebi, H. Zhang, F. Yang, P. Milanfar, A. Bovik, and Y. Li, “MaxViT: Multi-axis vision transformer,” in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, 2022, pp. 459–479.

[109] P. Langley, a. W. Iba, and K. Thompson, “An analysis of Bayesian classifiers,” in Proc. AAAI, vol. 90, 1992, pp. 223–228.

[110] C. D. Manning, *An Introduction to Information Retrieval*. Cambridge, U.K.: Cambridge Univ. Press, 2009.

[111] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997.

[112] A. Barredo Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, S. Garcia, S. Gil-Lopez, D. Molina, R. Benjamins, R. Chatila, and F. Herrera, “Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI,” Inf. Fusion, vol. 58, pp. 82–115, Jun. 2020.

[113] K. Sokol and P. Flach, “Explainability fact sheets: A framework for systematic assessment of explainable approaches,” in Proc. Conf. Fairness, Accountability, Transparency, Jan. 2020, pp. 56–67.

[114] H. P. Young, “Monotonic solutions of cooperative games,” Int. J. Game Theory, vol. 14, no. 2, pp. 65–72, Jun. 1985.

[115] A. Khan, “Model-specific explainable artificial intelligence techniques: State-of-the-art, advantages and limitations,” Fac. Elect. Eng., Math. Comput. Sci. (EEMCS), Delft Univ. Technol., Delft, The Netherlands, Tech. Rep., 2022.

[116] L. Van der Maaten and G. Hinton, “Visualizing data using t-SNE,” J. Mach. Learn. Res., vol. 9, no. 11, pp. 2579–2605, 2008.

[117] L. McInnes, J. Healy, and J. Melville, “UMAP: Uniform manifold approximation and projection for dimension reduction,” 2018, arXiv:1802.03426.

[118] O. Biran and C. Cotton, “Explanation and justification in machine learning: A survey,” in Proc. IJCAI Workshop Explainable AI (XAI), 2017, pp. 8–13.

[119] R. M. J. Byrne, “Counterfactuals in explainable artificial intelligence (XAI): Evidence from human reasoning,” in Proc. 28th Int. Joint Conf. Artif. Intell., Aug. 2019, pp. 6276–6282.

[120] R. Guidotti, A. Monreale, F. Giannotti, D. Pedreschi, S. Ruggieri, and F. Turini, “Factual and counterfactual explanations for black box decision making,” IEEE Intell. Syst., vol. 34, no. 6, pp. 14–23, Nov. 2019.

[121] T. Miller, “Explanation in artificial intelligence: Insights from the social sciences,” Artif. Intell., vol. 267, pp. 1–38, Feb. 2019.

[122] S. Wachter, B. Mittelstadt, and C. Russell, “Counterfactual explanations without opening the black box: Automated decisions and the GDPR,” Harvard J. Law Technol., vol. 31, no. 2, p. 841, 2017.

[123] U. Bhatt, A. Xiang, S. Sharma, A. Weller, A. Taly, Y. Jia, J. Ghosh, R. Puri, J. M. Moura, and P. Eckersley, “Explainable machine learning in deployment,” in Proc. Conf. Fairness, Accountability, Transparency, 2020, pp. 648–657.

[124] J. G. Santillán-Benítez, H. Mendieta-Zerón, L. M. Gómez-Oliván, J. J. Torres-Juárez, J. M. González-Bañales, L. V. Hernández-Peña, and A. Ordóñez-Quiroz, “The tetrad BMI, leptin, leptin/adiponectin (L/A) ratio and CA 15-3 are reliable biomarkers of breast cancer,” J. Clin. Lab. Anal., vol. 27, no. 1, pp. 12–20, Jan. 2013.

[125] A. M. A. Assiri and H. F. M. Kamel, “Evaluation of diagnostic and predictive value of serum adipokines: Leptin, resistin and visfatin in postmenopausal breast cancer,” Obesity Res. Clin. Pract., vol. 10, no. 4, pp. 442–453, Jul. 2016.

[126] M. Patrício, J. Pereira, J. Crisóstomo, P. Matafome, M. Gomes, R. Seiça, and F. Caramelo, “Using resistin, glucose, age and BMI to predict the presence of breast cancer,” BMC Cancer, vol. 18, no. 1, pp. 1–8, Dec. 2018.

[127] M. Ahmed, T. Bibi, R. A. Khan, and S. Nasir, “Enhancing breast cancer diagnosis in mammography: Evaluation and integration of convolutional neural networks and explainable AI,” in Proc. 26th Int. Multi-Topic Conf. (INMIC), Dec. 2024, pp. 1–6.

[128] A. Adadi and M. Berrada, “Peeking inside the black-box: A survey on explainable artificial intelligence (XAI),” IEEE Access, vol. 6, pp. 52138–52160, 2018.

[129] M. Tulio Ribeiro, S. Singh, and C. Guestrin, “Model-agnostic interpretability of machine learning,” 2016, arXiv:1606.05386.

[130] J.-B. Lamy, B. Sekar, G. Guezennec, J. Bouaud, and B. Séroussi, “Hierarchical visual case-based reasoning for supporting breast cancer therapy,” in Proc. 5th Int. Conf. Adv. Biomed. Eng. (ICABME), Oct. 2019, pp. 1–4.

[131] S. M. Lundberg, B. Nair, M. S. Vavilala, M. Horibe, M. J. Eisses, T. Adams, D. E. Liston, D. K.-W. Low, S.-F. Newman, J. Kim, and S.-I. Lee, “Explainable machine-learning predictions for the prevention of hypoxaemia during surgery,” Nature Biomed. Eng., vol. 2, no. 10, pp. 749–760, Oct. 2018.

[132] A. Chaddad, J. Peng, J. Xu, and A. Bouridane, “Survey of explainable AI techniques in healthcare,” Sensors, vol. 23, no. 2, p. 634, Jan. 2023.

[133] D. Alvarez-Melis and T. S. Jaakkola, “On the robustness of interpretability methods,” 2018, arXiv:1806.08049.

[134] R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi, “A survey of methods for explaining black box models,” ACM Comput. Surv., vol. 51, no. 5, pp. 1–42, Sep. 2019.

[135] M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in Proc. Int. Conf. Mach. Learn., 2017, pp. 3319–3328.

[136] S. Jain and B. C. Wallace, “Attention is not explanation,” 2019, arXiv:1902.10186.

[137] S. Abnar and W. Zuidema, “Quantifying attention flow in transformers,” 2020, arXiv:2005.00928.

[138] H. Chefer, S. Gur, and L. Wolf, “Transformer interpretability beyond attention visualization,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 782–791.

[139] Z. Wang, Z. Wu, D. Agarwal, and J. Sun, “MedCLIP: Contrastive learning from unpaired medical images and text,” in Proc. Conf. Empirical Methods Natural Lang. Process., 2022, pp. 3876–3887.

[140] E. Tiu, E. Talius, P. Patel, C. P. Langlotz, A. Y. Ng, and P. Rajpurkar, “Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning,” Nature Biomed. Eng., vol. 6, no. 12, pp. 1399–1406, Sep. 2022.

[141] European Parliament and Council. (2024). *Regulation (EU) 2024/1689 (Artificial Intelligence Act)*. Official Journal of the European Union. Jul. 2024. Accessed: Jul. 12, 2024. [Online]. Available: https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng

[142] FDA, Health Canada, and MHRA. (2021). *Good Machine Learning Practice for Medical Device Development: Guiding Principles*. [Online]. Available: https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-medical-device-development-guiding-principles

[143] (Jun. 2024). *Transparency for Machine Learning-Enabled Medical Devices: Guiding Principles*. [Online]. Available: https://www.fda.gov/medical-devices/software-medical-device-samd/transparency-machine-learning-enabled-medical-devices-guiding-principles

[144] *Information Technology—Artificial Intelligence—Guidance on Risk Management*, International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC), Standard ISO/IEC 23894:2023, 2023. [Online]. Available: https://www.iso.org/standard/77304.html

[145] *Software as a Medical Device (SAMD): Clinical Evaluation*, International Medical Device Regulators Forum (IMDRF), document IMDRF/SaMD WG/N41FINAL:2017, 2017. [Online]. Available: https://www.imdrf.org/documents/software-medical-device-samd-clinical-evaluation

[146] S. C. Rivera, X. Liu, A.-W. Chan, A. K. Denniston, and M. J. Calvert, “Guidelines for clinical trial protocols for interventions involving artificial intelligence: The SPIRIT-AI extension,” Lancet Digit. Health, vol. 2, no. 10, pp. e549–e560, 2020.
[147] X. Liu, S. C. Rivera, D. Moher, M. J. Calvert, and A. K. Denniston, "Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: The CONSORT-AI extension," *Lancet Digit. Health*, vol. 2, no. 10, pp. e537-e548, 2020.

[148] M. Nauta, J. Trienes, S. Pathak, E. Nguyen, M. Peters, Y. Schmitt, J. Schlötterer, M. van Keulen, and C. Seifert, "From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI," 2022, arXiv:2201.08164.

[149] J. Kim, H. Maathuis, and D. Sent, "Human-centered evaluation of explainable AI applications: A systematic review," *Frontiers Artif. Intell.*, vol. 7, Oct. 2024, Art. no. 1456486. [Online]. Available: https://www.frontiersin.org/articles/10.3389/frai.2024.1456486/full

[150] I. Donoso-Guzmán, K. S. Kacafírková, M. Szymanski, A. Jacobs, D. Parra, and K. Verbert, "A systematic review of user-centred evaluation of explainable AI in healthcare," 2025, arXiv:2506.13904.

[151] S. Goh, R. S. J. Goh, B. Chong, Q. X. Ng, G. C. H. Koh, K. Y. Ngiam, and M. Hartman, "Challenges in implementing artificial intelligence in breast cancer screening programs: Systematic review and framework for safe adoption," *J. Med. Internet Res.*, vol. 27, May 2025, Art. no. e62941.

[152] K. Borys, Y. A. Schmitt, M. Nauta, C. Seifert, N. Krämer, C. M. Friedrich, and F. Nensa, "Explainable AI in medical imaging: An overview for clinical practitioners—Beyond saliency-based XAI approaches," *Eur. J. Radiol.*, vol. 162, May 2023, Art. no. 110786, doi: 10.1016/j.ejrad.2023.110786.

[153] Z. Sadeghi, R. Alizadehsani, M. A. Çiççi, S. Kausar, R. Rehman, P. Mahanta, P. K. Bora, A. Almasri, R. S. Alkhawaldeh, S. Hussain, B. Alatas, A. Shoeibi, H. Moosaei, M. Hladík, S. Nahavandi, and P. M. Pardalos, "A review of explainable artificial intelligence in healthcare," *Inf. Fusion*, vol. 118, Jan. 2024, Art. no. 109370.

[154] C. M. van Leersum and C. Maathuis, "Human centred explainable AI decision-making in healthcare," *J. Responsible Technol.*, vol. 21, Mar. 2025, Art. no. 100108.

[155] M. A. TelemedToday, "The role of explainable AI and evaluation frameworks for safe and trusted medical AI," *Telehealth Med. Today*, vol. 9, no. 1, pp. 485-500, 2024.

[156] M. Rozanec, E. Montini, V. Cutrona, D. Papamartzivanos, T. Klemencic, B. Fortuna, D. Mladenic, E. Veliou, T. Giannetsos, and C. Emmanouilidis, "Human in the AI loop via xAI and active learning for visual inspection," in *Artificial Intelligence in Manufacturing: Enabling Intelligent, Flexible and Cost-Effective Production Through AI*, J. Soldatos, Ed., Cham, Switzerland: Springer, 2023, pp. 381-406, doi: 10.1007/978-3-031-46452-2_22.

[157] M. A. Kadir, A. Mosavi, and D. Sonntag, "Evaluation metrics for XAI: A review, taxonomy, and practical applications," in *Proc. IEEE 27th Int. Conf. Intell. Eng. Syst. (INES)*, Jul. 2023, pp. 111-124. [Online]. Available: https://ieeexplore.ieee.org/document/10297629

[158] B. Ko?ak, A. Ponsiglione, A. Stanzione, C. Bluethgen, J. Santinha, L. Ugga, M. Huisman, M. E. Klontzas, R. Cannella, and R. Cuocolo, "Bias in artificial intelligence for medical imaging: Fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects," *Diagnostic Interventional Radiol.*, vol. 31, no. 2, pp. 75-88, 2025.

[159] F. Pahde, M. Dreyer, W. Samek, and S. Lapuschkin, "Reveal to revise: An explainable ai life cycle for iterative bias correction of deep models," in *Medical Image Computing and Computer Assisted Intervention–(MICCAI)* (Lecture Notes in Computer Science), vol. 14221. Cham, Switzerland: Springer, 2023, pp. 596-606.

<figure><img src="image_11.png" /></figure>

**SAMITA BAI** received the Ph.D. degree in computer science from the Institute of Business Administration (IBA), Karachi, Pakistan, in 2021. She is currently a Postdoctoral Fellow with Canadian Institute of Cybersecurity (CIC), University of New Brunswick, Fredericton, NB Canada. Her research interests include explainable artificial intelligence (XAI), semantic web, information retrieval, machine learning, and AI applications in cybersecurity and healthcare.

<figure><img src="image_12.png" /></figure>

**SIDRA NASIR** is currently pursuing the Ph.D. degree in computer science with Università di Verona, Italy, with a focus on multimodal large language models (LLMs) and large vision models (LVMs) for real-world decision support. Her research interests include explainable AI (XAI), trustworthy ML, and cybersecurity of AI systems.

<figure><img src="image_13.png" /></figure>

**RIZWAN AHMED KHAN** received the Ph.D. degree in machine learning and computer vision from Université Claude Bernard Lyon 1, France. He is currently a Professor with the Department of Computer Science, Institute of Business Administration (IBA), Karachi, Pakistan. He is also an Invited Professor and a Researcher with Université Jean Monnet, France, where he contributes to teaching and research in computer vision and artificial intelligence. He was a Postdoctoral

Researcher in immersive gaming environments, France. He has authored numerous publications and actively contributes to advancing AI for healthcare and societal impact. His research interests include responsible AI, explainable AI (XAI), medical imaging, and computer vision. He is an Erasmus Mundus Scholar. He was a recipient of the Best University Teacher Award, in 2023.

<figure><img src="image_14.png" /></figure>

**ALEXANDRE MEYER** received the Ph.D. degree in computer science from Université Grenoble 1, France, in 2001. From 2002 to 2003, he was a Postdoctoral Fellow with the University College London. Since 2004, he has been an Associate Professor with Université Claude Bernard Lyon 1, France, and a member of the LIRIS Research Laboratory. His current research interests include computer animation and computer vision of characters.

<figure><img src="image_15.png" /></figure>

**HUBERT KONIK** received the Ph.D. degree in computer science from Université Jean Monnet, in 1995. He is currently an Associate Professor with Télécom Saint-Etienne and a member of the Image Science and Computer Vision Team, Laboratoire Hubert Curien (LHC), Saint-Etienne, France. His research interests include image processing and analysis, more particularly, content-aware image processing for new services and usages.

<figure><img src="image_16.png" /></figure>