## 1 Introduction  
### Backdrop  
In recent years, ML and DL techniques have achieved unprecedented success in diverse applications, ranging from image recognition to natural language processing (NLP). These powerful algorithms have the potential to revolutionize medical practice, driving innovation in diagnostics, treatment, and patient care [1]. In particular, the intersection of ML and medicine holds great promise for improving the accuracy and efficiency of diagnosis in complex diseases, such as CAD [2], which is the main problem this paper deals with. Despite the remarkable strides made by ML and DL in medical applications, their success comes with an inherent drawback--the \"black-box\" nature of these models [3]. Black-box models are characterized by their complex, high-dimensional architectures, where the relationship between inputs and outputs becomes increasingly inscrutable [4]. While these models demonstrate outstanding predictive capabilities, their lack of explainability poses significant challenges in critical domains, especially healthcare [5]. The opacity of black-box models not only hampers the acceptance of artificial intelligence (AI) solutions in medical practice but also raises concerns related to safety, ethics, and regulatory compliance [3]. Explainability, in the context of ML and DL, refers to the ability to comprehend and interpret the decision-making process of a model [6]. Traditionally, simpler models, such as linear regression and decision trees, offered transparent and understandable insights into their predictions [6]. However, the advent of more sophisticated algorithms, such as deep neural networks, has shifted the focus towards performance optimization at the expense of interpretability [7]. The field of medical diagnosis, and especially CAD diagnosis, heavily relies on the expertise of healthcare professionals, who meticulously analyze medical data, patient history, and other relevant factors to arrive at accurate diagnoses [8]. As AI algorithms are integrated into the diagnostic process, ensuring the transparency of their predictions becomes paramount. Physicians and clinicians demand an understanding of why an AI model arrives at a specific diagnosis to instill confidence in its use and facilitate better-informed clinical decisions [9]. Explainability bridges the gap between AI models' predictive capabilities and human comprehension, fostering trust and facilitating AI adoption in healthcare [5; 7]. Moreover, from an ethical perspective, explainability enhances transparency and accountability. The \"right to explanation\" is an emerging ethical principle, particularly in the context of AI deployment [9]. This principle asserts that individuals have the right to understand the rationale behind AI-driven decisions concerning their health and well-being. In medical practice, this principle not only aligns with regulatory requirements but also safeguards patients' autonomy and informed consent [3]. Amid the growing concern over the black-box nature of AI models, researchers have explored alternative techniques that offer boosted interpretability. Fuzzy Cognitive Maps [10; 11], an established method in the realm of soft computing, have emerged as a promising paradigm to address the challenge of explainability in ML and DL applications. FCMs are graph-based models that excel in capturing complex causal relationships in a dynamic system [11]. Comprising a collection of interconnected concepts represented as nodes, FCMs leverage fuzzy logic to model imprecise relationships between these concepts [10; 11]. The strengths and directions of connections, represented by weighted edges, enable the representation of expert knowledge and domain expertise, making FCMs particularly suitable for medical applications [12; 13; 14]. Recent publications, such as those by [PERSON] et al. [15; 16; 17], have further advanced the field by enabling the processing of images within the FCM framework.