—Within the past few years, the accuracy of deep learning and machine learning models has been improving significantly while less attention has been paid to their respon- sibility, explainability, and interpretability. eXplainable Artificial Intelligence (XAI) methods, guidelines, concepts, and strategies offer the possibility of models' evaluation for improving fidelity, faithfulness, and overall explainability. Due to the diversity of data and learning methodologies, there needs to be a clear definition for the validity, reliability, and evaluation metrics of ex- plainability. This article reviews evaluation metrics used for XAI through the PRISMA systematic guideline for a comprehensive and systematic literature review. Based on the results, this study suggests two taxonomy for the evaluation metrics. One taxonomy is based on the applications, and one is based on the evaluation metrics. Keywords —XAI, machine learning, deep learning, explainable artificial intelligence, explainable AI, explainable machine learn- ing; metrics; evaluation I. I NTRODUCTION eXplainable Artificial Intelligence (XAI[1]) is concerned with the development of methodologies for understanding the underlying logic behind machine learning and deep learning models' decision and generally, widely AI applications [2, 3]. The ultimate aspiration of XAI is to facilitate users in con- structing a comprehensive and accurate cognitive representa- tion of machine learning algorithm to encourage confidence in its outputs [4, 5, 6]. Despite the numerous methods and approaches proposed for explainability, scholars still have no agreement on what an Explanation truly is and what properties should be considered for it to be practical and understandable for the end user [7]. Future research should focus on the definition of explainability and structured formats of Explanations that can encompass various attributes and dimensions, incorporating as many aspects as possible. Identify applicable funding agency here. If none, delete this.Additionally, explainability should be considered a con- cept from psychology. It is linked to constructs such as trust, transparency, and privacy, and humans are the final consumers of explanations[8]. It reminds us that interactive visual explanations are not an optional aspect of XAI. Thus, in order to develop successful XAI models, research on psychometrics should be performed. Many authors studied the explainability in AI sub-domains through case studies or surveys, motivating the need for fundamental research on measuring explainability[9, 10, 11]. Researchers reviewed methods for explanations with neural and Bayesian networks and clustered scientific contributions devoted to extracting rules. The goals are to create rules interpretable by humans while maintaining accuracy [12]. Few scholars attempted a comprehensive survey and organization of explainability methods as a whole [13]. Others identified a broad set of requirements an explanation should meet to be understandable by laypeople and provide actionable in- formation to support decision-making [14, 15, 16]. In their works, a thorough evaluation analysis highlighted the need for a systematic analysis of the metrics related to explainability. Consequently, this paper aims to fill this gap by systemati- cally reviewing research studies in XAI, focusing on a subset of peer-reviewed articles that tackled explainability from a conceptual and theoretical point of view and proposed ap- proaches to evaluate XAI methods. The conceptual framework is that explainers build explainability methods that can be evaluated using evaluation metrics. When creating interactive visual explanations, explainers often rely on complex models or numerical approximations [17, 18, 19, 20]. However, without a robust evaluation metric for explana- tions, it is not easy to generalize the causality between a trained model and its visual explainer. The absence of robust evaluation can lead to inconsistencies and inaccuracies in the explanations provided, making it challenging for users to un- derstand the underlying concepts fully. Therefore, developing a reliable evaluation metric is essential to ensure interactive visual explanations are compelling and informative. With such a metric, explanation quality and reliability can be better assessed, allowing them to create more accurate and valuable visualizations for AI practitioners. II. M ETHODOLOGY The methodology is based on the comprehensive literature search integrated with systematic screening and literature review following the PRISMA guideline. The fundamen- tal search database is Scopus1complemented with Google Scholar2. Scopus is a reliable search engine for peer-reviewed scientific literature. Scopus indexes journals and conference proceedings articles that meet adequate scientific standards and measures. For this research, Scopus has been used to find relevant articles. However, pioneers of AI may also use Arxiv3 and other preprints to share an early version of their articles to make the results available faster. We also explore Google Scholar when Scopus fails to find adequate literature or when complementary information is needed. The initial queries are ‘explainable artificial intelligence’, ‘XAI’, ‘explainable AI’, ‘explainable machine learning’, and ‘explainable deep learning’. The queries result in 6122 articles, including various methods and applications of explainable machine learning and deep learning. The research methodology flow diagram, which follows the PRISMA4guideline, is visualized in Fig. 1. Through the methodology ' fundamental three phases, the initial 6122 documents are refined to the desired and relevant articles, including the XAI evaluation metrics. The initial documents include a vast number of documents written about XAI. Further screening is essential to refine what articles include explainability evaluation metrics. The challenge is that there is no common keyword to look for as explainability in various applications and scientific domains might be defined differently. We included a comprehensive list of keywords and relevant phrases generally used to define explainability. Here, it is worth mentioning that relying on the keywords of ‘evaluation metrics’ or ‘metrics’ cannot be efficient in finding the relevant articles for explainability evaluation metrics be- cause explainability can be defined and communicated through different phrases, and also, the term ‘evaluation metrics’ might not be directly communicated throughout the literature. There- fore, exploring the literature using the relevant keywords to explainability complemented with the keywords of ‘evaluation metrics’ or ‘metrics’ is considered an efficient method to remove the irrelevant articles. Consequently, screening the keywords of metrics, eval- uation, and explainability through the entire fields of the articles results in 884 documents. Following keywords related 1https://www.scopus.com/home.uri 2https://scholar.google.com 3https://arxiv.org 4http://www.prisma-statement.org/Studies included in quantitative synthesis (meta-analysis) 75Studies included in quantitative synthesis 70Full-text articles assessed for eligibility 225Records screened 884Records excluded 5238Records after duplicate removed 6122Records identified through database searching 6122Included Eligibility Screening Identification Fig. 1. The research methodology flow diagram follows the PRISMA guidelines to identify relevant literature and screen it to narrow down the amount of literature. to explainability were explored in this stage, i.e., Action- ability ,Transparency ,Transferability ,Completeness ,Satisfac- tion,Sensitivity ,Stability ,Informativeness ,Robustness ,Under- standability ,Monotonicity ,Comprehensibility ,Correctability , Interpretability ,Efficiency ,Explicability ,Explicitness ,Faith- fulness ,Intelligibility ,Interactivity ,Interestingness after fur- ther screening the titles, abstract and keywords the number of relevant articles is reduced to 225. After a second screening by reading and going through articles the 155 articles had been removed. The excluded articles were concerned with the evaluation metrics irrelevant to explainability. Often the performance and accuracy had been measured and the explainability had been discussed shortly. In the final stage the articles were reduced to 75. At the next stage the original studies and most relevant articles are selected for review resulting in 70 articles classified according to the metrics presented in the tables. III. R ESULT XAI research has exponentially expanded during the past five years. Fig. 2 on the next page illustrates the research progress based on the number of published articles. The initial inquiry is to find articles devoted to XAI results in 6122 articles. Explainability metrics are used to evaluate the quality of the explainer or model for investigating how good the expla- nation is. A limited number of articles in the literature include evaluation metrics for measuring explainability. Around 884 Fig. 2. The initial queries for XAI literature resulted in 6122 articles. The graph also indicates an upward trend in XAI research. Fig. 3. After the first screening for explainability evaluation metrics, the literature review revealed that there were less than half a thousand research articles containing explanation metrics in 2022. However, there is an upward trend in the use of explanation metrics in research. articles include the evaluation inquiry in the title, abstract, or keywords. Fig. 3 illustrates the distribution of other results during the past five years. Table III on the next page includes many studies on Explainable Artificial Intelligence (XAI) written between 2021 and 2023. The authors have looked into various XAI-related ideas, methods, and applications. The papers span multiple topics, including sentiment analysis, cybersecurity, healthcare, education, and industry. The authors used deep learning, knowledge distillation, statistical testing, and other techniques to create more accessible models for people to understand and use. They have suggested assessment techniques, metrics, and algorithms to measure how trustworthy and compre- hensible AI models are. To increase the interpretability of black-box models, the authors have also leveraged various XAI tools, including SHAP, LIME, and LEAF. In general, the contributions of the authors seek to improve human-AI collaboration and address the difficulties of implementing AI systems. As an AI application becomes ubiquitous, the demand for explainability also grows. To address this, researchers have proposed different evaluation metrics to measure the quality of explanations produced by AI models. According to Chinu and Bansal [21], the explanation metric is crucial for assessing the effectiveness of answers in reliefapplication submissions. According to a study of Schwalbe and Finzel [13], applications based on explainable techniques and their matrices are gaining popularity. For example, Machlev et al. [27] emphasizes the significance of explainability metrics for power experts in understanding power quality distribution classification to guarantee reliable decision-making. The im- portance of explanation and its evaluation metric in locating anomalies in IoT data for enhancing security was stated by [28]. Vilone and Longo [41] discovered that despite the sub- stantial body of knowledge created regarding explainability, academics still need to have a general agreement regarding how to define an explanation, and its validity and reliability can be evaluated. In addition, Li et al. [32] note that although many works have contributed to this line, the present endeavor lacks a clear taxonomy and systematic review. To address this, Mualla et al. [33], Li et al. [42] propose new explanation techniques and use the metric used in LIME to evaluate the quality of the explanation. Meanwhile, Palatnik de Sousa et al. [34] argue that per- formance metrics achieved by AI models can give users the impression that there is no bias. Hence, explaining classifica- tion and evaluating the explanation based on proper metrics is necessary. Additionally, Amparore et al. [35] addresses the problem of identifying a clear and unambiguous set of metrics for evaluating Local Linear Explanations. They also propose a LEAF framework for explanation evaluation to end-users. Finally, for practical medical applications, Theunissen and Browning [29] suggests that metrics for evaluating post-hoc explanations are necessary. The metrics should evaluate the accuracy of the explanation, and there should be procedures for auditing the system to prevent biases and failures from going unaddressed. In summary, various researchers have proposed different metrics and frameworks to evaluate the quality of explanations produced by AI models. While there is still no consensus on how to define and evaluate explanations and explainability metrics’ importance in understanding AI models and ensuring trustworthy decision-making cannot be overstated. In our recent review of application-related research, we have identified that the evaluation technique is not the sole focus of interest but rather the explanation method itself. We found that in many cases, explanation evaluation was only qualitatively assessed, and the quality of the explanation was taken for granted without using any specific evaluation technique. How- ever, several terminologies were reintroduced, such as local explanation, attribute, post-hoc explanation, sensitivity, trust- worthiness, causal interpretation, traceability, and auditing. We discovered that sensitivity measurement was used frequently in the literature. This method is closely related to the taxonomy in Fig. III on page 5. The sensitivity measurement evaluates the impact of input features on the model’s output, which helps to identify the most critical features. It allows us to understand the contribution of each input feature to the model’s prediction and to evaluate the explanation’s quality. However, other terminologies, such as trustworthiness, causal interpretation, traceability, and auditing, can provide additional insights into TABLE I THIS TABLE PROVIDES A SUMMARY OF THE MOST RELEVANT LITERATURE THAT APPLIED XAI TECHNIQUES IN SOLVING PROBLEMS WITH REAL -WORLD DATA . IT HIGHLIGHTS THE RESEARCH THAT HAS UTILIZED XAI TECHNIQUES AND THEIR PRACTICAL APPLICATIONS . Authors Year Source title Concept Application Chinu and Bansal [21] 2023 New Generation Computing Explainable AI: To Reveal the Logic of Black-Box ModelsInterpretable; Transparency; Quality metrics; Schwalbe and Finzel [22] 2023 Data Mining and Knowledge DiscoveryA taxonomy for XAI methods XAI; Interpretability; Meta-analysis Xi et al. [23] 2023 Biomedical Signal Processing and ControlXAI Evaluation metric: Traceability rateDrug recommendation; Explainability; Traceability Kadir et al. [24] 2023 IUI'23 Companion Explaining Machine Learning Model ExplanationsInterpretability; GUI for Explanation Melo et al. [25] 2022 Education Sciences XAI methods evaluation metricEducational data science; Learning analytics Mi et al. [26] 2022 Computers in Biology and MedicineKDE-GAN: A multimodal medical image-fusion model knowledge distillation and explainable AIXAI in Medicine; Image generation Machlev et al. [27] 2022 IEEE Transactions on Industrial InformaticsMeasuring Explainability and Trustworthiness of Power Quality Disturbances ClassifiersXAI in Power; Power quality disturbances (PQDs) Khan et al. [28] 2022 IEEE Internet of Things Journal A New Explainable Deep Learning Framework for Cyber Threat DiscoveryAnomaly detection; IIoT; industrial networks Theunissen and Browning [29] 2022 Ethics and Information Technology Putting explainable AI in context: institutional explanations for medical AIAI and health; Epistemic risk; Ethical design Ferraro et al. [30] 2022 Artificial Intelligence Review Evaluating eXplainable artificial intelligence tools for hard disk drive predictive maintenancePredictive maintenance Sarpietro et al. [31] 2022 IEEE Access Explainable Deep Learning System for Advanced Silicon and Silicon Carbide Electrical Wafer Defect Map AssessmentExplainable architectures; Hierarchical clustering Li et al. [32] 2020 IEEE Transactions on Knowledge and Data EngineeringA Survey of Data-Driven and Knowledge-Aware eXplainable AIKnowledge-base; Mualla et al. [33] 2022 Artificial Intelligence A human-agent architecture for explanation formulationHCI; Multi-agent systems Vilone and Longo [12] 2021 Information Fusion Notions of explainability and evaluation approaches for explainable artificial intelligenceEvaluation methods; Notions of explainability Palatnik de Sousa et al. [34] 2021 Sensors Explainable artificial intelligence for bias detectionComputerised Tomography Amparore et al. [35] 2021 PeerJ Computer Science To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methodsLocal linear explanation; Machine Learning Auditing Karn et al. [36] 2021 IEEE Transactions on Parallel and Distributed SystemsCryptomining Detection in Container Clouds Using System Calls and Explainable Machine LearningAnomaly detection; Cryptomining; explainability Lobner et al. [37] 2021 IEEE Access Explainable Machine Learning for Default Privacy Setting PredictionPrivacy preference Hartmann et al. [38] 2022 Springer link Explaining AI with Narratives Explainability, NL Hartmann et al. [39] 2021 Trustworthy AI in the wild Interaction with Explanations User interaction Hartmann et al. [38] 2022 ACL A survey on improving NLP models with human explanationsUser interaction Biswas et al. [40] 2020 Springer Link Explanatory Interactive Image CaptioningImage captioning the explanation’s reliability and usability. An alternative taxonomy is proposed in 5. In our recentreview of application-related research, we have identified that the evaluation technique is not the sole focus of interest but Local explantionRetraining Input perturbation Ground truth comparisonSensitivityROAR [43] Remove feature Add featureROAD[44]Infidelity [45] IROF [46]Sensitivity-N [47] Correlation [48] PPs [49] Region perturbation [50] Pixel flipping [51] Faithfulness correlation[52] Explicitness, Faithfulness, & Stability [53] AIC & SIC [54] Non-sensitivity [55] Weakly supervise localization SSR [56]DistanceCLEVR-AI [57]CEM [58] Pointing Game [59]Average drop, increase confidence, win[60] Fig. 4. Proposed taxonomy based the methodologies of the explainability evaluation rather the explanation method itself. We found that in many cases, explanation evaluation was only qualitatively assessed, and the quality of the explanation was taken for granted without using any specific evaluation technique. However, several terminologies were reintroduced, such as local expla- nation, attribute, post-hoc explanation, sensitivity, trustwor- thiness, causal interpretation, traceability, and auditing. We discovered that sensitivity measurement was used frequently in the literature. This method is closely related to the taxonomy in 5. The sensitivity measurement evaluates the impact of input features on the model’s output, which helps to identify the most critical features. It allows us to understand the contribution of each input feature to the model’s prediction and to evaluate the explanation’s quality. However, other terminologies, such as trustworthiness, causal interpretation, traceability, and auditing, can provide additional insights into the explanation’s reliability and usability. In Table III on the previous page, we found that local explanation is necessary for plenty of applications. A local explanation can be defined as an explanation that we get individual basis based on each decision the model makes. They can be post-hoc and generated after deploying a machine-XAI [2] Interpretablity [13]Transparency [21]Auditing [35]Traceability [23]User satisfaction [61]Sensitivity [45]Local Explanation [62]Attribution [17] Causal interpretation [63] Dependent feature [63] Saliency [64] Fig. 5. Proposed taxonomy based on the XAI applications learning model. Evaluation of local explanation can be done in three ways. After removing the relevant feature from the dataset based on the explanation, they retrained a proxy model after evaluating the new model’s performance on untouched test data. Suppose the test accuracy of the newly created model is lower than the original model's accuracy. In that case, training on data with missing features creates an entirely different model than the original model. It signifies that the futures removed from training data contribute to the original model's decision. This method has high computational demand due to the retraining process. The second approach is ground-truth-based evaluation, and the explanation is compared with the ground-truth explanation data. Different distance metrics are used to identify how far the explanation is from the ground truth. Ground truth can also be user feedback on a model’s local explanation [57, 65]. Some researchers used weakly supervised localization techniques to see how a saliency-based explanation can be meaningful to localize an object in an image [56, 60]. They proposed some metrics called SSR, Point Game, Average drop, Increase confi- dence, and Win. Kapishnikov et al. [54], Rguibi et al. [66] used Accuracy Information Curves (AICs), Softmax Information Curves (SICs), and Performance Information Curves PICs XAI evaluation. [67, 68, 69] has used the area under perturbation curve (AOPC) for understanding the decisions of CNN using MoRF curve and evaluates the explainability of their proposed model. Recently, Veldhuis et al. [70] leveraged explainable AI methods for DNA analysis. Xi et al. [23], Apicella et al. [71, 72], Schinle et al. [73] reported their experiment results with MoRF curve or its variations as reliable evaluation metrics. There has been tremendous interest in unsupervised tech- niques for evaluating explanations in the last decade. Most of these methods work based on removing or adding information from the input data and measuring the changes in the output of the mode. SIC and AIC scores [54] Non-sensitivity [55] scores are measured based on the output of the model. When data is fed to an input, the output scores represent the influence of essential and nonimportant features in the model output. Similarly, removing features from input data also influences the model. Sensitivity-N can measure the influence [74], and Faithfulness Correlation [52]. The feature removal from the input is a tricky process, and the feature removal should have the property of missingness [75]. Such algorithms are also proposed by [46, 67]. A. Sensitivity analysis Explanation sensitivity refers to how much a machine learning model’s output is affected by different types of explanations or interpretability methods applied to it. In other words, it measures how much the output of a model changes when different explanations are provided for it. Sensitivity analysis is a key part of explainable AI and helps researchers and practitioners understand how reliable and robust the ex- planations of machine learning models are. Table II on the following page represents the sensitivity analysis methods usedfor the evaluation of the XAI methods. The definition of classic explanation sensitivity [45] can be expressed as follows: For anyj∈ {1, ..., d}, [∇xϕ(f(x))]j= lim ϵ→0ϕ(f(x+ϵej))−ϕ(f(x)) ϵ(1) where ej∈Rdis the jthcoordinate basis vector, with jthentry one and all others zero. and all others zero. It quantifies how the explanation changes as the input is varied infinitesimally Where fis the model, ϕis the explainer and ejis the changes in the input features. Table II on the next page includes various research articles that employ Explainable Artificial Intelligence (XAI) methods in different applications. Sensitivity Analysis is one of the XAI methods used to analyze the impact of input features on the model's output. Some of the applications include Covid-19 diagnosis, self-driving cars, brain-computer interface systems, seismic facies classification, predicting the functional impact of gene variants, discovering bias in structured pattern clas- sification datasets, smart agriculture, compression, feature se- lection, volcano detection, optical water types, feature impor- tance analysis, threat detection, survival analysis, and COVID- 19 screening using chest X-ray images. The XAI methods employed in these applications include LIME, SHAP, Multi- Objective Sensitivity Pruning, Graph embedding, Grad-CAM, Gaussian processes, Hierarchical Interpretable models, Attack trees, Bayesian networks, and Grad-CAM++. They employed an explanation evaluation technique for evaluating the output of the explanation methods. For example, Kim and Joe [77] used sensitivity analysis for evaluating explanations in self- driving cars' decision-making process. In anomaly detection explanation, sensitivity can be used to evaluate the model's decision [93]. B. Faithfulness Correlation and Faithfulness Estimate metrics Faithfulness correlation measures the linear relationship be- tween the model predictions and the training data. It quantifies how well the model can capture the patterns and relationships in the training data. A high faithfulness correlation indicates that the model faithfully detects patterns and information in the training data. In contrast, a low faithfulness correlation indicates that the model may be over-fitting or under-fitting the data. Faithfulness Correlation [52] iteratively replaces a random subset of given attributions with a baseline value. Then it measures the correlation between the attribution subset and the difference in function output. On the other hand, Faithfulness Estimate [53] computes the correlation between probability drops and attribution scores on various points. Table III on page 9 summarizes XAI studies, including the Faithfulness Correlation and Faithfulness Estimate metrics. According to [52], the faithfulness of an explanation function gto a predictor fat a point xwith a subset size of |S|is defined as follows: µF(f, g;x) =corr S∈([d] |S|)ÅX i∈Sg(f, x)i, f(x)−f(x[xs=ˆxs]ã (2) TABLE II SENSITIVITY ANALYSIS FOR XAI METHODS Reference Year Source title XAI method Application Sharma and Mishra [76] 2022 Pattern Recognition Covid-MANet Sensitivity analysis; Lesion localisation Kim and Joe [77] 2022 PLoS ONE An XAI method for convolutional neural networksSelf driving car; CNN; Sensitivity of features Ieracitano et al. [78] 2022 Neural Computing and ApplicationsA novel explainable machine learning approach for EEG-based brain-computer interface systemsBrain–computer interface; Explainable machine learning Lubo-Robles et al. [79] 2022 Interpretation Quantifying the sensitivity of seismic facies classification to seismic attribute selectionSensitivity of seismic attributes; Seismic geomorphology Phul et al. [80] 2022 PLoS Computational Biology Predicting the functional impact of KCNQ1 variants with artificial neural networksProtein structure N´apoles and Koutsoviti Koumeri [81] 2022 Pattern Recognition Letters Discover bias in features of structured pattern classification datasetsUnderstanding bias; Fairness Cartolano et al. [82] 2022 IEEE 8th International Conference on Multimedia Big DataExplainable AI at Work! What Can It Do for Smart Agriculture?Explainability in Agriculture data Sabih et al. [83] 2022 IEEE International Green and Sustainable ComputingMOSP: Pruning of Deep Neural NetworksNeural network compression Taskin [84] 2022 IEEE Geoscience and Remote Sensing LettersA Feature Selection Method via Graph Embedding and Global Sensitivity AnalysisFeature engineering Beker et al. [85] 2022 International Geoscience and Remote Sensing SymposiumExplainability Analysis of CNN in Detection of V olcanic Deformation SignalV olcanic Deformation analysis Blix et al. [86] 2022 IEEE Geoscience and Remote Sensing LettersLearning Relevant Features of Optical Water TypesUnderstanding See water Chen et al. [87] 2021 IEEE Sensors Journal Deep belief network framework and its application for feature importance analysisFeature engineering Apicella et al. [72] 2021 CEUR Workshop Proceedings Explanations in terms of Hierarchically organised Middle Level FeaturesFeature understanding Wu et al. [88] 2021 USENIX Security Symposium Adversarial policy training against deep reinforcement learningPreventing adversarial attacks Hoyt and Owen [89] 2021 SIAM-ASA Journal on Uncertainty QuantificationEfficient Estimation of the ANOV A Mean Dimension, with an Application to Neural Net ClassificationDimensionality reduction Pappaterra and Flammini [90] 2021 Studies in Computational IntelligenceBayesian Networks for Online Cybersecurity Threat DetectionThreat detection and analysis Kovalev and Utkin [91] 2020 Neural Networks A robust algorithm for explaining unreliable machine learning survival modelsReducing data demand Lee et al. [92] 2020 Journal of Personalized MedicineEvaluation of scalability and degree of fine-tuningMedical imaging; Low training data Kauffmann et al. [93] 2020 Pattern Recognition A deep Taylor decomposition of one-class modelsOutlier detection; Unsupervised learning Molnar et al. [63] 2020 Communications in Computer and Information ScienceInterpretable Machine Learning A Brief HistoryDependent features; Causal interpretation Mathews [94] 2019 Advances in Intelligent Systems and ComputingXAI Applications in NLP; Biomedical ClassificationDrawback of blackbox model dis the dimension of x. Table III on the next page lists various studies and research papers that showcase the application of faithfulness metrics in explainable AI (XAI). Faithfulness is one of the essential metrics used to evaluate the performance of XAI methods. It measures how well an AI model’s explanations align with its underlying decision-making processes. For instance, in the medical image analysis study by Jin et al. [14], the authors proposed guidelines to evaluate the faithfulness of clinical XAI models. Similarly, the G-LIME method introduced by Li et al. [42] aims to provide interpretable deep learning by ensuring the faithfulness of local interpretations of deep neural networks using global priors. Other studies in the table that uti- lize faithfulness metrics include those in autonomous driving and natural language processing. These studies illustrate the significance of faithfulness in XAI and its application across different domains. C. Monotonicity Metric Monotonicity Metric introduced by Luss et al. [49] gener- ates contrastive explanations with monotonic attribute func- tions. Arya et al. [48] further elaborates on these metrics. It starts from a reference baseline to incrementally place each feature on the baseline surface from a sorted attribution vector, measuring the effect on model performance. Recently Monotonicity Metric has been employed by several studies [112, 113, 114, 115]. D. Pixel Flipping Pixel Flipping [51] captures the impact of perturbing pixels in descending order according to the attributed value on the classification score. Wullenweber et al. [116], Pitroda et al. [117] used Pixel Flipping metric for evaluating explanations for the predictions of COVID-19 cough classifiers and lung disease classification. dk(p) =P N∈digits (k)N(p) PM i=0P N∈digits (i)N(p)(3) dk(p)is the effect of pixel pon model corresponds to class k.N(p)is the models output probability. E. Region Perturbation Region Perturbation introduced by Aopc Samek et al. [50] is an extension of Pixel-Flipping to flip an area rather than a single pixel. It has been used in several XAI experiments. Table 3 summarises XAI studies, including Region Perturba- tion. Region perturbation metric gives Area Under Perturbation which defines by the following equation. AOPC =1 L+ 1≠LX k=0fx(0) MoRF−fx(k) MoRF∑ p(x)(4) Where fis tthe model, Lis the number of samples, ⟨.⟩p(x denotes the average over all samples and x(k) MoRF is the cumu- lative removal of up to kthMost Relevant Feature (MoRF). Singla et al. [118] propose a counterfactual approach to explain black-box models used for chest X-ray diagnosis. Jinet al. [14] discuss generating post-hoc explanations from deep neural networks for multi-modal medical image analysis tasks. ˇSimi ´c et al. [119] introduce a perturbation effect metric to counter misleading validation of feature attribution methods in deep learning for time-series data. Huang et al. [121] focus on understanding spatiotemporal prediction models, while Narteni et al. [122] study the sensitivity of logic learning machines in safety-critical systems. [123] propose an integrated gradient- optimized saliency method for explainable AI in medical imaging. In contrast, [124] provide a general overview of explainable AI for process mining with a focus on a novel local explanation approach. Mishra et al. [125] discuss reliable local explanations for machine listening, and Lenis et al. [126] introduce domain-aware medical image classifier interpretation by counterfactual impact analysis. Finally, Fong and Vedaldi [127] explain deep neural network predictions for computer vision tasks without giving detain on the evaluation of ex- planation. Most of the papers used pixel flipping or variants of it to evaluate the local explanations. Table IV on page 10 presents the list of papers that have mentioned pixel flipping technique in their papers. F . Selectivity Selectivity [128] in a metrics for evaluation used in several recent XAI models, which measures how quickly a prediction function starts to drop when removing features with the highest attributed values. Goswami et al. [129], Vangala et al. [130], Kim and Park [131], Feldmann and Bajorath [132], Wang et al. [133]. It can be calculated using the AOPC curve or pixel flipping curve. G. Sensitivity-N Sensitivity-N [47] computes the correlation between the sum of the attributions and the variation in the target output while varying the fraction of the total number of features and averages it over several test samples. This metric had been recently used by [134, 135]. For a number of features nin data, selectivity-n defines the sum of the attributionsPN i=1Rc i(x)and variation in the target output correlates on a particular task for different explanation algorithms. H. IROF IROF introduced by Rieger and Hansen [46] computes the area over the curve per class for sorted mean importance of feature segments (superpixels) as they are iteratively removed (and prediction scores are collected), averaged over several test samples. Fel et al. [136] elaborate on the model explainability using IROF. They investigate how good the explanation is by evaluating algorithmic stability measures. IROF (ej) =1 NNX n=1AOCÅF(Xl n)y F(Xl 0)yãL l=0(5) I. Infidelity Infidelity is an evaluation metric introduced by [45]. It represents the expected mean square error between 1) a dot TABLE III FAITHFULNESS CORRELATION AND FAITHFULNESS ESTIMATE METRICS IN XAI