Faithfulness and the Notion of Adversarial Sensitivity in NLP Explanations

Authors: supriya  manna
niladri  sett
Venue: EMNLP_2024_Workshop_BlackBoxNLP
Published: 2024-08-15 17:59:59

Abstract: Faithfulness is arguably the most critical metric to assess the reliability of explainable AI. In NLP, current methods for faithfulness evaluation are fraught with discrepancies and biases, often failing to capture the true reasoning of models. We introduce Adversarial Sensitivity as a novel approach to faithfulness evaluation, focusing on the explainer's response when the model is under adversarial attack. Our method accounts for the faithfulness of explainers by capturing sensitivity to adversarial input changes. This work addresses significant limitations in existing evaluation techniques, and furthermore, quantifies faithfulness from a crucial yet underexplored paradigm.

Main paper: 
 introduction: 
 Deep learning-based Language Models (LMs) are increasingly used in high-stakes Natural Language Processing (NLP) tasks (Minaee et al., 2021; Samant et al., 2022) . However, these models are extremely opaque. To build user trust in these models' decisions, various post-hoc explanation methods (Madsen et al., 2022) have been proposed (Jacovi et al., 2021) . Despite their popularity, these explainers are frequently criticized for their 'faithfulness', which is loosely defined as how well the explainer reflects the underlying reasoning of the model (Lyu et al., 2024; Jacovi and Goldberg, 2020) . In the context of NLP, explainers assign weights to each token indicating their importance in prediction, and faithfulness is measured by how consistent these assignments are with the model's reasoning. However, since the explainer is not the model itself (Rudin, 2019) , practitioners have developed several heuristics to measure the quality of these assignments (DeYoung et al., 2019; Zhou et al., 2022a; Nguyen, 2018; Jain and Wallace, 2019; Hooker et al., 2019; Lyu et al., 2024) .A common assumption behind many of these heuristics is the linearity assumption, which posits that the importance of each token is independent of the others (Jacovi and Goldberg, 2020) . Based on this, a group of practitioners hypothesised that removing important tokens indicated by a faithful explainer should change the prediction, whereas removing the least important ones should not. Jacovi et al. (Jacovi and Goldberg, 2020) addressed these as erasure. DeYoung et al. (DeYoung et al., 2019) generalize the same with comprehensiveness and sufficiency. However, it has been exhaustively shown that the removal of features can produce counterfactual inputs1 that are out of distribution (Hase et al., 2021; Chrysostomou and Aletras, 2022; Lyu et al., 2024; Janzing et al., 2020; Haug et al., 2021; Chang et al., 2018) , socially misaligned (Jacovi and Goldberg, 2021) , and often severely pathogenic (Feng et al., 2018) . Furthermore, evaluation metrics such as Area Under the Perturbation Curve (AUPC) (Samek et al., 2016) are suspected to be severely misinformative (Ju et al., 2021) . Instead of evaluating faithfulness, these methods primarily compute the similarity between the evaluation metric and explanation techniques, assuming the evaluation metric itself to be the ground truth (Ju et al., 2021) .Another line of work, known as adversarial robustness (Baniecki and Biecek, 2024) , assumes that similar inputs with similar outputs should yield similar explanations. However, Ju et al. (Ju et al., 2021) has empirically shown that the change in attribution scores may be because the model's reasoning process has genuinely changed, rather than because the attribution method is unreliable. Moreover, this assumption is mainly valid when the model is 'astute' (Bhattacharjee and Chaudhuri, 2020; Khan et al., 2024) and doesn't necessarily apply to explainers that don't perform local function approximation for feature importance estimation (Han et al., 2022) . As a result, this assumption is practically restrictive and vague, leading practitioners to hesitate in endorsing this approach for assessing faithfulness (Lyu et al., 2024) .Across almost all popular lines of thought, the settings in which faithfulness is quantified are linear (Jacovi and Goldberg, 2020), restrictive (Khan et al., 2024) , misinformative (Ju et al., 2021) , and thus the judgements on explainer quality based on such quantification could be arguable. Since, understanding the model's reasoning is challenging, and aforementioned assumptions are often deceptive, in this work, we take a fundamental approach. Previous research has demonstrated that deep models are not only opaque but also severely fragile (Goodfellow et al., 2014; Szegedy et al., 2013) . As explainers are primarily to facilitate trust on these complex models, we argue that a faithful explainer is obligated to uncover such vulnerabilities and anomalous behaviour of the model to the end user. In this context, we introduce the notion of 'adversarial sensitivity' for the explainers. We seek the most similar (semantically and/or visually) counterpart(s) from the entire input space (subjected to certain constraints) that produces a different output, aka 'adversarial examples' (Goodfellow et al., 2014) . These pairs of inputs are always bounded by a certain distance, ensuring they are sufficiently comparable. Consequently, unlike counterfactuals, these pairs are much less likely affected by abrupt semantic shifts (Lang et al., 2023) that often lead to out-of-distribution scenarios (Hendrycks and Gimpel, 2016; Sun and Li, 2022; Sun et al., 2021; Liang et al., 2017) , making our comparisons more nuanced and robust. However, since these pairs yield different outputs, their underlying reasoning in the model is bound to differ (Jacovi and Goldberg, 2020; Adebayo et al., 2018) . Faithful explanations should reflect these changes, highlighting the difference in the model's inherent reasoning. We formally define the same as 'adversarial sensitivity' of the explainers. Our contributions in this paper are summarised as:• we introduce the notion of 'adversarial sensitivity' of an explainer, and propose a necessary test for faithfulness based on it;• we present a robust experimental framework to conduct the faithfulness test;• we conduct the proposed faithfulness test on six state-of-the-art post-hoc explainers over three text classification datasets, and report its (in)consistency with popular erasure based tests.This paper is organised as follows: We introduce the notion of adversarial sensitivity, exploring its significance and relation with faithfulness in Section 2. In Section 3, we details our methodology, outlining the framework used to conduct our investigations. In Section 5, we present our findings, offering in-depth analysis and interpretations of the data. We contextualize our work within the broader research landscape in Section 6, highlighting how our study contributes to and extends existing knowledge. Finally, in Section 7, we conclude by summarizing our key findings and proposing directions for future research, emphasizing the potential avenues for further exploration. 
adversarial sensitivity: 
 In this section, we introduce the notion of adversarial sensitivity, exploring its significance and relation with faithfulness. Thereafter, we propose the guideline for evaluating faithfulness with adversarial sensitivity.Definition 1. Adversarial Example (AE): Given a model f : X → Y , where X is the space of textual inputs and Y is the set of classes, if there exists x ′ for a given input x ∈ X such that:{x ′ ∈ X | S(x, x ′ ) ≥ θ and f (x) ̸ = f (x ′ )},we call x ′ an adversarial example (AE), where S(•,•) is a similarity measure and θ is a predefined similarity threshold. Definition 2. Local Explanation: A local feature importance function I takes an instance x ∈ X and the model f as input, and produces a weight vector as output:I(f, x) = W x,f = (w 1 , w 2 , . . . , w n ),where w i represents the importance of the i-th token x i for the prediction f (x). Definition 3. Adversarial Sensitivity: Adversarial Sensitivity for a local explainer I for (x, x ′ ) is given by d(W x,f , W x ′ ,f ). Here, x ′ is an AE of input x, and d(•, •)2 is a distance measure.Adversarial Sensitivity and Faithfulness: Given x ′ is an AE of x for f, if I is 'faithful' to f, then W x,f and W x ′ ,f should be dissimilar. In our setup, we report the mean distance over all obtained pairs of (x, x ′ ). This is a necessary but not sufficient condition for faithfulness. Currently (at the time of writing this paper) there is no necessary and sufficient condition for faithfulness (Lyu et al., 2024) . However, following the argument of Lyu et al. (Lyu et al., 2024) , as these metrics are primarily (meta)heuristic based evaluations, accessing faithfulness with several necessary tests is much more practical than attempting to formulate an exhaustive list of necessary and sufficient conditions and then evaluating against all of them. Adversarial Sensitivity is one of such necessary tests to evaluate the faithfulness of explainers.Adversarial Machine Learning research has extensively demonstrated that even minimal perturbations in the input space can deceive well-trained models (Alzantot et al., 2018; Garg and Ramakrishnan, 2020; Li et al., 2020; Gao et al., 2018; Ebrahimi et al., 2017; Kuleshov et al., 2018; Zang et al., 2019; Pruthi et al., 2019; Jin et al., 2020; Li et al., 2018; Ren et al., 2019) . Given the discrete and combinatorially large nature of the input space, finding all possible adversarial examples (AEs) under all possible constraints is often impractical, especially in a black-box setting. Therefore, we advocate for greedily searching for AEs within a well-tested set of constraints to avoid obfuscating and low-quality examples. In this study, we select extensively used word-level, character-level, and behavioural invariance constraints. Whether methods like back-translation, paraphrasing, or hybrid attacks etc (Zhang et al., 2020) maintain semantic and structural similarity while generating AEs, and suitability for faithfulness evaluation are kept for further study.Obtaining AEs is conducted in two ways: assuming the model to be either white-box or blackbox. In a white-box setting, gradients are primarily used first to identify the importance of tokens and then perturb them to create an adversarial input if the output changes. For our setting, this approach has two distinct problems. Firstly, gradient-based feature importance can be untrustworthy and manipulative (Wang et al., 2020; Feng et al., 2018) . Secondly, a class of post-hoc explainers (e.g., Gradient, Integrated Gradient) also uses the gradient to retrieve the importance of tokens. Comparing these with explainers that do not use gradient informa-tion, such as LIME or SHAP, may lead to biased comparisons. Lastly, popular gradient-based attacks such as HotFlip (Ebrahimi et al., 2017) are often less likely to adhere to perturbation constraints while crafting adversarial examples (Wang et al., 2020) . Therefore, we do not consider investigation on white-box attacks for adversarial sensitivity and adhere to a more practical, model-agnostic, and transferable black-box attacking framework. However, even in the black-box settings we employ some ad-hoc heuristics for greedily perturbing the words based on its relative importance (Zhang et al., 2020) , but modern explainers do not use such ad-hoc methods for calculating feature importance (Lyu et al., 2024; Madsen et al., 2022) . Therefore, our faithfulness test is unbiased towards the underlying mechanisms of (almost) all types of modern post-hoc explainers.3 Faithfulness Test Setup 
obtaining aes: 
 Primarily, obtaining AEs (an adversarial attack on the model) is a greedy or brute-force procedure, where a search algorithm iteratively selects locally optimal constrained perturbations until the label changes (Morris et al., 2020) . As mentioned in Section 2, we devise our attacks in three constraint classes: word level, character level, and behaviorul invarince. We brief the implementation details of these attacks as follows.We adhere to the constraints proposed in the strong baseline 'TextFooler' (Jin et al., 2020) while implementing our word-level attack (A1). Initially, we assign weights to each word based on its impact on the model's prediction when removed. Then, in decreasing order of importance, we take each word (except stopwords), find semantically and grammatically correct K (we set K = 50) words to replace the selected word, and generate all possible intermediate corpus and query the model. If the best result (which alters the prediction the most) from this pool exceeds the one from the previous iteration, we select the new one as the current result; otherwise, we stick to the previous one. This process iterates until the current result yields a different output or we have exhaustively searched the set of possible results and found none that alter the output. Although the constraints, including vocabulary se-lection and stopwords filtering, were effective in crafting adversarial examples, we observed some discrepancies with off-the-shelf hyperparameter selections. Consequently, we adjusted the minimum word embedding cosine similarity to 0.5 (instead of 0.7) and set an angular similarity threshold of 0.84 within a 15-token window.3 For character-level attack (A2), we assign weights to each word based on its impact on the model's prediction when replaced with an unknown token ('[UNK]'). The rest of the procedure is the same as the A1, but instead of semantically similar words, we replace the selected word after applying a combination of character-level perturbations proposed by Gao et al. (Gao et al., 2018) , subject to a predefined edit distance threshold, proposed in (Gao et al., 2018) . Li et al. (Li et al., 2018) empirically showed that character-level perturbation can change semantic alignment in the embedding space. Therefore, after filtering with edit distance, we also employ the universal sentence encoder (Cer et al., 2018) and use the similarity threshold proposed in (Li et al., 2018) to select the final candidate.Recently, Ribeiro et al. (Ribeiro et al., 2020) emphasised that models are hypersensitive not only to minute perturbations but also to 'invariant' tokens. Ribeiro et al. proposed 'Checklist' that evaluates models across diverse linguistic capabilities such as vocabulary, syntax, semantics, and pragmatics. For our setting, we adopt the 'Invariance Testing' they proposed (A3). We change names, locations, numbers, etc., wherever feasible in the sentences and check if these alterations affect the prediction. As Ribeiro et al. (Ribeiro et al., 2020) showed, a model should not be sensitive to such parameters. If it is, it indicates an inability to handle commonly used linguistic phenomena, which are subsequently characterised as a type of adversarial example (Morris et al., 2020) . We employ the off-the-shelf implementation of the invariance testing from 'TextAttack' (Morris et al., 2020) . In our datasets, we do not have a lot of instances where phone numbers, locations, age etc are present and as we are changing these only once in this attack (else it could lead to an infinite loop), the success rate of this attack is lesser than the other attacks. However, from a linguistic perspective, this attack is crucial to make our experiments exhaustive. In all these attacks, we do not perturb stop-words. Next, we only consider the example as successful AE if the prediction confidence crosses a certain threshold (we set it to be at least 70%). Finally, as we are conducting model-agnostic attacks, we acknowledge that even if the constraints are reasonably restrictive, there is always a chance that any of these examples could be out-of-distribution (OOD). To mitigate such issues, we follow a robust baseline wherever required for detecting OOD scenarios by computing the 'maximum/predicted class probability' (MCP) from a softmax distribution for the predicted class of each AE (Hendrycks and Gimpel, 2016) . MCP has been evaluated as a strong baseline, particularly when the underlying model is fine-tuned (e.g., BERT, RoBERTa) (Hendrycks et al., 2020; Desai and Durrett, 2020) . We empirically selected only those adversarial examples that had a probability exceeding 70% across all attacks and datasets.To measure the dissimilarity of the explanations, we follow the distance measure given by Ivankay et al. (Ivankay et al., 2022) , that is:d = 1 - τ (W x,f , W x ′ ,f ) + 1 2 (1)where τ (•, •) is a correlation measure. Ivankay et al. (Ivankay et al., 2022) chose Pearson correlation for their distance measure. But while creating adversarial examples, a common phenomenon is obtaining unequal token vectors for (x, x ′ ) due to tokenisation (Sinha et al., 2021) . Correlation measures like Pearson, Kendall, and Spearman cannot handle disjoint and unequal ranked lists. Sinha et al. (Sinha et al., 2021) used heuristics like Location of Mass (LOM) (Ghorbani et al., 2019) to mitigate such issues. But Burger et al. (Burger et al., 2023) highlighted their shortcomings and employed Rank Based Overlap (RBO) (Webber et al., 2010) metric. While RBO may be robust, it introduces complications, particularly with its selection of free parameter 'p' determining the user persistence.4 Moreover, the assumption on the depth in RBO using Bernoulli's random variable and weights of overlaps in explanation using geometric distribution may not be always adequate as per our setting. Furthermore, the selection between the base and extrapolated versions of RBO gives rise to the disparity in 'sensitivity', especially when the residual is significant (Webber et al., 2010) . Following the arguments of Jacovi et al. (Jacovi and Goldberg, 2020) we, do not endorse unnecessary human intervention in faithfulness studies. As RBO inherently carries the notion of the persistence of users, we didn't select RBO for this work.We have extensively investigated selecting the similarity measures in previous works, but none of the works has tackled the problem of unequal and/or disjoint rank lists from an axiomatic perspective that will be adequate for our setting. Emond et al. (Emond and Mason, 2002) proposed a new correlation coefficient designed to accommodate incomplete and non-strict rankings; however, this metric is not considered due to the lack of formal proof or empirical evidence. Later, Monero et al. (Moreno-Centeno and Escobedo, 2016) introduced essential axioms for a distance measure between incomplete rankings, establishing the existence and uniqueness of such a measure and demonstrating its superiority in generating intuitive consensus rankings compared to alternative methods. Following these axioms, we adopt the nonparametric correlation coefficient 'τ x ' presented in Yoo et al. (Yoo et al., 2020) , which highlights the inadequacy of the τ x ranking correlation coefficient devised in (Emond and Mason, 2002) in ensuring a neutral treatment of incomplete rankings. Moreover, our employed non-parametric correlation coefficient 'τ x ' is a generalization of Kendall τ on the aforementioned axiomatic foundation established by Monero et al. (Moreno-Centeno and Escobedo, 2016) for handling a variety of ranking inputs, including incomplete and non-strict ones. Therefore, τx is foundationally much robust and can handle several types of tokenization discrepancies. Furthermore, this very distance is a nonparametric generalization of the kemeny-snell distance (Kemeny and Snell, 1962) for nonstrict, incomplete ranking space (Moreno-Centeno and Escobedo, 2016). As a result, unlike the previous distance metric, 'τ x ' is not only robust but also enjoys the properties that the Kemeny-snell distance retains for all types of rankings produced by the tokenisers.Our proposed test is a necessary test for faithfulness based on the desideratum that the explainers should produce different explanations for AEs. Obtaining AEs is always subject to different sets of constraints. As a result, each attack type i.e. A1, A2, A3 is disjoint in nature thus, each of them independently conducts a necessary test given they produce successful AEs. Theoretically, there can be finitely many AEs if we keep changing the set of constraints but in this paper, we followed three extensively evaluated, diverse sets of constraints to empirically demonstrate the adversarial sensitivity of explainers around these disjoint constraint sets. As a result, our setup consists of three disjoint necessary tests for inspecting faithfulness using adversarial sensitivity. We evaluate the explainers on the basis of how much sensitivity they obtain for how many number of discrete constraint sets. However, as these are all necessary tests, the primary objective is to reject the unfaithful ones. Also, it is highly seek-worthy that explainers perform consistently well across constraint sets. Now, if the results across constraint sets are fluctuating for a given setup, it could be confusing for the end user to evaluate the explainers holistically. This is why, for an aggregated ranking we recommend using a consensus aggregation (e.g., Kemeny-young aggregation (Kemeny, 1959)) over empirical evaluation. Although, in our experiments, we obtained consistent results across A1, A2, A3. 
datasets and models: 
 We conducted our experiments on SST-2 (Socher et al., 2013) , and Tweet-Eval (Hate) (Barbieri et al., 2020) for binary classification, and on AG News (Zhang et al., 2015) for multi-class classification. We fine-tuned a Distill BERT and a BERTbased model (Devlin et al., 2018) until it achieved a certain level of accuracy for each dataset, and attacked it with the three attack methods A1, A2, and A3 described in the Section 3.1. We report the models' accuracy before and after each attack5 in Table 1 . We've addressed 'Tweet-Eval (Hate)' as 'Twitter' throughout the paper and Distill BERTbased model (Sanh, 2019) as DERT in Table 1 . We used the standard train, test split for each dataset from the huggingface library and reported results up to the second decimal place.Commonly used post-hoc local explainers can be broadly categorised in two types: perturbationbased and gradient-based explainers (Madsen et al., 2022) We run our experiments on an NVIDIA DGX workstation, leveraging Tesla V100 32GB GPUs. We use ferret with default (hyper)parameter selection (Attanasio et al., 2022) for both erasure metrics and explanation methods, TextAttack (Morris et al., 2020) , universal sentence encoder (Cer et al., 2018) across attacking mechanism. We wrote all experiments in Python 3.10. Our total computational time to execute all experiments is roughly 18 hours. We report the consolidated findings for both models below in Table 2 . 
results & discussion: 
 From Table 2 , it is clearly observable that as per Adv. Sens., LIME, SHAP, Gradient × Input, and Integrated Gradient × Input all perform competitively across various datasets and attacks. However, the vanilla versions of gradient-based methods are not as effective. Notably, the Gradient itself exhibits the least sensitivity to adversarial inputs, followed by Integrated Gradient. Furthermore, Integrated Gradient's adv. sens. remains almost invariant to the type of attacks across all datasets, unlike comp. and suff. Interestingly, all explainers except Gradient show a drop in sensitivity in 6 partition shap documentation: the AG News dataset across all attacks. Gradient performs best on all attacks in AG News amongst datasets. Perturbation-based explainers like LIME and SHAP are among the best performers across datasets. Gradient × Input and Integrated Gradient × Input perform well within the group of white-box explainers, with LIME and SHAP.Under erasure methods across all datasets, Gradient is a moderately well-performing explainer, whereas Gradient × Input performs much worse. However, according to Adversarial Sensitivity, Gradient × Input is one of the best performers, with Gradient being the worst among all. Like Gradient × Input, Integrated Gradient also largely performs worse than Gradient in erasure, but it remains consistently moderate according to Adv. Sens. Both LIME and SHAP not only perform very well in both Adv. Sens. and erasure metrics but also the difference b/w their magnitudes for both erasure metrics and adv. sens. are (considerably) nominal. Integrated Gradient × Input is substantially similar to LIME, SHAP in adv sens., but we observe a considerable drop in comprehensiveness for SST-2 and AG News for both the models, unlike adv. sens.To demonstrate, how to evaluate the explainers based on the consensus ranking, we are considering the case of SST-2 for the BERT Model. We use the Kemeny-Young method here; as this has been extensively used for Condorcet ranking (Young, 1988) ; it also satisfies highly desirable social choice properties for fair voting (Owen and Grofman, 1986; Young, 1995) . Kemeny-Young aggregation also have been used in biology and social science extensively (Brancotte et al., 2014; Andrieu et al., 2021; Arrow et al., 2010) . We first convert the columns of A1, A2, A3 into ranking vectors using a ranking function. In our case, we used the traditional ranking: the higher the score (here the score is average distance obtained), the lower the ranking. We obtained the consensus ranking vector as [2, 3, 6, 5, 4, 1] . Here, the indices of the vector denote the respective position of explainers (starting from 1 onwards) in the 'Explainer' column.DeYoung et al. (DeYoung et al., 2019) advocated for both high comprehensiveness and low sufficiency for adequate explanations but unlike us; they did not propose any consensus evaluation for explainers with these two parameters taken together. According to the definition, both metrics measure two different aspects of explanations. This makes the evaluation of explainers even confusing (Jacovi and Goldberg, 2020) reported the same observation previously. As Javoci et al. said, "Lacking a standard definition, different works evaluate their methods by introducing tests to measure properties that they believe good interpretations should satisfy. Some of these tests measure aspects of faithfulness. These ad-hoc definitions are often unique to each paper and inconsistent with each other, making it hard to find commonalities." (Jacovi and Goldberg, 2020) . Although evaluation metrics are inherently different from one another, for the sake of demonstrating an inter-comparison between erasures and adv. sens. 7 , we rank the explainers based on the scores they obtain in individual erasure methods in the case of SST-2 for the BERT Model in Table 2 . We consider the same ranking function used for adv. sens. for Comprehensiveness and LOO score and the inverse of the same ranking function for Sufficiency due to its opposite nature with respect to the former. First, we take the Kemeny-Young aggregation of comprehensiveness and sufficiency; the 7 we do not necessarily endorse this rank-based comparison as an axiomatic comparison in the presence of different type of faithfulness evaluation parameters but a (hard) estimate in the absence of such comparisons.ranking obtained is [1, 2, 4, 6, 5, 3 ]. LOO's ranking is: [1, 2, 3, 6, 5, 4] . Next, we combine all erasure columns and get the aggregation as [1, 2, 4, 6, 5, 3] . The obtained aggregated ranking for adv. sens. was [2, 3, 6, 5, 4, 1] . From this comparison, we retrieve all explainers have obtained different rankings for comprehensiveness-sufficiency, LOO, and combined aggregation of erasures, as compared with adv sens. Throughout our experiments for both models, we observed explainers except for LIME & SHAP (as mentioned earlier) are largely inconsistent with one or more erasure method(s).Nevertheless, erasure has been used in several novel affairs and benchmarkings (Mathew et al., 2021; Atanasova et al., 2023; Liu et al., 2022; Babiker et al., 2023) due to its easy-to-implement and seemingly reasonable assumption. However, we observe in our experimentation that erasure methods are inconsistent except perturbation based explainers with our proposed metric. Unlike erasure, which makes simplistic assumptions about the independence of the token's importance and absence of non-sensical OOD results while removing tokens (Lyu et al., 2024) , adversarial sensitivity is founded on the assumption that faithful explainers should capture the intrinsic dissimilarity of model reasoning when fooled. We, therefore, advocate for the adoption of adversarial sensitivity as a foundational metric for a necessary test of faithfulness for assessing explainers.Faithfulness evaluation, based on previous literature, can be broadly categorised in six ways: axiomatic evaluation, predictive power evaluation, robustness evaluation, perturbation-based evaluation, white-box evaluation, and human perception evaluation (Lyu et al., 2024) . The commonly used erasure is primarily a perturbation-based evaluation: it hypothesised that the change in model's output caused by the removal tokens is proportional to the importance of the tokens for the prediction. If a local explainer is faithful, removal of important tokens as identified by the explainer should align with the hypothesis. Comp., Suff., LOO are different instances of the erasure hypothesis. Our hypothesis is also somewhat related to the perturbation-based evaluation. We hypothesised that a faithful explainer should be sensitive to anomalous input that fools the model. We perturb at several levels in the input to deceive the model, not to interpret. Next, we evaluate how much the explainer is sensitive towards the subtle changes that deceive the model. As the deep models are known to be severely fragile, we argue this is a necessary quality for the explainer to be faithful when the model is not showing its expected behaviour.Following the hypothesis of adversarial robustness, which comes under the robustness evaluation category, successful adversarial attack on explainer aims to perturb the input such that an explainer generates dissimilar (non-robust) explanations subject to 'similar' input and 'similar' (bounded by a certain distance) output (Baniecki and Biecek, 2024 ) (AdvxAI). However, rather than any ad-hoc distance to compare the similarity of explainer Alvarez et al. (Alvarez-Melis and Jaakkola, 2018) emphasize on the (local) lipschitz continuity measurement in this setting. Khan et al. (Khan et al., 2024) has recently analysed the theoretical bounds of (dis)similarity under this setting when the explainer and classifier (Bhattacharjee and Chaudhuri, 2020) are astute. Anyways, Ad-vxAI is not a formally accepted measure of faithfulness (Ju et al., 2021; Zhou et al., 2022b; Lyu et al., 2024) , as the model may yield different reasoning rather than the explanation is non-robust. Anyhow, in this work we conduct attacks to deceive the model, not the explainer following the aforesaid hypothesis. For a broad overview on faithfulness evaluation we suggest the reader to refer to (Lyu et al., 2024) .Adversarial examples (AdvAI) can be crafted at several levels: word level, character level, phrase level, paraphrasing, back translation, invariance testing, etc. in white-box and black-box settings primarily (Zhang et al., 2020) . We employed word level, char level and invarience testing attacks. Noppel et al. (Noppel and Wressnegger, 2023) systematised the underlying relations of AdvAI and Ad-vxAI. For a broder overview, we refer the reader to (Qiu et al., 2022; Zhang et al., 2020) . Adversarial attacks on NLP systems have been carried out primarily in 2 types: white box and black box (Zhang et al., 2020) , we didn't go with white box ones as they primarily leverage gradient information also, as several explainers such as Integrated Gradient or Gradient access the same information which could constitute a biased evaluation (Ju et al., 2021) as the attacking mechanism and the explanation method are similar and both leverage gradient information.Counterfactual explanations (Mothilal et al., 2020) , which demonstrate the changes would produce a distinct outcome, differ fundamentally from adversarial examples (Freiesleben, 2022) , which aim to deceive models with minimal input changes. Counterfactuals should be semantically and/or visually different (Yang et al., 2020) . Thus, it is not intended to deceive the underlying model. In the context of Natural Language Inference (NLI), Atanasova et al. (Atanasova et al., 2023) experimented with counterfactuals to investigate faithfulness. Camburu et al. (Camburu et al., 2019) explored inconsistencies in explanations for NLI but did not adhere to the constraints necessary for generating adversarial inputs required in our setting. Moreover, counterfactual explanations can potentially highlight necessary features but may miss sufficient ones for prediction (Hsieh et al., 2020) .Similarity measures in previous works, especially in AdvxAI (Sinha et al., 2021; Burger et al., 2023; Ivankay et al., 2022) , have used mainly correlations, distance measures, and top 'k'% intersection in tokens. Burger et al. (Burger et al., 2023) comprehended the common issues with such metrics due to tokenization discrepancies and employed RBO (Webber et al., 2010) . We did not select RBO having the free parameter user persistence (p), as we argue that faithfulness should not be based on the unnecessary human evaluations. We rather select the distance invented by Moreno et al. (Moreno-Centeno and Escobedo, 2016 ) that satisfies all the axioms for non-strict, incomplete rank-ings and also satisfies the desirable social choice properties of the Kemeny-snell distance (Kemeny and Snell, 1962) for fair and conclusive rankings. 
conclusion and further work: 
 In this work, we explored the shortcomings of widely used faithfulness measures in NLP and proposed a test to evaluate explainers based on their sensitivity to adversarial inputs. Through extensive experiments on six post-hoc explainers, we found that gradient & integrated gradient aren't (sufficiently) sensitive, while LIME, SHAP, and Gradient × Input, and Integrated Gradient × Input show better sensitivity. We also observed notable differences between our evaluation and traditional erasure-based faithfulness measures.Future work will explore adversarial sensitivity for multilingual datasets, low-resource languages, and advanced lms. 
 
 Appendix: Deep models are not only fragile but also opaque. Our work lies at the intersection of these two critical aspects. Building on the arguments presented by Jacovi et al. (Jacovi and Goldberg, 2020) , we introduce a necessary test for assessing faithfulness. Given that the underlying assumption of adversarial sensitivity is applicable to (nearly) all data types and models, this concept can be extended across (almost) all domains and explanation mechanisms.Faithfulness is a key component in explainable AI (Miller, 2019) . When a model behaves deceptively under any form of adversarial intervention, it becomes imperative that explainers provide faithful explanations in such scenarios, rather than merely those where the model performs according to user expectations. Adversarial sensitivity aids end-users in identifying explainers that are responsive to adversarial instances. We strongly believe that the nuanced notion of adversarial sensitivity opens up a new direction for evaluating explainers, particularly in situations where being unfaithful could lead to a misinterpretation of why the model produces deceptive results.Adversarial attacks are computationally expensive. Our work therefore is much computationally expensive and non-trivial than erasures. Our work is a necessary test faithfulness of explainers therefore, from a practitioner's perspective (Lyu et al., 2024) we employ our tests primarily to identify unfaithful explainers. It's important to note that our test does not take into account other criteria, such as biases in models, during the evaluation process. The scope of the work, for the time being, is restricted to NLP.Counterfactual inputs (CI) & counterfactual explanations (CE) are completely different. Removing features from the main input makes CI wrt the actual input.Miller et al. used this terminology(Miller,2019). We've discussed CE in Section 6. Hase et al.(Hase et al., 2021) debunked the same confusion of the reviewers here.in details at SectionWe discovered that the authors of TextAttack(Morris et al., 2020) identified bugs in the original implementation of TextFooler(Jin et al., 2020) and suggested a set of hyperparameters that were mostly coherent in our setup. Details can be found here.Burger et al. (Burger et al., 2023) used LIME's feature importance along with explanation's average length to determine the value of 'p' for their experimentation and Goren et al.(Goren et al., 2018) apparently used an ad-hoc value of p = .7 in their experimental setup.If AE is not obtained, the attack is failed and vice-versa. 


Decision: This paper presents a novel approach for evaluating explanation faithfulness through adversarial sensitivity, which utilizes adversarial examples to assess the stability of explanations. Reviewers agree that the paper is well-motivated and structured, with comprehensive experiments. However, the identified weaknesses, such as issues with paper presentation, clarity, and experimental design, should be addressed in the camera-ready version.

Reviews: review: paper_summary: paper_summary: This work tackles the difficult problem of evaluating explanation faithfulness for instance wise feature attribution explanations on classification tasks. It introduces a novel evaluation approach called adversarial sensitivity that uses adversarial examples, hence examples in the input space that are close to the original input vector, with respect to some distance metric, but generate a different prediction. The evaluation metric is based on the assumption that adversarial examples, should have a high degree of dissimilarity since a different prediction is being produced. Additionally, the authors suggest generating adversarial examples using only a selected number of perturbations to the input example to avoid out of distribution examples, which is what prior work is often accused of doing. The authors empirically compare the results of their evaluation metric for different kinds of perturbations with existing faithfulness metrics.
strength_weakness: strength_weakness: I do not fully agree with the definition of adversarial sensitivity. It relies on the assumption that if two input examples are similar with respect to S but produce different outcomes then the model uses different input features for the prediction. Yet, this does not need to be true. See below for examples. 

On the same line of argumentation (again see below for details) I think the framing of this work should change slightly, away from faithfulness evaluation, but rather define adversarial stability of explanations as a way to analyse the model. 

Some notation and design choices are not defined making reproducibility of this work questionable. 

Even though there is a long paragraph discussing the choice of similarity measure between different explanations, I feel like the authors are somewhat missing an important point: A similarity metric only makes sense if we know what attribution values relate to each other. For more details on this see below. 

Overall, result and discussion section should could be structured better. Some of the results are not discussed sufficiently or carefully enough. Also some choices in the evaluation should be reconsidered.
questions: questions: Statement made in line 44: Since erasure suggests also removing sets of features rather than only individual features, the claim that this is based on the assumption that the importance of tokens is linear. 

It would be good to explicitly state what model f is in Definition 1 (It seems like this work assumes that f is a classification model) 

In general, a more careful definition of notation as well as assumptions would help the reader to get a more concrete understanding. Like what does it mean that f(x)\ne f(x')? For this one class or is this somehow defined on class probability of all classes? 

Definition 3 (Adversarial Sensitivity) of line 167 does not seem to give us a well defined notion of adversarial sensitivity. Since there is not one unique Adversarial Example (AE) but a set of possible AEs, as defined here lambda could take lots of different values. I suggest defining it as the maximum, mean or some other way of aggregating the distances over all AEs (for some similarity threshold), and then explicitly approximating it through sampling. 

I do not fully agree with the definition of adversarial sensitivity. It relies on the assumption that if two input examples are similar with respect to S but produce different outcomes then the model uses different input features for the prediction. Yet, this does not need to be true. As an example consider the adversarial example having changed one token, let's say exchanging words like "elevator and lift" or "mom and mother", which are semantically very similar. The model might rely on the same word, but consider small nuances in formal/ informal language or different local accents to generate different model predictions. Yet, the correct attribution could still look exactly the same, namely assigning a high degree of importance on "elevator" and lift respectively, even though model prediction based on this input change has changed. Hence, while I agree that adversarial sensitivity is an interesting thing to evaluate and analyse, the framing of this way of evaluating explanations as faithfulness evaluation seems incorrect. 

"more" missing in 183

S is nowhere explicitly defined. 

The authors mention in line 318 that they consider an AE successful if the prediction confidence crosses a threshold of 70%. Yet, this seems to depend strongly on the task, definition of S, number of classes and should probably also be dependent on the prediction confidence of the model for the original example. 

On the choice of similarity measure between explanations: The authors discuss that, since the input is perturbed in different ways, the length of the explanation vectors might also change. The discussion of defining the right similarity measure between original and perturbed examples then focuses entirely on the possibly unequal length of those explanation vectors. Yet, the authors seem to miss a more important point: Even if we aim to use a similarity metric between incomplete rankings, there still needs to be a 1 on 1 relationship between different entries in the vectors. I would argue, we should not search for a metric that can take vectors of different length as an input since a one-to-one correspondence of the tokens in the perturbed and original example can not be ensured (like one nth token being perturbed into two, making the explanation vector longer but also causing the attribution values corresponding to the nth+1 token now to appear at the nth+2nd position in the explanation vector). Since the goal of the distance metric is to measure how much the attribution values of certain tokens has changed, we should rather try to define something like a one-to-one correspondence, potentially including one-to-two correspondences (or n-to-m correspondences) where a semantically similar substitution has happened defining a extension of existing 1-to-1 based similarity measures for those correspondences.  

Line 416 might have a typo since "infinitely" would make more sense here. 

Table 1: Results should be reported until the last significant decimal, rather than up to a random decimal. Also if you are reporting up to a decimal, make sure even values with zeros as decimals will be reported that way. (like 25.00 for AG News A1 Acc in table 1) 

Table 1 needs more discussion in the text. Why are attack 1 and 2 decreasing accuracy this much? Is this maybe a sign that model has overfitted too much to the data? Why is the accuracy after attack 3 nearly perfect? This seems odd to me, since further analysis is based on these models and attacks, more discussion would be in place here. 

A big part of the Results and Discussion sections describes how a ranking of explainers is determined for each evaluation metric and how then a total ranking is determined. This should be part of the experimental setup section rather than the results as it interrupts the reading flow. Also, often combining several evaluation metrics into one is not desirable, since it is strongly dependent on use case which property on an explanation should be weighted

The framing of these tests as necessary conditions (which usually then would be considered to be a binary test) is not consistent with the use of the proposed tests as evaluation metrics with a score rather than a fail/pass of a necessary condition. 

Evaluating comprehensiveness in steps of 10%, 20%, 30%,... can be rather coarse dependent on the input size. Usually we want compact explanations, I would suggest evaluating more on smaller explanation sizes. Explanations of size >30% or so of the input size do not seem to be very useful anymore and considering those in the calculated average for the final score might out-weight the actual relevant explanation sizes, especially for comprehensiveness since for higher explanation sizes the comprehensiveness scores will be much higher. The results shown in the table seem to support my criticism on the way the reported scores are being averaged over the top10%, 20%,... of features. Comp scores are very high overall pointing to too large explanation size. Similarly, Sufficiency scores are mostly very low, sometimes even 0 or negative. The analysis could benefit from using different evaluation steps.
paper_summary: paper_summary: The paper introduces a faithfulness test for model explainers that measures the explainers sensitivity to adversarial inputs. The measurement of sensitivity is based on a distance measure using Pearson correlation. The paper includes experiments testing multiple explainers using BERT and Distal BERT models.
strength_weakness: strength_weakness: The experiments are conducted with two very similar models BERT and Distill BERT. It would have been interesting to see how the approach works with other models.
questions: questions: Reference for Distill BERT missing.
paper_summary: paper_summary: 1. The paper introduces a faithfulness test to evaluate various explainers based on their sensitivity to adversarial inputs
2. They conduct tests on 6 post-hoc explainers over three text classification datasets and provide a thorough analysis of the results
3. They find perturbation-based methods are more sensitive to AE than those vanilla gradient-based methods.
strength_weakness: strength_weakness: 1. Overall, the paper is well-written, but having one example/figure to introduce adversarial examples and how we expect the explainer to behave would be very helpful for readers to understand this paper.
2. Simplify unnecessary mathematical notations.  For example, the author uses $\lambda$ to denote adversarial sensitivity in Definition 3, while it has never been used in later sections. The method of this paper is quite intuitive, and simplifying and merging those equations would help improve its readability.
questions: questions: See summary of weakness