Year: 2021

Abstract:
State-of-the-art text classification models are becoming increasingly reliant on deep neural networks (DNNs). Due to their black-box nature, faithful and robust explanation methods need to accompany classifiers for deployment in reallife scenarios. However, it has been shown in vision applications that explanation methods are susceptible to local, imperceptible perturbations that can significantly alter the explanations without changing the predicted classes. We show here that the existence of such perturbations extends to text classifiers as well. Specifically, we introduce TEXTEXPLANATIONFOOLER (TEF), a novel explanation attack algorithm that alters text input samples imperceptibly so that the outcome of widely-used explanation methods changes considerably while leaving classifier predictions unchanged. We evaluate the performance of the attribution robustness estimation performance in TEF on five sequence classification datasets, utilizing three DNN architectures and three transformer architectures for each dataset. TEF can significantly decrease the correlation between unchanged and perturbed input attributions, which shows that all models and explanation methods are susceptible to TEF perturbations. Moreover, we evaluate how the perturbations transfer to other model architectures and attribution methods, and show that TEF perturbations are also effective in scenarios where the target model and explanation method are unknown. Finally, we introduce a semi-universal attack that is able to compute fast, computationally light perturbations with no knowledge of the attacked classifier nor explanation method. Overall, our work shows that explanations in text classifiers are very fragile and users need to carefully address their robustness before relying on them in critical applications.

text: State-of-the-art text classification models are becoming increasingly reliant on deep neural networks (DNNs). Due to their black-box nature, faithful and robust explanation methods need to accompany classifiers for deployment in reallife scenarios. However, it has been shown in vision applications that explanation methods are susceptible to local, imperceptible perturbations that can significantly alter the explanations without changing the predicted classes. We show here that the existence of such perturbations extends to text classifiers as well. Specifically, we introduce TEXTEXPLANATIONFOOLER (TEF), a novel explanation attack algorithm that alters text input samples imperceptibly so that the outcome of widely-used explanation methods changes considerably while leaving classifier predictions unchanged. We evaluate the performance of the attribution robustness estimation performance in TEF on five sequence classification datasets, utilizing three DNN architectures and three transformer architectures for each dataset. TEF can significantly decrease the correlation between unchanged and perturbed input attributions, which shows that all models and explanation methods are susceptible to TEF perturbations. Moreover, we evaluate how the perturbations transfer to other model architectures and attribution methods, and show that TEF perturbations are also effective in scenarios where the target model and explanation method are unknown. Finally, we introduce a semi-universal attack that is able to compute fast, computationally light perturbations with no knowledge of the attacked classifier nor explanation method. Overall, our work shows that explanations in text classifiers are very fragile and users need to carefully address their robustness before relying on them in critical applications.

heading: 1 INTRODUCTION
text: Deep neural networks (DNNs) have undoubtedly become the state-of-the-art architectures for many existing machine learning tasks (Choi et al., 2016). Yet, their black-box nature has raised the need for developing methods to mitigate the lack of interpretability caused by their increased complexity (Simonyan et al., 2013; Zeiler & Fergus, 2014; Hendricks et al., 2018; Bahdanau et al., 2014). These methods give intuitive, easily understandable explanations that do not require significant domain knowledge. This is especially desired in safety-critical scenarios, such as healthcare, where the users of such DNNs - the medical professionals for instance - need to understand the decision process and reasoning behind it. However, they have been shown to lack local robustness towards carefully crafted, imperceptible perturbations in the input (Ghorbani et al., 2019). While resulting in the same predictions, these altered inputs yield significantly different explanations and attributions maps (Figure 1). Interpretation methods fragile towards small input perturbations not only fail to provide faithful explanations, a desiderata commonly required in explainable AI (Jacovi & Goldberg, 2020), but also damages user trust in DNNs and prevents them from being deployed on high-stakes, safetycritical applications, such as in healthcare (Adadi & Berrada, 2020). The previously described phenomenon has been widely studied in the image domain by Etmann et al. (2019); Moosavi-Dezfooli et al. (2019b) or Ivankay et al. (2020). However, in discrete-input domains like text, there has been limited progress. This is especially problematic given the increased reliance on and fragility of attention mechanisms as inherently explainable methods, as stated in Ghaeini et al. (2018). Therefore, we summarize our contributions as follows:
• We provide a novel baseline black-box adversarial attack, TEXTEXPLANATIONFOOLER (TEF) to estimate the local robustness of explanations in text classification problems
• We evaluate attribution robustness on widely used, state-of-the-art text datasets and model architectures, showing that explanation methods’ output can be significantly altered with our attack
• We provide insight into the transfer capability of TEF on different models and explanation methods as well as introduce semi-universal adversarial perturbations to alter explanations without requiring access to the model at attack-time

heading: 2 PRELIMINARIES

heading: 2.1 RELATED WORK
text: Adversarial attacks that alter the inference outcomes in DNNs have been widely studied both in the image (Goodfellow et al., 2014; Carlini & Wagner, 2017; Moosavi-Dezfooli et al., 2016; Modas et al., 2019) and text domain (Ebrahimi et al., 2017; Sun et al., 2020; Jin et al., 2019; Yang et al., 2020). Methods to alleviate the networks susceptibility to such attacks have also been proposed, including the works of Madry et al. (2017); Moosavi-Dezfooli et al. (2019a); Buckman et al. (2018) or Cisse et al. (2017). However, it has recently been shown by authors Ghorbani et al. (2019) that, in addition to DNN predictions, widely-used explanation methods also lack robustness to targeted, imperceptible alterations of the input. These attacks change the outcomes of such explanation methods significantly, while predictions of the DNNs are unaltered. This violates the Prediction Assumption of faithful explanations and crucially degrades user trust in such explanation methods, as significantly different interpretations are provided for similar inputs and outputs (Jacovi & Goldberg, 2020). The aforementioned phenomenon of fragile explanations has mostly been investigated in the image domain (Etmann et al., 2019; Singh et al., 2019; Chen et al., 2019; Ivankay et al., 2020), with less focus on discrete input spaces like text. However, faithful and robust interpretations are arguably equally important in the discrete text domain, for instance in electronic health record classification (Girardi et al., 2018) or precision medicine (Binder et al., 2021), where critical decisions often need to be based on DNN explanations. The work of La Malfa et al. (2021) constructs inherently robust explanations for NLP models, however only towards perturbations in the embedding space, not the input space that adversaries can operate on. Moreover, they do not give a method to evaluate robustness of already existing explanation algorithms. The authors of Feng et al. (2018) show that
interpretation methods in NLP lack completeness (Sundararajan et al., 2017) by removing words deemed least important by explanation methods. Moreover, attention mechanisms (Bahdanau et al., 2014) have been increasingly relied on as inherently interpretable systems. However recent work has questioned their faithfulness and plausibility (Serrano & Smith, 2019; Jain & Wallace, 2019; Wiegreffe & Pinter, 2019) by proving that they often do not highlight input components that are most important to a DNN decision. We, paralleled by the very recent work of Sinha et al. (2021), are the first to show that imperceptible perturbations in the input space can alter the outcome of explanations of text classifiers significantly, giving an efficient attack to estimate explanation robustness. However, our work is the first to give an extensive evaluation of the robustness of widely-used explanation methods on large datasets, comparing several state-of-the-art architectures. Further, we are the first to address robustness of attention weights in transformer architectures and to provide insight on transfer capabilities and universal attacks.

heading: 2.2 BACKGROUND
text: Let S = {s1, s2, ..., sN} be a dataset of N text samples si, each with a label from a predefined set of labels L = {l1, l2, ..., l|L|}. Each sample si contains a sequence of tokens (or words) wi taken from a discrete vocabulary set W = {w1, w2, ..., w|W|}. A generic sequence classifier then consist of a non-injective, non-surjective embedding function E : S ! Rd⇥p, E(s) = X , which maps the input sample s to its embedding matrix X , and a function f : Rd⇥p ! R|L|, f(X) = o, representing a (DNN) classifier function. d and p denote the embedding dimension and sequence length respectively. Let F : S ! R|L|, F (s) = f E be the full sequence classifier with final prediction y = argmaxi2{1:|L|}oi. We define an attribution map as A : S ! Rp, A(s, F, l) = a that assigns a scalar value to each input token wi in the text sample s, resulting in the attribution vector a 2 Rp. This vector represents each token’s influence towards the prediction outcome y of classifier F . Our work considers three widely-used attribution methods in text classification, namely Saliency Maps (S) (Simonyan et al., 2013), Integrated Gradients (IG) (Sundararajan et al., 2017) and Attention (A) (Bahdanau et al., 2014), defined in the following Equations (1), (2) and (3) respectively.
ASi (s, F, l) = X
j2{1:d}
|rXf(X)l|j,i (1)
AIGi (s, F, l,B) = X
j2{1:d}
⇥ (X B) ·
Z 1
↵=0 rX̃f(X̃)l|X̃=B+↵(X B) d↵
⇤ j,i
(2)
AAtti (s, F, l) = exp eiP
j2{1:p} exp ej (3)
where B denotes the null matrix 0d⇥p, f is the classifier function of F , ↵ a scaling factor and X = E(s). rXf denotes the matrix-derivative of f to X , as defined in Goodfellow et al. (2016). An attention head is a layer that transforms its inputs into scores e and calculates its output by linear combination of each input score, with coefficients normalized to a distribution. These coefficients are the attention weights AAtti (s, F, l) denoted in Equation (3). It is commonly agreed to give intuitive explanations on how much the model attends to the given inputs through its attention weights (Jacovi & Goldberg, 2020).

heading: 3 METHODS
text: In this section, we describe our novel method TEXTEXPLANATIONFOOLER (TEF) to estimate attribution robustness (AR) in sequence classification problems. Specifically, we define the problem formulation, introduce our threat model and present the algorithm used to alter explanations by imperceptibly changing the inputs.

heading: 3.1 PROBLEM FORMULATION
text: Given an input text samples s and s̃, labels l; a text classifier F with embedding function E and classifier function f ; and attribution method A, we define attribution robustness (also explanation
robustness, AR) as written in Equation (4).
r(ã,a) = 1 max ã d(ã,a) = 1 max s̃
d ⇥ A(s̃, F, l), A(s, F, l) ⇤ (4)
with argmax i2{1:|L|} F (s̃) = argmax i2{1:|L|} F (s), (5)
where d denotes a distance measure between the attributions ã and a of the the two input samples s and s̃. The rest of the notation is kept as in Section 2. Equation (4) quantifies how different the attributions of two input samples are, given the constraint in Equation (5) that enforces the inputs having the same prediction outcome. The attribution robustness estimation is then solved utilizing the following Equation (6).
sadv = argmax s̃
d ⇥ A(s̃, F, l), A(s, F, l) ⇤ (6)
where sadv denotes the solution to the estimation, i.e. the adversarial input. s denotes the original, unperturbed input and s̃ the perturbed input, optimized during estimation. The solution sadv gives a robustness estimate by minimizing r defined in Equation (4) within a local neighbourhood of s. This neighbourhood is defined by both linguistic constraints described in the next section that encourage semantic proximity to the original text and the perturbed samples having the same prediction outcome as the unperturbed ones, see Equation (5). This formulation is backed by current research (Ghorbani et al., 2019; Dombrowski et al., 2019; Ivankay et al., 2020) and the Prediction Assumption of faithful explanations (Jacovi & Goldberg, 2020).

heading: 3.2 THREAT MODEL AND ATTACK
text: Algorithm 1 TextExplanationFooler (TEF) Input: Input sentence s with predicted class l, classifier F , attribution A, attribution distance d, number of synonyms N , maximum perturbation ratio ⇢max Output: Adversarial sentence sadv
1: sadv s, dmax 0, r 0 2: for wi 2 s do 3: Iwi = d ⇥ A(swi!0, F, l), A(s, F, l) ⇤ 4: for wj 2 hw1, ..., w|s|i | Iwm 1 Iwm 8m 2 {2, ..., |s|} do 5: if wj 2 SStop words then 6: continue 7: Cj SynonymEmbeddings(wj , N) 8: Cj POSFilter(wj ,Cj , s) 9: for ck 2 Cj do 10: s̃wj!ck Replace token wj in sadv with ck 11: if argmax i2{1:|L|} F (s̃wj!ck) = l then 12: d̃ d ⇥ A(s̃wi!ck , F, l), A(s, F, l) ⇤
13: if d̃ > dmax then 14: sadv s̃wi!ck 15: dmax d̃ 16: r r + 1 17: if ⇢ = r|s| + 1 > ⇢max then 18: break
We define our algorithm to estimate AR as a black-box attack. It only queries the model to obtain its output logits and the accompanied explanations of the inference process. The model might access its gradients to compute explanations, but the attack only utilizes the resulting explanations, no gradient or architectural information. We restrict the valid input perturbations to token substitutions, specifically insertions and deletions of tokens are forbidden, as they alter the input lengths. Algorithm 1 contains the schematic code for TEF, consisting of the following two steps.
Step 1 - Word importance ranking First, an importance ranking is extracted for each token of the input sample. Specifically, we compute Iwi = d ⇥ A(swi!0, F, l), A(s, F, l) ⇤ for each token i in s, where swi!0 denotes the input sequence s with the i-th word masked to the zero embedding token. The input tokens are then sorted by the Iwi values in a decreasing fashion. Then, high importance words are prioritized during substitution.
Step 2 - Candidate selection For each word wi in s sequentially, a set of substitution candidates C of N elements is extracted. This candidate set is constructed from the counter-fitted GloVe (Pennington et al., 2014) synonym embeddings by the authors of Mrkšić et al. (2016). The candidates are then filtered by Part-Of-Speech (POS), tagged by SpaCy (Honnibal et al., 2020), only allowing replacements with equal POS. Stop words are also discarded from C. A final selection as replacement
for wi is then made to be the ck 2 C that maximizes d ⇥ A(s̃wi!ck , F, l), A(s, F, l) ⇤ . The algorithm is aborted when the number of replacements to sentence length exceeds the maximum value ⇢max.

heading: 4 EXPERIMENTS AND RESULTS
text: In this section, we present an extensive evaluation of our attribution robustness (AR) estimation attack, TEF, for sequence classification problems. We examine the performance of TEF and study the impact of different factors on its robustness evaluation performance. We find that our attack effectively reduces the correlation of original and attacked attributions on all datasets and models. Moreover, we describe our transfer and semi-universal attacks and examine their robustness estimation performance, showing that even under circumstances where the model and explainer are unknown to the attacker, TEF perturbations transferred from other models decrease attribution robustness effectively.

heading: 4.1 MODELS, DATASETS AND EVALUATION
text: Our TEF attack is evaluated on five commonly used public sequence classification datasets, AG’s News (Zhang et al., 2015), MR reviews (Zhang et al., 2015), IMDB Movie Reviews (Maas et al., 2011), Fake News Dataset 1 and Yelp (Asghar, 2016). We train six different word embedding-based architectures for each dataset, namely a CNN, an LSTM, an LSTM containing a single attention layer with one head (LSTMAtt) and three state-of-the-art finetuned transformer-based architectures, BERT (Li et al., 2020), RoBERTa (Liu et al., 2019) and XLNet (Yang et al., 2019). Table 1 contains a summary of our model performances as well as details on the datasets. The text samples are tokenized with the default English SpaCy (Honnibal et al., 2020) tokenizer for the CNN, LSTM and LSTMAtt models and embedded with the pretrained GloVe 6B 300-dimensional word vectors (Pennington et al., 2014). The transformer-based models use their own pretrained tokenizers and embeddings. We use PyTorch (Paszke et al., 2019) with Captum (Kokhlikyan et al., 2020) to implement our models and explainers and the Huggingface Transformers library (Wolf et al., 2020) to finetune the transformer architectures on our datasets. We evaluate the robustness of three commonly used explanation methods in natural language processing with our TEF attack. These are Saliency Maps (S), Integrated Gradients (IG) and the Attention mechanism (A), defined in Section 2. We use S and IG in combination with all our architectures, Attention only with LSTMAtt, BERT, RoBERTa and XLNet. During the attack, we set the
attribution distance d of Equation (6) to be d(ã,a) = 1 PCC(ã,a) + 1 2
, with PCC denoting the Pearson Correlation Coefficient (Pearson, 1895) of original and adversarial attributions ã and a. We then report the standard Pearson Correlation Coefficient (PCC), Kendall’s Rank Order Correlation (ROC) (Kendall, 1938), Spearman’s Correlation Coefficient (SCC) (Myers & Sirois, 2004) and the Top-10%, Top-30% and Top-50% intersections to measure AR in Equation (4). These are common metrics that correspond to human measures of AR (Ghorbani et al., 2019; Dombrowski et al., 2019; Ivankay et al., 2020). Additionally, in order to quantify imperceptibility of perturbations, the semantic similarity of adversarially perturbed and unchanged sentences is reported, along with the relative increase of average perplexity of the perturbed samples, given by the GPT-2 (Radford et al., 2019) language model. Semantic similarity (Sem. sim.) is measured by the cosine distance between the
1https://www.kaggle.com/c/fake-news/data
the metrics for our RANDOMATTACK (RA). The ACC indicates the area under the PCC ( and ) curve, lower values correspond to overall lower feature attribution correlations in the overall operation interval of ⇢. The perplexity increases are indicated on the right axis, all other metrics on the left.
embeddings produced by the Universal Sentence Encoder (USE) (Cer et al., 2018). This is a stateof-the-art sentence embedding widely used in adversarial attacks on text (Sun et al., 2020; Jin et al., 2019). Perplexity increase (PPL inc.) indicates how much the likelihood of the perturbed data has decreased, given a language model, and is often used to validate language models (Keselj, 2009). Due to the lack of related work in this field, we compare the AR estimation performance of TEF to our RANDOMATTACK (RA) baseline. RA serves as an agnostic attack, utilizes a random word importance ranking in Step 1 of TEF and selects a random synonym in the final selection in Step 2. POS and stop word filters (see Section 3) are still utilized in RA to keep linguistic constraints intact.

heading: 4.2 ROBUSTNESS OF EXPLANATIONS
text: Attribution robustness estimation. In order to estimate the attribution robustness (AR) of the aforementioned models and explainers, we vary the parameter ⇢max of TEF, which denotes the maximum ratio of perturbed tokens in the input sample. A larger ⇢max value leads to lower attribution correlation, as potentially more words are substituted in the input. We then capture the aforementioned metrics PCC, ROC, SCC, Sem. sim, Top-10%/30%/50% intersections and PPL inc. to evaluate AR. Additionally, in order to quantify performance of our attack over the whole operation interval of 0  ⇢max  0.4, we compute the Area under the Pearson Correlation Curve (ACC). A lower value of ACC corresponds to lower robustness overall, as correlation values are lower. We note that a particular value of ⇢max does not guarantee that all input samples have exactly ⇢max ratio of perturbed tokens. Therefore, we quantize our samples based on their actual, resulting perturbed ratio ⇢ such that samples with similar ⇢ are grouped together. These bins are computed per dataset, ensuring the comparability of resulting curves and ACCs for each plot. Moreover, we choose the number of candidates in Step 2 of TEF to be N = |C| = 15, as it is a good trade-off between TEF estimation performance and attack run time. As expected, we find that TEF is able to significantly outperform the baseline provided by RA in terms of all AR metrics, on all datasets, models and explanation methods considered in this work. A subset of these results is shown in Figure 2, the rest can be found in Appendix A.1. Moreover, we do not find that any architecture is significantly more robust to TEF perturbations for explainers S and IG. However, the self-attention mechanism of transformers seems to be more robust to perturbations than non-transformer-based architectures and explanations. The semantic similarity decreases with increasing ⇢ and stays above 0.7 in most cases. This, together with the fact that resulting samples share predictions with the non-perturbed ones effectively highlights that the explanations given by these models and attribution methods lack faithfulness.
Ablation study. In addition to the fully random attack described in the previous paragraph, we compare TEF to our semi-random attacks RANDOMIMPORTANCE (RI) and RANDOMSYNONYM (RS). We randomize the word importance ranking of TEF (RI) but keep the selection of best final synonym, and we randomize the final synonym selection of TEF (RS) but keep the word importance ranking respectively. Figure 3 shows our findings for these experiments, along with comparisons to RANDOMATTACK (RA). The PCC curves and the ACC values show that RI consistently outperforms RS in terms of PCC over the whole operation interval of ⇢. Moreover, the impact of word importance ranking diminishes with increasing ⇢, especially for shorter datasets like MR. This can be observed by RS performing closer to RA for high ⇢ values.
BERT’s attention layers and heads. BERT’s attention weights can be used to help gain insight into a models prediction by understanding which parts of the input are most attended to (Vig, 2019). Our BERT models have 12 layers with 12 attention heads (144 heads in total), each producing a distribution of attention weights over its inputs and outputs. Estimating the AR of all heads together is not useful, as effects would average out. Therefore, we run TEF to estimate the robustness of each head separately. Figure 4 contains the average PCCs of the attention weights before and after perturbing the inputs with TEF. We find that attention weights in later layers tend to be more susceptible to input perturbations than earlier layers. Moreover, heads within a layer tend to be comparably ro-
timated PCC of TEF perturbations transferred from the indicated models and explanations. and indicate the PCC curve of optimal TEF and RA perturbations respectively, without transfer.
bust. We leave a thorough, theoretical analysis of this phenomenon to future work. We conclude that the increasing reliance on attention weights to provide inherent interpretations to BERT predictions needs careful investigation, especially in safety-critical applications.

heading: 4.3 TRANSFERABILITY AND SEMI-UNIVERSAL PERTURBATIONS
text: Transferability of perturbations to models and explanations. The adversary does not necessarily possess information about the deployec model nor the exact method to produce the accompanying explanations. Therefore, it is crucial for systems to be as resistant to transfer attacks as possible in order to evade perturbations constructed on similar models and explanations. Thus, we examine how our classifiers and attribution methods react to transfer attacks computed with TEF. We alter the input samples for a given model and explanation method with TEF, then evaluate the PCC of attributions on the same samples but different architectures and explainers. The results are found in Figure 5. We observe that transfer attacks perform better than RA, some even by approx. 0.4 in terms of average PCC decrease in the operation area of ⇢ ⇡ 0.1. However, as expected, they significantly fall short of the performance of TEF. Therefore, we conclude that transferring TEF perturbations across models and explainers effectively highlights fragility of explanations, but TEF provides tighter AR bounds without transfer.
Semi-universal perturbations. In this section, we take a step towards defining universal perturbations, similarly to the work of the authors Moosavi-Dezfooli et al. (2017) and Gao & Oates (2019). These provide fast and computationally cheap perturbations during attack time that are able to mislead classifiers with pre-computed perturbations. However, we attack the explanations of text classifiers, instead of their predictions and call our perturbations semi-universal attack policies. We split the test dataset into two equally sized parts, the attack set and the evaluation set. We utilize the former for constructing our semi-universal attack policies and the latter to evaluate how effectively our semi-universal attack alters the attributions maps of our models. First, for each sample in the attack set, we compute the optimal TEF perturbation for all our models and explainers. We then extract statistics of these perturbations, which are the most common
replacement and the replacement frequency for each replaced token and sort these by decreasing frequency. These are our semi-universal attack policies, seen in Figure 6. During this phase, we query the model for predictions and explanations, as we compute optimal TEF perturbations. Second, we evaluate our semi-universal attack that utilizes the aforementioned policies to alter explanations of classifiers. The inputs to this attack are a text sample, a semi-universal policy and a maximum perturbed ratio ⇢max. The attack iterates over the policy, starting with the token in the first row and finishing with the last. Whenever the current token is found in the input text sample, it is replaced with the replacement token in the list. If the perturbed ratio exceeds ⇢max, the attack is aborted. In such a way, perturbed inputs are created without querying the model during attack time. The actual perturbation for each text sample depends on the sample, hence the name semi-universal attack policy. The resulting samples are then evaluated on a given model and explanation method. Representative results are given in Figure 7. We conclude that our semi-universal policies are effective in reducing attribution correlation when the adversary has no access to the target model and explanation method, as indicated by the lower ACC values of the semi-universal PCC curves.

heading: 5 CONCLUSION
text: In this work, we introduced a novel black-box attack called TEXTEXPLANATIONFOOLER, that successfully perturbs input data such that the outcome of popular explanation methods in sequence classification, but not the prediction of the classifier. This attack provides a baseline estimator for attribution robustness and highlights the lack of robustness of current explanation methods. We compared it to the random attack, showing its superior performance to it on five different, widely used text classification datasets. Moreover, the transfer capabilities of the attack are evaluated. Finally, we showed the existence of semi-universal perturbation policies that are capable of altering explanations without querying the model during attack-time, even without having access to perturbations for those models. In future work, we plan to examine whether a similar white-box attack that has access to model gradients can improve robustness estimation. Moreover, instead of synonym embeddings, we plan to use BERT-based masked language models to extract possible candidates, further improving imperceptible word substitutions.

Review summaries:
This paper proposes an attacking method to attack the explanations of text classification in NLP. The proposed method is easy to follow. In particular, they first do the word importance ranking for each token of the input sample. High-important words are prioritized during substitution.  Second, candidate selection via counter-fitted GloVe is proposed to replace the word to attack explanations while maintaining the same confidence in the correctly predicted class. The authors conducted comprehensive experiments to evaluate the effectiveness of their methods.  

This paper focuses on the issue of assessing the robustness of the "explanations" for NLP models (how the explanation, not the prediction, for NLP models, change with respect to small perturbations in the input). Particularly, the authors  1. formalized the problem of attribution (explanation) robustness estimation as an optimization tasks in eqn (4, 5, 6), which involves finding the perturbations that keep the model's output the same, while maximizing the difference in attribution (explanation) compared to the case without these perturbations.  2. developed a black-box attack model that estimate AR, without requiring gradient information. The authors also further conducted ablation studies on the components of the black-box model.  3. conducted evaluations of the authors' attack model, analyzing three existing popular attribution estimation methods (saliency maps, integrated gradients and attention as explanation)  

The authors introduce a text explanation attack algorithm called “Text Explanation Fooler”, that changes output of explanation methods while keeping the classification output the same. This is done by maximizing the distance between attribution maps of original and perturbed sequences, while forcing these inputs have the same prediction outcome. In the proposed attack, the authors apply word importance ranking for substitutions, and take POS tags and stopping words into account. They evaluate their methods with AG’s News, IMDB, Fake News, MR, and Yelp datasets along with CNN, LSTM, LSTMAtt, and BERT models and report significant decrease in correlations compared to RandomAttack (RA).