Incorporating Attribution Importance for Improving Faithfulness Metrics
Year: 2023

Abstract:
Feature attribution methods (FAs) are popular approaches for providing insights into the model reasoning process of making predictions. The more faithful a FA is, the more accurately it reflects which parts of the input are more important for the prediction. Widely used faithfulness metrics, such as sufficiency and comprehensiveness use a hard erasure criterion, i.e. entirely removing or retaining the top most important tokens ranked by a given FA and observing the changes in predictive likelihood. However, this hard criterion ignores the importance of each individual token, treating them all equally for computing sufficiency and comprehensiveness. In this paper, we propose a simple yet effective soft erasure criterion. Instead of entirely removing or retaining tokens from the input, we randomly mask parts of the token vector representations proportionately to their FA importance. Extensive experiments across various natural language processing tasks and different FAs show that our soft-sufficiency and softcomprehensiveness metrics consistently prefer more faithful explanations compared to hard sufficiency and comprehensiveness.1

heading: 1 Introduction
text: Feature attribution methods (FAs) are popular posthoc explanation methods that are applied after model training to assign an importance score to each token in the input (Kindermans et al., 2016; Sundararajan et al., 2017). These scores indicate how much each token contributes to the model prediction. Typically, the top-k ranked tokens are then selected to form an explanation, i.e. rationale (DeYoung et al., 2020). However, it is an important challenge to choose a FA for a natural language processing (NLP) task at hand (Chalkidis et al., 2021; Fomicheva et al., 2022) since there is no sin-
1Our code: https://github.com/casszhao/So ftFaith
gle FA that is consistently more faithful (Atanasova et al., 2020).
To assess whether a rationale extracted with a given FA is faithful, i.e. actually reflects the true model reasoning (Jacovi and Goldberg, 2020), various faithfulness metrics have been proposed (Arras et al., 2017; Serrano and Smith, 2019; Jain and Wallace, 2019; DeYoung et al., 2020). Sufficiency and comprehensiveness (DeYoung et al., 2020), also referred to as fidelity metrics (Carton et al., 2020), are two widely used metrics which have been found to be effective in capturing rationale faithfulness (Chrysostomou and Aletras, 2021a; Chan et al., 2022). Both metrics use a hard erasure criterion for perturbing the input by entirely removing (i.e. comprehensiveness) or retaining (i.e. sufficiency) the rationale to observe changes in predictive likelihood.
However, the hard erasure criterion ignores the different importance of each individual token, treating them all equally for computing sufficiency and comprehensiveness. Moreover, the hard-perturbed input is likely to fall out of the distribution the model is trained on, leading to inaccurate measurements of faithfulness (Bastings and Filippova,
4732
2020; Yin et al., 2022; Chrysostomou and Aletras, 2022a; Zhao et al., 2022). Figure 1 shows an example of two toy FAs, A and B, identifying the same top two tokens (“like”, “movie”) as a rationale for the prediction. Still, each of them assigns different importance scores to the two tokens resulting into different rankings. According to the hard erasure criterion, comprehensiveness and sufficiency will assign the same faithfulness score to the two rationales extracted by the two FAs.
In this paper, we aim to improve sufficiency and comprehensiveness in capturing the faithfulness of a FA. We achieve this by replacing the hard token perturbation with a simple yet effective soft erasure criterion (see Figure 1 for an intuitive example). Instead of entirely removing or retaining tokens from the input, we randomly mask parts of token vector representations proportionately to their FA importance.
Our main contributions are as follows:
• We propose two new faithfulness metrics, softcomprehensiveness and soft-sufficiency that rely on soft perturbations of the input. Our metrics are more robust to distribution shifts by avoiding entirely masking whole tokens;
• We demonstrate that our metrics are consistently more effective in terms of preferring more faithful rather than unfaithful (i.e. random) FAs (Chan et al., 2022), compared to their “hard” counterparts across various NLP tasks and different FAs.
• We advocate for evaluating the faithfulness of FAs by taking into account the entire input rather than manually pre-defining rationale lengths.

heading: 2 Related Work

heading: 2.1 Feature Attribution Methods
text: A popular approach to assign token importance is by computing the gradients of the predictions with respect to the input (Kindermans et al., 2016; Shrikumar et al., 2017; Sundararajan et al., 2017). A different approach is based on making perturbations in the input or individual neurons aiming to capture their impact on later neurons (Zeiler and Fergus, 2014). In NLP, attention mechanism scores have been extensively used for assigning token importance (Jain and Wallace, 2019; Serrano and Smith, 2019; Treviso and Martins, 2020;
Chrysostomou and Aletras, 2021b). Finally, a widely used group of FA methods is based on training simpler linear meta-models to assign token importance (Ribeiro et al., 2016).
Given the large variety of approaches, it is often hard to choose an optimal FA for a given task. Previous work has demonstrated that different FAs generate inconsistent or conflicting explanations for the same model on the same input (Atanasova et al., 2020; Zhao et al., 2022).

heading: 2.2 Measuring Faithfulness
text: One standard approach to compare FAs and their rationales is faithfulness. A faithful model explanation is expected to accurately represent the true reasoning process of the model (Jacovi and Goldberg, 2020).
The majority of existing methods for quantitatively evaluating faithfulness is based on input perturbation (Nguyen, 2018; DeYoung et al., 2020; Ju et al., 2022). The main idea is to modify the input by entirely removing or retaining tokens according to their FA scores aiming to measure the difference in predictive likelihood .
Commonly-used perturbation methods include comprehensiveness, i.e. removing the rationale from the input), and sufficiency, i.e. retaining only the rationale (DeYoung et al., 2020). Another common approach is to remove a number of tokens and observe the number of times the predicted label changes, i.e. Decision Flip (Serrano and Smith, 2019). On the other hand, Monotonicity incrementally adds more important tokens while Correlation between Importance and Output Probability (CORR) continuously removes the most important tokens (Arya et al., 2021). (In)fidelity perturbs the input by dropping a number of tokens in a decreasing order of attribution scores until the prediction changes (Zafar et al., 2021). Additionally, Yin et al. (2022) proposed sensitivity and stability, which do not directly remove or keep tokens. Sensitivity adds noise to the entire rationale set aiming to find a minimum noise threshold for causing a prediction flip. Stability compares the predictions on semantically similar inputs.
One limitation of the metrics above is that they ignore the relative importance of each individual token within the selected rationale, treating all of them equally. Despite the fact that some of them might take the FA ranking into account, the relative importance is still not considered. Jacovi
and Goldberg (2020) have emphasized that faithfulness should be evaluated on a “grayscale” rather than “binary” (i.e. faithful or not) manner. However, current perturbation-based metrics, such as comprehensiveness and sufficiency, do not reflect a “grayscale” fashion as tokens are entirely removed or retained (e.g. comprehensiveness, sufficiency), or the rationale is entirely perturbed as a whole (e.g. sensitivity).

heading: 2.3 Evaluating Faithfulness Metrics
text: Quantitatively measuring the faithfulness of model explanations is an open research problem with several recent efforts focusing on highlighting the main issues of current metrics (Bastings and Filippova, 2020; Ju et al., 2022; Yin et al., 2022) and comparing their effectiveness (Chan et al., 2022).
A main challenge in comparing faithfulness metrics is that there is no access to ground truth, i.e. the true rationale for a model prediction (Jacovi and Goldberg, 2020; Ye et al., 2021; Lyu et al., 2022; Ju et al., 2022). Additionally, Ju et al. (2022) argue that it is risky to design faithfulness metrics based on the assumption that a faithful FA will generate consistent or similar explanations for similar inputs and inconsistent explanations for adversarial inputs (Alvarez-Melis and Jaakkola, 2018; Sinha et al., 2021; Yin et al., 2022).
Chan et al. (2022) introduced diagnosticity for comparing the effectiveness of faithfulness metrics. Diagnosticity measures the ability of a metric on separating random explanations (non-faithful) and non-random ones (faithful). They empirically showed that two perturbation metrics, sufficiency and comprehensiveness, are more ‘diagnostic’, i.e. effective in choosing faithful rationales compared to other metrics.
Despite the fact that sufficiency and comprehensiveness are in general more effective, they suffer from an out-of-distribution issue (Ancona et al., 2018; Bastings and Filippova, 2020; Yin et al., 2022). More specifically, the hard perturbation (i.e. entirely removing or retaining tokens) creates a discretely corrupted version of the original input which might fall out of the distribution the model was trained on. It is unlikely that the model predictions over the corrupted input sentences share the same reasoning process with the original full sentences which might be misleading for uncovering the model’s true reasoning mechanisms.

heading: 3 Faithfulness Evaluation Metrics

heading: 3.1 Sufficiency and Comprehensiveness
text: We begin by formally defining sufficiency and comprehensiveness (DeYoung et al., 2020), and their corresponding normalized versions that allow for a fairer comparison across models and tasks proposed by Carton et al. (2020).
Normalized Sufficiency (NS): Sufficiency (S) aims to capture the difference in predictive likelihood between retaining only the rationale p(ŷ|R) and the full text model p(ŷ|X). We use the normalized version:
S(X, ŷ,R) = 1−max(0, p(ŷ|X)− p(ŷ|R))
NS(X, ŷ,R) = S(X, ŷ,R)− S(X, ŷ, 0) 1− S(X, ŷ, 0)
(1)
where S(x, ŷ, 0) is the sufficiency of a baseline input (zeroed out sequence) and ŷ is the model predicted class using the full text x as input.
Normalized Comprehensiveness (NC): Comprehensiveness (C) assesses how much information the rationale holds by measuring changes in predictive likelihoods when removing the rationale p(ŷ|X\R). The normalized version is defined as:
C(X, ŷ,R) = max(0, p(ŷ|X)− p(ŷ|X\R))
NC(X, ŷ,R) = C(X, ŷ,R) 1− S(X, ŷ, 0)
(2)

heading: 3.2 Soft Nomralized Sufficiency and Comprehensiveness
text: Inspired by recent work that highlights the out-ofdistribution issues of hard input perturbation (Bastings and Filippova, 2020; Yin et al., 2022; Zhao et al., 2022), our goal is to induce to sufficiency and comprehensiveness the relative importance of all tokens determined by a given FA. For this purpose, we propose Soft Normalized Sufficiency (Soft-NS) and Soft Normalized Comprehensiveness (SoftNC) that apply a soft-erasure criterion to perturb the input.
Soft Input Perturbation: Given the vector representation of an input token, we aim to retain or remove vector elements proportionately to the token importance assigned by a FA by applying a Bernoulli distribution mask to the token embedding. Given a token vector xi from the input X and its FA score ai, we soft-perturb the input as follows:
x′i = xi ei, ei ∼ Ber(q) (3)
where Ber a Bernoulli distribution and e a binary mask vector of size n. Ber is parameterized with probability q:
q = { a, if retaining elements 1− a, if removing elements
We repeat the soft-perturbation for all token embeddings in the input to obtain x′. Our approach is a special case of dropout (Srivastava et al., 2014) on the embedding level.
Following Lakshmi Narayan et al. (2019), we have also tested two other approaches to soft perturbation in early-experimentation: (1) adding Gaussian noise to the embeddings; and (2) perturbing the attention scores, both in proportion to the FA scores. However, we found that dropout outperforms these two methods. Perhaps this is due to their sensitivity to hyperparameter tuning (e.g. standard deviation) which potentially contributes to their poor performance. Hence, we only conduct full experiments using dropout-based soft perturbation. Details on these alternative methods to perturb the input are included in Appendix C.
Soft Normalized Sufficiency (Soft-NS): The main assumption of Soft-NS is that the more important a token is, the larger number of embedding elements should be retained. On the other hand, if a token is not important most of its elements should be dropped. This way Soft-NS takes into account the complete ranking and importance scores of the FA while NS only keeps the top-k important tokens by ignoring their FA scores. We compute Soft-NS as follows:
Soft-S(X, ŷ,X′) = 1−max(0, p(ŷ|X)− p(ŷ|X′))
Soft-NS(X, ŷ,X′) = Soft-S(X, ŷ,X′)− S(X, ŷ, 0)
1− S(X, ŷ, 0) (4)
where X′ is obtained by using q = ai in Eq. 3 for each token vector x′i.
Soft Normalized Comprehensiveness (SoftNC): For Soft-NC, we assume that the more important a token is to the model prediction, the heavier the perturbation to its embedding should be. Soft-NS is computed as:
Soft-C(X, ŷ,X′) = max(0, p(ŷ|X)− p(ŷ|X′))
Soft-NC(X, ŷ,X′) = Soft-C(X, ŷ,X′) 1− S(X, ŷ, 0)
(5)
where X′ is obtained by using q = 1− ai in Eq. 3 for each token vector x′i.

heading: 4 Experimental Setup

heading: 4.1 Tasks
text: Following related work on interpretability (Jain et al., 2020; Chrysostomou and Aletras, 2022b), we experiment with the following datasets:
• SST: Binary sentiment classification into positive and negative classes (Socher et al., 2013).
• AG: News articles categorized in Science, Sports, Business, and World topics (Del Corso et al., 2005).
• Evidence Inference (Ev.Inf.): Abstract-only biomedical articles describing randomized controlled trials. The task is to infer the relationship between a given intervention and comparator with respect to an outcome (Lehman et al., 2019).
• MultiRC (M.RC): A reading comprehension task with questions having multiple correct answers that should inferred from information from multiple sentences (Khashabi et al., 2018). Following DeYoung et al. (2020) and Jain et al. (2020), we convert this to a binary classification task where each rationale/question/answer triplet forms an instance and each candidate answer is labelled as True/False.

heading: 4.2 Models
text: Following Jain et al. (2020), we use BERT (Devlin et al., 2019) for SST and AG; SCIBERT (Beltagy et al., 2019) for EV.INF. and RoBERTa (Liu et al., 2019) for M.RC. See App. A for hyperparameters. Dataset statistics and model prediction performance are shown in Table 1.

heading: 4.3 Feature Attribution Methods
text: We experiment with several popular feature attribution methods to compare faithfulness metrics. We
do not focus on benchmarking various FAs but to improve faithfulness evaluation metrics.
• Attention (α): Token importance is computed using the corresponding normalized attention score (Jain et al., 2020).
• Scaled attention (α∇α): Attention scores scaled by their corresponding gradients (Serrano and Smith, 2019).
• InputXGrad (x∇x): It attributes importance by multiplying the input with its gradient computed with respect to the predicted class (Kindermans et al., 2016; Atanasova et al., 2020).
• Integrated Gradients (IG): This FA ranks input tokens by computing the integral of the gradients taken along a straight path from a baseline input (i.e. zero embedding vector) to the original input (Sundararajan et al., 2017).
• DeepLift (DL): It computes token importance according to the difference between the activation of each neuron and a reference activation, i.e. zero embedding vector (Shrikumar et al., 2017).

heading: 4.4 Computing Faithfulness with Normalized Sufficiency and Comprehensiveness
text: Following DeYoung et al. (2020), we compute the Area Over the Perturbation Curve (AOPC) for normalized sufficiency (NS) and comprehensiveness (NC) across different rationale lengths. AOPC provides a better overall estimate of faithfulness (DeYoung et al., 2020). We evaluate five different rationale ratios set to 1%, 5%, 10%, 20% and 50%, similar to DeYoung et al. (2020) and Chan et al. (2022).

heading: 4.5 Comparing the Diagnosticity of Faithfulness Metrics
text: Comparing faithfulness metrics is a challenging task because there is no a priori ground truth rationales that can be used.
Diagnosticity: Chan et al. (2022) proposed diagnosticity to measure the degree of a given faithfulness metric favors more faithful rationales over less faithful ones. The assumption behind this metric is that the importance scores assigned by a FA are highly likely to be more faithful than simply assigning random importance scores to tokens. Given an
explanation pair (u, v), the diagnosticity is measured as the probability of u being a more faithful explanation than v given the same faithfulness metric F . u is an explanation determined by a FA, while v is a randomly generated explanation for the same input. For example the NC score of u should be higher than v when evaluating the diagnosticity of using NC as the faithfulness metric. More formally, diagnosticity Dε(F ) is computed as follows:2
Dε(F ) ≈ 1 |Zε| ∑
(u,v)∈Zε 1(u F v) (6)
where F is a faithfulness metric, Zε is a set of explanation pairs, also called ε-faithfulness golden set, 0 ≤ ε ≤ 1. 1· is the indicator function which takes a value 1 when the input statement is true and a value 0 when it is false.
Chan et al. (2022) randomly sample a subset of explanation pairs (u, v) for each dataset and also randomly sample a FA for each pair. In our experiments, we do not sample but we consider all the possible combinations of data points and FAs across datasets.

heading: 5 Results

heading: 5.1 Diagnosticity of Faithfulness Metrics
text: We compare the diagnosticity of faithfulness metrics introduced in Section 3. Tables 2 and 3 show average diagnosticity scores across FAs and tasks, respectively. See App. B for individual results for each faithfulness metric, FA and dataset.
In general, we observe that Soft-NC and Soft-NS achieve significantly higher diagnosticity scores (Wilcoxon Rank Sum, p < .01) than NC and NS across FAs and datasets. The average diagnosticity of Soft-NC is 0.529 compared to 0.394 of NC while the diagnosticity of Soft-NS is 0.462 compared to NS (0.349). Our faithfulness metrics outperform NC and NS in 16 out of 18 cases, with the exception of Soft-NC on AG and Soft-NS on M.RC.
In Table 2, we note that both NC and Soft-NC consistently outperform Soft-NS and NS, which corroborates findings by Chan et al. (2022). We also see that using different FAs result into different diagnosticity scores. For example, diagnosticity ranges from 0.514 to .561 for Soft-NC while SoftNS ranges from .441 to .480. We also observe similar behavior for NC and NS confirming results
2For a proof of Eq. 6, refer to Chan et al. (2022).
from Atanasova et al. (2020). Furthermore, we surprisingly see that various faithfulness metrics disagree on the rankings of FAs. For example DL is the most faithful FA measured by Soft-NC (.561) while NC ranks it as one of the least faithful (.372). However, Soft-NC and Soft-NS appear to be more robust by having less variance.
In Table 3, we observe that the diagnosticity of all four faithfulness metrics is more sensitive across tasks than FAs (i.e. wider range and higher variance). Also, we notice that in AG and M.RC, there is a trade-off between (Soft-)NS and (Soft-)NC. For example, on AG, Soft-NC is .649, the highest among all tasks but Soft-NS is the lowest. This result may be explained by the larger training sets of AG (102,000) and M.RC (24,029), compared to SST (6,920) and Ev.Inf (5,789) which might make the model more sensitive to the task-specific tokens.

heading: 5.2 Qualitative Analysis
text: We further conduct a qualitative analysis to shed light on the behavior of faithfulness metrics for different explanation pairs consisting of real and random attribution scores. Table 4 shows three examples from Ev.Inf, SST and AG respectively.
Repetitions in rationales affect faithfulness: Examining Example 1 (i.e. a biomedical abstract from Ev.Inf), we observe that the rationale (top 20% most important tokens) identified by DL contains repetitions of specific tokens, e.g. “aliskiren”, “from”, “in”. On one hand, “aliskiren” (i.e. a drug for treating high blood pressure) is the main subject of the biomedical abstract and have been correctly identified by DL. On the other hand, we observe that many of these repeated tokens might not be very informative (e.g. many of them are stop
words), however they have been selected as part of the rationale. This might happen due to their proximity to other informative tokens such as “aliskiren” due to the information mixing happening because of the contextualized transformer encoder (Tutek and Snajder, 2020).
We also notice that the random attribution baseline (Rand) selects a more diverse set of tokens that appear to have no connection between each other as expected. The random rationale also contains a smaller proportion of token repetitions. These may be the reasons why the random rationales may, in some cases, provide better information compared to the rationales selected by DL (or other FAs), leading to lower diagnosticity. Furthermore, NC between DL (.813) and Rand (.853) is very close (similar for NS) which indicates similar changes to predictive likelihood when retaining or removing rationales by DL and Rand. However, this may misleadingly suggest a similar model reasoning on the two rationales. We observe similar patterns using other FAs. Incorporating the FA importance scores in the input embeddings helps Soft-NC and Soft-S to mitigate the impact of issues above as they use all tokens during the evaluation.
Evenly distributed FA scores affect NC and NS: We also notice that for some inputs, the token importance assigned by FAs is very close to each other as demonstrated in Example 3, i.e. a news article from AG. The evenly distributed importance scores lead to similar low NC and NS between the FA (IG) and the random baseline attribution. Considering that the FA scores and ranking truly reflect the model reasoning process (i.e. the model made this prediction by equally weighing all tokens), then the faithfulness measurements provided by NS and NC might be biased.
We conjecture that this is likely to happen because these metrics entirely ignore the rest of the tokens even though these could represent a non-
faithfulness metrics scores. Highlighted tokens are the rationales by a given FA and the random baseline. The tints indicate their importance scores, the lighter the less important. The three examples are from Ev.Inf, SST and AG, respectively.
negligible percentage of the FA scores distribution. However, Soft-NC and Soft-NS take into account the whole FA distribution without removing or retaining any specific tokens, hence they do not suffer from this limitation.
Different part of speech preferences for tasks We find that FAs tend to favor different parts of speech for different tasks. In Example 1 where the task is to reason about the relationship between a given intervention and a comparator in the biomedical domain, FAs tend to select proper nouns (e.g. “aliskiren”) and prepositions (e.g. “on”, “in” and “to”). On the other hand, in Example 2 which shows a text from SST, FAs favor adjectives (e.g. “unconscious” and “hard”) for the sentiment analysis task. In Example 3, we see that articles such as “the” and proper nouns such as “Greece” and “Bustos” are selected.

heading: 6 Impact of Rationale Length on Faithfulness and Diagnosticity
text: Up to this point, we have only considered computing cumulative AOPC NC and NS by evaluat-
ing faithfulness scores at multiple rationale lengths together (see Section 3). Here, we explore how faithfulness and diagnosticity of NC and NS at individual rationale lengths compare to Soft-NC and Soft-NS. We note that both ‘soft’ metrics do not take the rationale length into account.

heading: 6.1 Faithfulness
text: Figure 2 shows the faithfulness scores of NC and NS at different rationale lengths for all FAs including random baseline attribution in each dataset.3 We observe that the faithfulness scores of NC and NS follow an upward trend as the rationale length increases. This is somewhat expected because using information from an increasing number of tokens makes the rationale more similar to the original input.
In AG and SST, NC and NS lines appear close by or overlap. One possible reason is that the input text in SST and AG is relatively short (average length of 18 and 36 respectively), possibly leading to higher contextualization across all tokens. There-
3For brevity, we do not highlight the different FAs as they follow similar patterns.
fore, removing or retaining more tokens results in a similar magnitude of changes in predictive likelihood.
In M.RC and Ev.Inf, two comprehension tasks that consist of longer inputs (average length is 305 and 365 respectively), we observe a different relationship between NC and NS. For instance, NC in Ev.Inf tends to be less impacted by the rationale length. This maybe due to the token repetitions in rationales discussed in Section 5.2. For example, when taking 2% of the top-k tokens out, e.g. 6 out of 300 tokens, all the task-related tokens may have been removed already.

heading: 6.2 Diagnosticity
text: Figure 3 shows the diagnosticity scores of NS and NC on different rationale lengths (average across FAs) together with the diagnosticity of Soft-NC and Soft-NS. Overall in all datasets, we see that the diagnosticity of NC and NS does not monotonically increase as we expected. In SST and AG, the diagnosticity of NS and NC both initially increase and then decrease. This happens because after increasing to a certain rationale length, the random selected rationales (used in the diagnosticity metric) contain sufficient information making it hard for FAs to beat. In M.RC and Ev.Inf, Soft-NC and Soft-NS have higher diagnosticity than NC and NS. One possible reason is that the corrupted version of input could fall out-of-distribution, confusing the model. Our ‘soft’ metrics mitigate this issue by taking all tokens into account.
Based on the observations on Figures 2 and 3, we conclude that it is hard to define an optimal rationale length for NC and NS which also has been demonstrated in previous work (Chrysostomou and
Aletras, 2022b). In general, we see that diagnosticity decreases along with longer rationale length for NC and NS. On the other hand, faithfulness measured by NC and NS increases for longer rationales (Figure 2). Therefore, this might be problematic for selecting optimal rationale length for NC and NS. For example, if we want to select an optimal rationale length for M.RC by looking at its relation to faithfulness, we might choose a length of 30% over 20% because it shows higher NC and NS. However, the diagnosticity of NC and NS is lower at 30%, which means the higher NC and NS results to less trustful rationales. Our metrics bypass these issues because they focus on evaluating the FA scores and ranking as a whole considering all the input tokens. Soft-NC and Soft-NS do not require a pre-defined rationale length or evaluating faithfulness across different lengths.
We suggest that it is more important to identify the most faithful FA given a model and task by taking into account all tokens rather than pre-defining a rationale of a specific length that ignores a fraction of the input tokens when evaluating faithfulness. The choice of how the FA importance scores will be presented (e.g. a top-k subset of the input tokens or all of them using a saliency map) should only serve practical purposes (e.g. better visualization, summarization of model rationales).

heading: 7 Conclusion
text: In this work, we have proposed a new softperturbation approach for evaluating the faithfulness of input token importance assigned by FAs. Instead of perturbing the input by entirely removing or retaining tokens for measuring faithfulness, we incorporate the attribution importance by ran-
domly masking parts of the token embeddings. Our soft-sufficiency and soft-comprehensiveness metrics are consistently more effective in capturing more faithful FAs across various NLP tasks. In the future, we plan to experiment with sequence labeling tasks. Exploring differences in faithfulness metrics across different languages is also an interesting avenue for future work.
Limitations
This work focuses on binary and multi-class classification settings using data in English. Benchmarking faithfulness metrics in sequence labeling tasks as well as in multi-lingual settings should be explored in future work.

heading: Acknowledgements
text: ZZ and NA are supported by EPSRC grant EP/V055712/1, part of the European Commission CHIST-ERA programme, call 2019 XAI: Explainable Machine Learning-based Artificial Intelligence. This project made use of time on Tier 2 HPC facility JADE2, funded by EPSRC (EP/T022205/1). We thank Huiyin Xue for her invaluable feedback.

heading: A Model Hyperparameters
text: We use pre-trained models from the Huggingface library (Wolf et al., 2020). We use the AdamW optimizer (Loshchilov and Hutter, 2019) with an initial learning rate of 1e−5 for fine-tuning BERT. We fine-tune all models for 3 epochs using a linear scheduler, with 10% of the data in the first epoch as warming up. We also use a grad-norm of 1.0. The model with the lowest loss on the development set is selected. All models are trained across 5 random seeds, and we report the average. Experiments are run on a single Nvidia Tesla V100 GPU. Table 5 shows an overview of models and hyperparameters.

heading: B Detailed Diagnosticity Results

heading: C Alternative implementations for soft perturbation
text: Adding Gaussian noise We perturb the pretrained word embeddings with standard Gaussian noise. This Gaussian noise-based embedding perturbation is similar to the “statistical noise” used by Zhang and Yang (2018) and Lakshmi Narayan et al. (2019) for data augmentation. Specifically, we:
1. Multiply the token embedding with the token importance score, adding Gaussian noise. The resulting embedding is γλ xi in Equation 7, where xi is the original input embedding and λ is the FA scores (importance degree), γ is the hyperparameters based on the FA scores. is element-wise multiplication. As demonstrated by Lakshmi Narayan et al. (2019), adding Gaussian noise to the embedding requires tuning the standard deviation. Similarly, we tune the standard deviation σ2 ∈ {0.005, 0.01, 0.05, 0.1, 0.5, 1, 2} for softcomprehensiveness and soft-sufficiency separately.
2. Add the embedding γλ xi, to the token embedding (xi) to obtain a perturbed embedding (x′i).
x′i = xi + γλ xi, γ ∼ N (µ, σ2) (7)
An alternative way to add noise is to:
1. Generate a noise embedding by multiplying the token embedding with Gaussian noise with standard deviation, σ2, associated with the importance score of the token. The embedding γ xi in Equation 8, where xi is the original input embedding and λ is the importance score.
2. Add γ xi, to the token embedding (xi) to get the perturbed embedding (x′i).
x′i = xi + γ xi, γ ∼ N (µ, σ2) (8)
Continuous attention mask We simply replace the binary-valued attention mask with a continuousvalued mask, where the continuous value is associated with the FA score for each token. The remaining part of the embeddings and the model remain the same.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Left blank.
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Yes, we use multiple datasets which are cited and presented in section 4
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Open source
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Use of public datasets

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 4
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4
C 3 Did you run computational experiments? Section 4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Appendix
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 4 and Appendix

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 5

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? also will provide a comprehensive list for the environment, packages and dependencies in the to-be-released git repo
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: In the future, we plan to experiment with sequence labeling tasks. Exploring differences in faithfulness metrics across different languages is also an interesting avenue for future work. Limitations
This work focuses on binary and multi-class classification settings using data in English. Benchmarking faithfulness metrics in sequence labeling tasks as well as in multi-lingual settings should be explored in future work