Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification (2021)

Abstract: Neural network architectures in natural language processing often use attention mechanisms to produce probability distributions over input token representations. Attention has empirically been demonstrated to improve performance in various tasks, while its weights have been extensively used as explanations for model predictions. Recent studies (Jain and Wallace, 2019; Serrano and Smith, 2019; Wiegreffe and Pinter, 2019) have showed that it cannot generally be considered as a faithful explanation (Jacovi and Goldberg, 2020) across encoders and tasks. In this paper, we seek to improve the faithfulness of attention-based explanations for text classification. We achieve this by proposing a new family of Task-Scaling (TaSc) mechanisms that learn task-specific non-contextualised information to scale the original attention weights. Evaluation tests for explanation faithfulness, show that the three proposed variants of TaSc improve attention-based explanations across two attention mechanisms, five encoders and five text classification datasets without sacrificing predictive performance. Finally, we demonstrate that TaSc consistently provides more faithful attention-based explanations compared to three widely-used interpretability techniques.

## introduction natural language processing (nlp) approaches for text classification are often underpinned by large neural network models @xcite . despite the high accuracy and efficiency of these models in dealing with large amounts of data, an important problem is their increased complexity that makes them opaque and hard to interpret by humans which usually treat them as black boxes @xcite . attention mechanisms @xcite ) produce a probability distribution over the input to compute a vector representation of the entire token sequence as the weighted sum of its constituent vectors. a common practice is to provide explanations for a given prediction and qualitative model analysis by assigning importance to input tokens using scores provided by attention mechanisms @xcite @xcite as a mean towards model interpretability @xcite . a faithful explanation is one that accurately represents the true reasoning behind a model's prediction (jacovi and goldberg, 2020). a series of recent studies illustrate that explanations obtained by attention weights do not always provide faithful explanations @xcite while different text encoders can affect attention interpretability, e.g. results can differ when using a recurrent or non-recurrent encoder @xcite . a limitation of attention as an indicator of input importance is that it refers to the word in context due to information mixing in the model @xcite . motivated by this, we aim to improve the effectiveness of neural models in providing more faithful attention-based explanations for text classification, by introducing noncontextualised information in the model. our contributions are as follows: ## neural text classification models in a typical neural model with attention for text classification; one-hot-encoded tokens x i p r |v| are first mapped to embeddings e i p r d , where i p r1, ..., ts denotes the position in the sequence, t the sequence length, |v | the vocabulary size and d the dimensionality of the embeddings. the embeddings e i are then passed to an encoder to produce hidden representations h i " encpe i q, where h i p r n , with n the size of the hidden representation. a vector representation c for the entire text sequence x 1 , ..., x t is subsequently obtained as the sum of h i weighted by attention scores α i : vector c is finally passed to the output, a fullyconnected linear layer followed by a softmax activation function. ## comparing tasc with non-attention input importance metrics we finally compare explanations provided by using lin-tasc and α∇α to three standard non-attention input importance metrics without tasc which are strong baselines for explainability @xcite . word omission (wo) @xcite : ranking input words by computing the difference between the probabilities of the predicted class when including a word i and omitting it: wo i " ppŷ|xq ṕpŷ|x @xcite : ranking words by multiplying the gradient of the input by the input with respect to the predicted class: integrated gradients (ig) @xcite : ranking words by computing the integral of the gradients taken along a straight path from a baseline input to the original input, where the baseline is the zero embedding vector. ## qualitative analysis we finally examine qualitatively what type of information the parameter u from lin-tasc learns. similar to a bag-of-words model, our initial hypothesis is that u will assign high scores to the words that are most relevant to the task. figure 4 illustrates the 5 highest and lowest scored words from the imdb and adr datasets with a lstm encoder and dot attention and cnn encoder and tanh attention respectively. for brevity we include two examples, however observations hold similar throughout other configurations (e.g. encoders, datasets) and when increasing the number of top-k words. we first observe in 4a, that indeed words expressing sentiment are assigned with high scores (e.g. excellent, waste, perfect), either positive or negative. however, a positive or negative sign does tanh dot non-tasc tasc non-tasc tasc data enc() wo x∇x ig α∇α wo x∇x ig α∇α sst bert .29 .64 .51 .22 .32 .62 .49 .55 lstm .25 .24 .20 .19 .21 .23 .19 .19 gru .24 .22 .19 .18 .24 .25 .23 .19 mlp .36 .26 .24 .18 .22 .19 .18 .18 cnn .30 .25 .20 .19 .22 .20 .18 .19 adr bert .83 .91 .89 .31 .81 .90 .87 .50 lstm .82 .81 .80 .32 .87 .88 .87 .34 gru .84 .84 .84 .35 .79 .80 .80 .38 mlp .71 .63 .57 .31 .49 .43 .39 .40 cnn .80 .78 .78 .37 .77 .74 .74 .36 imdb bert .23 .69 .43 .07 .24 .72 .49 .20 lstm .18 .12 .07 .04 .26 .09 .07 .05 gru .18 .12 .07 .04 .27 .15 .08 .05 mlp .16 .05 .05 .05 .18 .07 .06 .05 cnn .21 .09 .07 .05 .27 .07 .06 .05 ag bert .62 .78 .56 .50 .56 .76 .60 .60 lstm .53 .51 .30 .38 .47 .52 .35 .46 gru .45 .36 .31 .20 .54 .40 .30 .22 mlp .53 .24 .25 .19 .44 .25 .23 .19 cnn .55 .38 .28 .20 .53 .35 .25 .21 mimic bert .24 .67 .43 .03 .21 .57 .26 .05 lstm .35 .32 .12 .01 .28 .40 .30 .01 gru .20 .24 .23 .01 .36 .18 .08 .01 mlp .40 .03 .22 .01 .13 .04 .03 .02 cnn .26 .15 .02 .01 .43 .09 .02 .02 ## conclusion we introduced tasc, a family of three encoderindependent mechanisms that induce contextindependent task-specific information to attention. we conducted an extensive series of experiments showing the superiority of tasc over vanilla attention on improving faithfulness of attention-based interpretability without sacrificing predictive performance. finally, we showed that attention-based explanations with tasc outperform other interpretability techniques. for future work, we will explore the effectiveness of tasc in sequence-tosequence tasks similar to @xcite .