Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience (2021)

Abstract: Pretrained transformer-based models such as BERT have demonstrated state-of-the-art predictive performance when adapted into a range of natural language processing tasks. An open problem is how to improve the faithfulness of explanations (rationales) for the predictions of these models. In this paper, we hypothesize that salient information extracted a priori from the training data can complement the task-specific information learned by the model during fine-tuning on a downstream task. In this way, we aim to help BERT not to forget assigning importance to informative input tokens when making predictions by proposing SaLoss; an auxiliary loss function for guiding the multi-head attention mechanism during training to be close to salient information extracted a priori using TextRank. Experiments for explanation faithfulness across five datasets, show that models trained with SaLoss consistently provide more faithful explanations across four different feature attribution methods compared to vanilla BERT. Using the rationales extracted from vanilla BERT and SaLoss models to train inherently faithful classifiers, we further show that the latter result in higher predictive performance in downstream tasks.

## introduction pretrained transformer-based @xcite language models (lms) such as bert @xcite , have achieved state-of-the-art results in various language understanding tasks @xcite . despite their success, their highly complex nature consisting of millions of parameters, makes them difficult to interpret @xcite . this has motivated new research on understanding and explaining their predictions. previous work has explored whether lms encode syntactic knowledge by studying their multi-head attention distributions @xcite @xcite . recent studies have evaluated the faithfulness of explanations foot_1 for predictions made by these models @xcite @xcite . in general, lms can provide faithful explanations, particularly using attention @xcite , but still fall behind other simpler architectures @xcite possibly due to increased information mixing and higher contextualization in the model @xcite @xcite . recent studies have attempted to improve the explainability of non transformer-based models, by guiding them through an auxiliary objective towards informative input importance distributions (e.g. human or adversarial priors) @xcite @xcite . in a similar direction, we propose salient loss (saloss), an auxiliary objective that allows the multi-head attention of the model to learn from salient information (i.e. token importance) during training to reduce the effects of information mixing @xcite . we compute a priori token importance scores @xcite using textrank @xcite (i.e. an unsupervised graph-based method) and penalize the model when the attention distribution deviates from the salience distribution. our contributions are as follows: ## related work model explainability explanations can be obtained by computing importance scores for input tokens to identify which parts of the input contributed the most towards a model's prediction (i.e. feature attribution). a common approach to attributing input importance is by measuring differences in a model's prediction between keeping and omitting an input token @xcite @xcite . input importance can also be obtained by calculating the gradients of a prediction with respect to the input @xcite @xcite . we can also use sparse linear meta-models that are easier to interpret @xcite . finally, recent studies propose using feature attribution to extract a fraction of the input as a rationale and then use it to train a classifier @xcite . ## improving explanation faithfulness with word salience even though attention scores are more faithful than other feature attribution approaches @xcite , they usually pertain to their corresponding input tokens in context and not individually due to information mixing @xcite . as such, we hypothesize that we can improve the ability of a pretrained lm in providing faithful explanations, by showing to the model alternative distributions of input importance (i.e. word salience). we assume that by introducing the salience distribution via an auxiliary objective @xcite , we can reduce information mixing by "shifting" the model's attention to other informative tokens. in a similar direction to ours, @xcite showed that by computing attention together with salience information from keyword extractors improves text summarization. computing word salience we compute word salience σ using textrank @xcite , an unsupervised graph-based model for keyword extraction. textrank calculates indegree centrality of graph nodes iteratively based on a markov chain, where each node is a wordpiece and each edge links wordpiece pairs within a context window @xcite . for each input document x, we construct an undirected graph and apply textrank to compute the local salience scores (σ i ) of its words by: we assume a standard text classification setting where a set of labeled documents is used for finetuning a pretrained lm by adding an extra output classification layer. we normalize the salience scores for compatibility with the kl divergence. ## experimental setup datasets we consider five natural language understanding tasks (see dataset statistics in appx. a): sst @xcite ; agnews (ag) @xcite ; evidence inference (ev.inf.) @xcite ; multirc (m.rc) @xcite and semeval 2017 task 4 subtask a (semeval) @xcite . models similar to @xcite we use: bert @xcite for (sst, ag, se-meval); scibert @xcite for ev.inf.; roberta @xcite for m.rc. ## qualitative analysis in in ag we observed similar performance between models trained with saloss and without. example 1 illustrates such a case, where both models predicted correctly but attended to different parts of the input. despite in different locations, both segments are closely associated with the label of "business". example 2 is an instance from the sst dataset, were the saloss rationale points to a phrase that is more associated with the task ("a promising unusual") compared to the base-line. this also aligns with previous observations from the pos analysis, that models trained with our proposed objective attend to more adjectives compared to baseline. example 3 considers an instance from the ev.inf. dataset, which shows that the model trained with saloss and base-line attended to two different sections. in fact what we observed in agreement with the pos analysis, is that models with saloss attend mostly to segments including words related to relationships, such as "significantly attenuated" in this particular example. ## conclusion we introduced salient loss (saloss), an auxiliary objective to incorporate salient information to attention for improving the faithfulness of transformer-based prediction explanations. we demonstrate that our approach provides more faithful explanations compared to vanilla lms on input erasure and rationale extraction. in the future, we plan to explore additional objectives to better optimize for contiguity of rationales. ag news @xcite : news articles categorized by the following topics; science, sports, business, and world. ev.inf (evidence inference) @xcite : abstract-only biomedical articles describing randomized controlled trials. the task is to infer the reported relationship between a given intervention and comparator with respect to an outcome. m.rc (multi rc) @xcite : a reading comprehension dataset composed of questions with multiple correct answers that depend on information from multiple sentences. similar to deyoung et al. (2020) and jain et al. (2020) we convert this to a binary classification task where each rationale/question/answer triplet forms an instance and each candidate answer has a label of true or false. semeval nakov et al. (2013): the semeval 2017 dataset for task 4 subtask a which consists of tweets and the task is to classify whether the message is of positive, negative, or neutral sentiment.