Towards Faithful Explanations for Text Classification with Robustness Improvement and Explanation Guided Training (2023)

Abstract: Feature attribution methods highlight the important input tokens as explanations to model predictions, which have been widely applied to deep neural networks towards trustworthy AI. However, recent works show that explanations provided by these methods face challenges of being faithful and robust. In this paper, we propose a method with Robustness improvement and Explanation Guided training towards more faithful EXplanations (REGEX) for text classification. First, we improve model robustness by input gradient regularization technique and virtual adversarial training. Secondly, we use salient ranking to mask noisy tokens and maximize the similarity between model attention and feature attribution, which can be seen as a self-training procedure without importing other external information. We conduct extensive experiments on six datasets with five attribution methods, and also evaluate the faithfulness in the out-of-domain setting. The results show that REGEX improves fidelity metrics of explanations in all settings and further achieves consistent gains based on two randomization tests. Moreover, we show that using highlight explanations produced by REGEX to train select-then-predict models results in comparable task performance to the end-to-end method.

## introduction as the broad adoption of pre-trained language models (plms) requires humans to trust their output, we need to understand the rationale behind the output and even ask questions regarding how the model comes to its decision @xcite . recently, explanation methods for interpreting why a model makes certain decisions are proposed and become more crucial. for example, feature attribution methods assign scores to tokens and highlight the important ones as explanations @xcite @xcite . post-hoc explanation methods deep neural networks model explanations model robustness explanation faithfulness a warm funny engaging film original: pertrubed: a warm funny engaging film positive positive model a: it is a warm funny engaging film . model a': it is a warm funny engaging film . figure 1 : visualization of positive and negative highlights produced by post-hoc explanation methods (e.g., feature attribution). however, these explanations suffer from unfaithfulness problems (e.g., same model framework a and a' with different attributions) and can be further fooled by adversarial manipulation without changing model output @xcite (see §4.4). to this end, we propose a method with robustness improvement and explanation guided training to improve the faithfulness of explanations (regex) while preserving the task performance for text classification. first, we apply the input gradient regularization technique and virtual adversarial training to improve model robustness. while previous works found that these mechanisms can improve the adversarial robustness and interpretability of deep neural networks @xcite , to the best of our knowledge, the faithfulness of model explanations by applying them has not been explored. secondly, our method leverages token attributions aggregated by the explanation method, which provides a local linear approximation of the model's behaviour @xcite . we mask input tokens with low feature attribution scores to generate perturbed text and then maximize the similarity between new attention and attribution scores. furthermore, we minimize the kullback-leibler (kl) divergence between model attention of original input and attributions. the main idea is to allow attention distribution of the model to learn from input importance during training to reduce the effect of noisy information. to verify the effectiveness of regex, we consider a variety of classification tasks across six datasets with five attribution methods. additionally, we conduct extensive empirical studies to examine the faithfulness of five feature attribution approaches in out-of-domain settings. the results show that regex improves the faithfulness of the highlight explanations measured by sufficiency and comprehensiveness @xcite in all settings while outperforming or performing comparably to the baseline, and further achieves consistent gains based on two randomization tests. moreover, we show that using the explanations output from regex to train select-then-predict models results in comparable task performance to the endto-end method, where the former trains an independent classifier using only the rationales extracted by the pre-trained extractor @xcite . considering neural network models may be the primary source of fragile explanations @xcite , our work can be seen as a step towards understanding the connection between explainability and robustness -the desiderata in trustworthy ai. the main contributions of this paper can be summarized as: ## related work model robustness and explainability as it has recently been shown that deep neural networks are vulnerable to adversarial attacks even with plms, several works are proposed to ensure that ai systems are trustworthy and reliable, which include quantifying the vulnerability and designing new attacks and better defense technologies @xcite . however, as the debug tools for black-box models, explanation methods also lack robustness to imperceptible and targeted perturbations of the input @xcite @xcite . while significantly different explanations are provided for similar models @xcite , how to elicit more reliable explanations is a promising direction towards interpretation robustness. different from @xcite that addresses the inconsistent phenomenon of explanations, we investigate the connection between model robustness and faithfulness of the explanations. ## experiments we conduct the experiments in six datasets under the in-domain/out-of-the-domain settings: sst @xcite , imdb @xcite , yelp @xcite , and amazdigimu/amazpantry/amazinstr @xcite (see details in appendix a). the baseline is a text classification model fine-tuned on the training set while the same pre-trained language model is applied to regex. in other words, the baseline is optimized by eqn. 1 without robustness improvement and explanation guided training mechanisms. ## conclusion we explore whether the fidelity of explanations can be further optimized and propose an explanation guided training mechanism. extensive empirical studies are conducted on six datasets in both in-and out-of-domain settings. results show that our method regex improves both fidelity metrics and performance of select-then-predict models. the analysis of explanation robustness further shows that the consistency of explanations has been improved. the observation suggests that considering model robustness yields more faithful explanations. in the future, we would like to investigate more plms architectures and faithfulness metrics under the standard evaluation protocol.