Using Interpretation Methods for Model Enhancement (2023)

Abstract: In the age of neural natural language processing, there are plenty of works trying to derive interpretations of neural models. Intuitively, when gold rationales exist during training, one can additionally train the model to match its interpretation with the rationales. However, this intuitive idea has not been fully explored. In this paper, we propose a framework of utilizing interpretation methods and gold rationales to enhance models. Our framework is very general in the sense that it can incorporate various interpretation methods. Previously proposed gradient-based methods can be shown as an instance of our framework. We also propose two novel instances utilizing two other types of interpretation methods, erasure/replace-based and extractor-based methods, for model enhancement. We conduct comprehensive experiments on a variety of tasks. Experimental results show that our framework is effective especially in low-resource settings in enhancing models with various interpretation methods, and our two newly-proposed methods outperform gradient-based methods in most settings. Code is available at https://github.com/Chord-Chen-30/UIMER.

## introduction deep neural networks have been extensively used to solve natural language processing (nlp) tasks and reach state-of-the-art performance. due to the black-box nature of neural models, there are a lot of studies on how to interpret model decisions by giving attribution scores to input tokens, i.e., how much tokens in an input contribute to the final prediction. we can roughly group these interpretation methods into four categories, namely gradientbased @xcite , attention-based @xcite , erasure/replace-based @xcite and extractor-based @xcite tokens add step to me to the 50 clásicos playlist attribution scores 0.7 0.3 0.1 0.1 0.1 0.3 0.0 0.1 0.6 gold rationales 1 0 0 0 0 0 0 0 1 figure 2: an example of attribution scores and gold rationales. in some scenarios, we have access to gold rationales (input tokens critical for predicting correct outputs) during training, or have simple and fast approaches to obtaining gold rationales. in that case, it is intuitively appealing to additionally train the model such that its most attributed tokens match gold rationales (see an example in fig. 2 ). in other words, when equipped with interpretation methods, we can train the model on where to look at in the input, in addition to the standard output-matching objective. this can be seen as injecting external knowledge embodied in rationales into the model and is especially beneficial to low-resource scenarios with few training data. this intuitive idea, however, has not been fully explored. there are only a few previous studies based on the gradientbased category of interpretation methods @xcite @xcite and they neither compare the utilization of different interpretation methods on model enhancement nor experiment on a comprehensive range of tasks. in this paper, we first propose a framework named uimer that utilizes interpretation methods and gold rationales to improve model performance, as illustrated in fig. 1 . specifically, in addition to a task-specific loss, we add a new loss that aligns interpretations derived from interpretation methods with gold rationales. we also discuss how the optimization of the task-specific loss and the new loss should be coordinated. for previous methods utilizing gradient-based interpretation for model enhancement @xcite , we show that they can be seen as instances of our framework. we then propose two novel instances of our framework based on erasure/replace-based and extractor-based interpretation methods respectively. specifically, in the first instance, we utilize input marginalization @xcite as the interpretation method in our framework, which computes attribution scores by replacing tokens with a variety of strategies and measuring the impact on outputs, and we design a contrastive loss over the computed attribution scores of rationales and non-rationales. in the second instance, we utilize rationale extractor @xcite as the interpretation method in our framework, which is a neural model that is independent of the task model and trained with its own loss. we again design a contrastive loss over the attribution scores computed by the extractor and in addition, design a new training process that alternately optimizes the task model (using the task-specific loss and our contrastive loss) and the extractor (using its own loss). in summary, our main contributions can be summarized as follows: (1) we propose a framework that can utilize various interpretation methods to enhance models. (2) we utilize two novel types of interpretation methods to enhance the model under our framework. (3) we comprehensively evaluate our framework on diversified tasks including classification, slot filling and natural language inference. experiments show that our framework is effective in enhancing models with various interpretation methods, especially in the low-resource setting. ## method we propose a framework uimer that utilizes an interpretation method to enhance models based on gold rationales on training data. setup consider a training example with input x and gold output y. we also have access to gold rationales g of input x indicating the subset of tokens that are critical for predicting the correct output y. the gold rationales g can be annotated by humans or generated automatically from external knowledge sources. in our setup, g is encoded as a 0/1 vector with g @xmath0. our method, however, can be easily extended to handle g encoded with real numbers indicating the importance of a token. given a model tasked with predicting output y from input x, an interpretation method produces attribution scores a for input x. a can be defined on different levels of granularity. in common cases, a i and x i are one-to-one and a i is defined by a measure calculated by the interpretation method based on the model indicating the importance of token x i for the model in producing its output. for example, in gradient-based interpretation methods, a i is usually some function of the gradient of x i in the model. a higher magnitude of the gradient implies higher importance of input x i . learning objective apart from the original taskspecific learning objective l task , our framework introduces an extra learning objective l int that embodies the idea of aligning attribution scores a with gold rationales g. the overall objective on one example x takes the form of: warm-up training l int can be seen as measuring whether the model pays attention to the gold rationale words in the input. we deem that compared to teaching a randomly initialized model focusing more on the task-specific rationale words, teaching a model with task knowledge is more effective and reasonable because it might be better at recognizing task-specific rationales with the help of task knowledge rather than rote memorization. thus, instead of optimizing l θ at the very beginning, our framework requires the model to be well or at least halfway trained before training on the objective l int , and during warm-up training, only the objective of the task is optimized. the empirical results (in sec. 4.2) also support our intuition and show that warm-up training is effective. in the following subsections, we introduce three instances of our framework. the first utilizes gradient-based interpretation methods and subsumes a few previous studies as special cases. the second and third are new methods proposed by us utilizing erasure/replace-based and extractor-based interpretation methods respectively. ## conclusion though many interpretation methods are studied for deep neural models, only sporadic works utilize them to enhance models. in this paper, we propose a framework that can utilize various interpretation methods to enhance models. we also propose two novel instances utilizing two other types of interpretation methods for model enhancement. in addition, we discuss how the optimization of the task-specific loss and the new loss should be coordinated. comprehensive experiments are conducted on a variety of tasks including intent classification, slot filling and natural language inference. experiments show that our framework is effective in enhancing models with various interpretation methods especially in the low-resource setting, and our two newly-proposed methods outperform gradient-based methods in most settings. for future work, we plan to extend our framework to utilize more forms of rationales and additional interpretation methods.