REFER : An End-to-end Rationale Extraction Framework for Explanation Regularization (2023)

Abstract: Human-annotated textual explanations are becoming increasingly important in Explainable Natural Language Processing. Rationale extraction aims to provide faithful (i.e. reflective of the behavior of the model) and plausible (i.e. convincing to humans) explanations by highlighting the inputs that had the largest impact on the prediction without compromising the performance of the task model. In recent works, the focus of training rationale extractors was primarily on optimizing for plausibility using human highlights, while the task model was trained on jointly optimizing for task predictive accuracy and faithfulness. We propose REFER, a framework that employs a differentiable rationale extractor that allows to back-propagate through the rationale extraction process. We analyze the impact of using human highlights during training by jointly training the task model and the rationale extractor. In our experiments, REFER yields significantly better results in terms of faithfulness, plausibility, and downstream task accuracy on both in-distribution and out-of-distribution data. On both e-SNLI and CoS-E, our best setting produces better results in terms of composite normalized relative gain than the previous baselines by 11% and 3%, respectively.

## introduction neural language models have emerged as stateof-the-art (sota) performers in a wide range of natural language processing (nlp) tasks @xcite . however, they are often perceived as opaque @xcite @xcite , sparking significant interest in the development of algorithms that can automatically explain the behavior of these models @xcite @xcite @xcite . in the field of self-explainable neural models, two prominent approaches have emerged: (i) ex-tractive rationales @xcite , which involve selecting a subset of input features responsible for a prediction, and (ii) natural language explanations @xcite @xcite , which generate human-readable justifications for predictions. the key aspects of interest for both ers and nles are plausibility, which measures the alignment between model explanations and ground truth, and faithfulness, which measures how accurately the explanations reflect the decision-making process of the model. ers offer concise explanations, serving as a means for users to assess the trustworthiness of a model. however, ers may lack important reasoning details, such as feature relationships @xcite . on the other hand, nles provide detailed justifications in natural language, complementing ers by potentially offering more comprehensive explanations. the evaluation of ers involves assessing their plausibility and faithfulness. plausibility refers to the extent to which a highlight explains a predicted label, as judged by human evaluators, or according to the similarity with gold highlights @xcite . faithfulness measures how accurately a highlight represents the decision process of the model -for example, by measuring to which extent the confidence in the predicted label changes after removing the highlighted words (comprehensiveness) or when only considering the highlighted words (sufficiency) @xcite . previous works largely focused on rationale extraction, which involves explaining the output of a model by identifying the input tokens that exert the greatest influence on model predictions @xcite @xcite and providing additional supervision signal @xcite . negat i v e negat i v e my poor puppy i s a depr es s my poor puppy i s a depr es s tas k los s mak e sens e??? ( pl aus i bl e) not muc h am i hones t ??? ( fai t hf ul ) my poor puppy i s a depr es s the majority of prior works in this area have revolved around explanation regularization, a technique aimed at improving generalization in neural models by aligning machine rationales with human rationales @xcite @xcite @xcite . however, ers are discrete distributions over the input text, which can be difficult to learn by neural models via back-propagation @xcite . in this work, we propose refer, an end-to-end rationale extraction framework for explanation regularization, which allows to back-propagate through the rationale extraction process. specifically, refer involves a differentiable rationale extractor, which selects the top-k% most important words from the textual input, which are then used by the model to generate a prediction. ## related works the inherent complexity of neural models has given rise to concerns regarding their opacity @xcite , particularly about the societal implications of employing neural models in high-stakes decision-making scenarios @xcite . therefore, explainability is of utmost importance for fostering trust, ensuring ethical practices, and maintaining the safety of nlp systems @xcite . learning to explain rationalization offers local explanations by providing a unique explanation for each prediction instead of a global explanation that covers the entire model @xcite . these explanations yield valuable insights for various purposes, including debugging, quantifying bias and fairness, understanding model behavior, and ensuring robustness and pri-vacy @xcite . however, obtaining direct supervision in the form of human-labeled rationales during training is not always feasible, which has led to the development of datasets that include human justifications for the true labels. these efforts enhance the interpretability of nlp models and address the limitations associated with direct supervision in learning to explain. post-hoc explanations post-hoc explanations are another branch of interpretability research. these explanations often involve token-level importance scores. in the quest for effective post-hoc explanations, a balance must be struck between the clarity of semantics and the avoidance of counterintuitive behaviors. gradient-based explanations @xcite provide clear semantics by describing the local impact of input perturbations on the outputs of the model. however, they can sometimes exhibit inconsistent behaviors @xcite , and their effectiveness relies on the differentiability of the model. alternatively, there are model-agnostic methods that do not rely on specific model properties. one notable example is local interpretable model-agnostic explanations (lime, @xcite . these approaches approximate the behavior of the model locally by repeatedly making predictions on perturbed inputs and fitting a simple, explainable model over the resulting outputs. learning from human rationales recent research has focused on leveraging rationales to enhance the training of neural text classifiers. @xcite introduced a rationale-augmented convolutional neural network that explicitly identifies sentences supporting categorizations. @xcite demonstrated that incorporating rationales during training improves the quality of predicted rationales, as preferred by humans compared to models trained without explicit supervision @xcite . in addition to integrated models, pipeline approaches have been proposed, where separate models are trained for rationale extraction and classification based on these extracted rationales @xcite . these approaches assume the availability of explicit training data for rationale extraction. extractive rationale objectives several prior works have aimed to enhance the faithfulness of extractive rationales using attribution algorithms (aas), which extract rationales via handcrafted lear ni ng f r om rat i onal es post -hoc expl anat i on lear ni ng t o expl ai n expl s i nput l abel expl s i nput l abel expl s i nput l abel functions @xcite @xcite . however, aas are not easily optimized and often require significant computational resources. @xcite ; @xcite tackle the computational cost by training a model to mimic the behavior of an aa. jain et al. (2020); yu et al. (2021); paranjape et al. (2020); bastings and filippova (2020); yu et al. (2019) ; @xcite use select-predict pipelines (spps) to generate faithful rationales. however, spps only guarantee sufficiency but not comprehensiveness @xcite , and generally produce less accurate results, since they can only observe a portion of the input, and due to the challenges associated with gradient-based optimization and discrete distributions. regarding the plausibility of the rationales, existing approaches typically involve supervising neural rationale extractors @xcite and spps @xcite @xcite ) using gold rationales. however, lm-based extractors lack training for faithfulness, and spps sacrifice task performance to achieve faithfulness by construction. other works mainly focus on improving the plausibility of rationales @xcite @xcite , often employing task-specific pipelines @xcite . in contrast, refer jointly optimizes both the task model and rationale extractor for faithfulness, plausibility, and task performance and reaches a better trade-off w.r.t. these desiderata without suffering from heuristic-based approaches (e.g., aas) disadvantages. ## model architecture task model consider f task as the task model for text classification, where it consists of an encoder @xcite and a head. let x @xmath0, and f task (x i ) ∈ r m be the logit vector for the output of the task model. we use y @xmath1 i nput ext encoder + ext head task encoder + task head sel ec t or out put figure 3 : the pipeline for explanation regularization is a fully end-to-end approach where the task model's output loss is back-propagated through all components, resulting in a compromised performance that considers all training criteria. to denote the class predicted by task model. given that cross-entropy loss is used to train f task to predict y * i , the task loss is defined as follow: rationale extractor let f ext denote a rationale extractor, such that s @xmath2. given f task , x i , and y i , the goal of rationale extraction is to output vector s @xmath3, such that each s t i is an importance score indicating how strongly token x t i influenced f task to predict class y i . the final rationales are typically obtained by binarizing s i as r (k) @xmath4, 1} n , via the top-k% strategy @xcite @xcite . to capture the degree to which the snippets within the extracted rationales are sufficient for a model to make a prediction, we measure the disparity in model confidence when considering the complete input versus only the extracted rationales. a small difference suggests the high importance of extracted rationales. following @xcite , to avoid negative losses, we can use margin m s to impose a lower bound on l suff-diff , yielding the following margin criterion: to compute comprehensiveness we create contrast examples for x i , @xmath5, which is x i with the predicted rationales r i removed @xcite . similar to equation (2), we measure the difference in model confidence between considering the complete input and the contrast set xi . a high score here implies that the rationales were influential in the prediction. repeatedly, we enforce l comp-diff to be positive as follows: finally, the selection of the tokens for matching the human highlights can be cast as a binary classification problem, and the plausibility loss is computed using the binary cross-entropy (bce) loss function: ## research questions rq1: does training the model on human highlights improve the generalization properties of the model? nowadays, machine learning systems can learn to capture spurious correlations in the data for solving any given task, and often struggle in more challenging cases @xcite . when models are allowed to make predictions without considering rationales-related criteria-faithfulness and plausibility-the rationales extracted by the model can be incomprehensible and lack meaningful interpretations @xcite . without understanding the factors and information that influence the predictions of the model, it becomes difficult to trust or explain its outputs. in certain contexts, faithful explanations are crucial -for example, they can be used to determine whether a model relies on protected attributes, such as gender or religious group @xcite . @xcite propose the hypothesis that neural natural language inference (nli) models might rely on three fallible syntactic heuristics: (i) lexical overlap, (ii) subsequences, and (iii) constituents. to evaluate whether the models have indeed adopted these heuristics, we use heuristic analysis for nli systems @xcite , which includes a variety of examples where such heuristics fail, providing a means to assess a model's reliance on these heuristics. rq2: how can we make machines imitate human rationales? human rationales are often derived from their extensive background knowledge and understanding of various concepts. while language models (lms) possess some degree of this knowledge, they face challenges in balancing between optimizing for task performance and meeting the criteria for extractive explanations. therefore, balancing plausibility, faithfulness, and task accuracy presents a challenging task. a model can reflect its inner process to make a prediction (faithful), but it may not make sense for humans (implausible). rat i onal ext r act or di f f er ent i abl e t op-k % tas k model w hat happens t o peopl e pl ay i ng games? w hat happens t o peopl e pl ay i ng games ? <i @xmath6. 121, 0. 955, 0. 433 ] tas k los s suf f los s comp los s [ 0. 098, 0. 490, 1. 231 ] [ 0. 445, 1. 115, 0. 920 ] <gol d @xmath7, " happi nes s " , " ent er t ai nment " ] <gol d rat i onal @xmath8.1, 1.4, -0.3, 0.9, 0.7, 0.8] pl aus los s rationale mask [0, 1, 0, 1, 0, 1] on the other hand, a model that returns convincing rationales (plausible) without using them during decision-making is not very useful (unfaithful). rq3: does training the model on a small number of human highlights improve its generalization properties? humans can efficiently learn new tasks with only a few examples by leveraging their prior knowledge. recent approaches for rationalizing rely on a large number of labeled training examples, including task labels and annotated rationales for each instance. obtaining such extensive annotations is often infeasible for many tasks. additionally, fine-tuning lms, which typically have billions of parameters, can be expensive and prone to overfitting. given the high cost of human annotations, a more practical approach involves incorporating a limited amount of human supervision. we investigate the characteristics of effective rationales and demonstrate that making the neural model aware of its rationalized predictions can significantly enhance its performance, especially in low-resource scenarios. rq4: do the learned rationale extractors generalize to ood data? the poor performance of models on ood datasets can stem from limitations in the model's architecture, insufficient signals in the ood training set, or a combination of both @xcite ). an nli system that correctly labels an example may not do so by understanding the meaning of the sentences but rather by relying on the assumption that any hypothesis with words appearing in the premise is entailed by the premise @xcite . @xcite raises doubts about whether models trained on the snli dataset truly learn language comprehension or primarily rely on spurious correlations, also known as artifacts. for instance, words like "friends" and "old" frequently appear in neutral hypotheses. to analyze this, we evaluate our model on contrast sets @xcite as well as unseen data, which are (mostly) label-changing small perturbations on instances to understand the true local boundary of the dataset. essentially, they help us understand if the rationale extractor has learned any dataset-specific shortcuts. ## conclusions in this paper, we propose refer, a rationale extraction framework that jointly trains the task model and the rationale extractor to optimize downstream task performance, faithfulness, and plausibility. being fully end-to-end, thanks to adaptive implicit maximum likelihood estimation @xcite , enables the task model and the rationale extractor to be jointly optimized for these criteria, therefore aware of each other behavior and adopting their parameter to improve their performance and obtain a better balance. we then analyze several aspects of the rationale extraction process, investigating how human rationales affect the model behavior; how the model can imitate human-generated rationales; and to what extent the learned models can generalize on ood datasets. finally, by answering all these questions, we compare refer performance with other methods and architectures and illustrate that our model outperforms previous models in most cases.