Why do you think that? Exploring Faithful Sentence-Level Rationales Without Supervision (2020)

Abstract: Evaluating the trustworthiness of a model’s prediction is essential for differentiating between ‘right for the right reasons’ and ‘right for the wrong reasons’. Identifying textual spans that determine the target label, known as faithful rationales, usually relies on pipeline approaches or reinforcement learning. However, such methods either require supervision and thus costly annotation of the rationales or employ non-differentiable models. We propose a differentiable training–framework to create models which output faithful rationales on a sentence level, by solely applying supervision on the target task. To achieve this, our model solves the task based on each rationale individually and learns to assign high scores to those which solved the task best. Our evaluation on three different datasets shows competitive results compared to a standard BERT blackbox while exceeding a pipeline counterpart’s performance in two cases. We further exploit the transparent decision–making process of these models to prefer selecting the correct rationales by applying direct supervision, thereby boosting the performance on the rationale–level.

## introduction large pre-trained language models, such as bert @xcite or roberta @xcite gain impressive results on a large variety of nlp tasks, including reasoning and inference @xcite . despite this success, research shows that their strong performance can rely, to some extent, on dataset-specific artifacts and not necessarily on the ability to solve the underlying task @xcite @xcite . thus, these observations undermine the models' trustworthiness and impede 1 code available at https://github.com/ukplab/ emnlp2020-faithful-rationales lorem ipsum dolor sit amet, consectetur adipisici elit, sed eiusmod tempor incidunt ut labore et dolore magna aliqua. robbie collins is a british film critic. ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquid ex ea ommodi consequat. robbie collins is a british film critic. ## related work understanding the deep neural networks' decisions has gained increasing interest in the research community @xcite @xcite . several works are concerned with post-hoc techniques to explain decisions of blackbox models @xcite @xcite . visualizing attention weights has been heavily used, but is known to be insufficient @xcite . other works focus on making the models themselves more interpretable via neural module networks @xcite , graph-based networks @xcite , pipeline models @xcite , or by generating textual explanations @xcite @xcite . rather than only producing this explanation as additional output, @xcite base the target prediction on this automatically created hypothesis. some approaches jointly use rationales to explain the predictions and boost performance without ensuring faithfulness @xcite @xcite . recent work use gumbel softmax (maddison et al., 2016) fever claim joan crawford has had four marriages. (supports) document [...] following a public appearance in 1974 , after which unflattering photographs were published , crawford withdrew from public life and became increasingly reclusive until her death in 1977 . (r1) crawford married four times . (r2) her first three marriages ended in divorce ; the last ended with the death of husband alfred steele . crawford 's relationships with her two older children , christina and christopher , were acrimonious . [...] multirc question what are we seeing when we see lightning ? answer the discharge of electrons (true) document [...] over time the differences increase . (r1) eventually the electrons are discharged . this is what we see as lightning . you can watch an awesome slow -motion lightning strike below . [...] to identify token-level rationales to avoid using reinforce @xcite . very recent work @xcite aims similarly to us, to infer faithful rationales based on its impact on the target prediction without supervision, thereby relying on a dedicated explanation technique to identify rationales and an additional model for the prediction. this work is different in that we (a) rely on the same network weights for rationale selection and target prediction, and (b) provide quantitative analysis about the decision criteria of the models on the reasoning tasks. ## results all experiments use allennlp @xcite and bert-base-uncased @xcite as provided by @xcite . we manually metrics as opposed to deyoung et al. (2020) we choose sentences as the lexical unit for rationales. we report precision, recall, and f1 for the rationales rather than token-level iou, to avoid that the length of sentences impacts the metrics 3 . as we are interested to understand whether a model makes the right prediction for the right reasons, we focus on sufficiency of selected rationales rather than comprehensiveness: the claim of fever in figure 2 shows two valid rationales. only one of these is required to support the claim. to compute precision, recall, and f1 w.r.t. sufficiency, we, therefore, compute these metrics based on the single, most similar 4 gold-rationale when evaluating any of the models. we additionally report the joint accuracy of the target task and the rationale. here we con3 to simplify comparisons with future work, we report the original eraser metrics in appendix a. observations the target columns in the closest measure for "right for the right reasons" is represented by acc. full. yet, it can only measure whether the prediction is based on the correct rationale on a sentence level, whereas it may still solely rely on certain contained words. assuming comprehensive rationale annotations foot_1 , the opposite can be said, i.e., 92.9% of multirc are not classified correctly for the right reasons. note that both, the single-sentence models and the bert-to-bert pipeline, are bound to reach an 0% for acc. full on multirc, since they can only select a single sentence as the rationale. ## analysis leveraging the information about the used rationales, we closer analyze decision criteria for fever and multirc, and why our method performed poorly on movies. further, except for the two-sentence s models on multirc, no other model selects two sentences as a rationale in more than 1.3%. we partly attribute this to the less-thanoptimal aggregation via max-pooling. as these are only selected due to the additional supervision, not for the utility to solve the overall task 6 , we focus on single-sentence models. ## discussion limitations from a technical perspective, a limitation is memory consumption, as the model must process all rationale candidates of the same instance within the same minibatch. though single-sentence rationale can be processed, encoding all combinations of multiple sentences via bert is problematic. future work could investigate better sampling strategies or a greedy breadth search to reduce the number of candidates. another limitation is the inability of coreference resolution between different sentences and the consideration of the context in general. solving this is non-trivial, as we essentially buy faithfulness by explicitly omitting all other information than the selected sentence(s). while this does not seem crucial in the evaluated datasets, it poses potential dangers for malicious attacks, most importantly, when considering the permutations of multiple sentences. therefore, we recommend to always show the identified evidence in context when using our approach in the real world. conclusion we proposed a conceptually simple approach to allow models to extract faithful rationales, which can compete with standard bert on two reasoning tasks without supervision and even improve the overall performance, when supervising on the rationale. we showed that by outputting faithful rationales, it is possible to not only compare models based on the target performance alone, but also quantify how well even those correct predictions are based on the correct evidence. our analysis showed that exploiting this knowledge about the selected rationales helps shed light on the models' the decision mechanism for debugging purposes and on the underlying data. sentences of model s, i.e., the selected rationale with the highest confidence score, we now look at the predictions of both selected sentences individually.