F i D -Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation (2021)

Abstract: Natural language (NL) explanations of model predictions are gaining popularity as a means to understand and verify decisions made by large black-box pre-trained models, for tasks such as Question Answering (QA) and Fact Verification. Recently, pre-trained sequence to sequence (seq2seq) models have proven to be very effective in jointly making predictions, as well as generating NL explanations. However, these models have many shortcomings; they can fabricate explanations even for incorrect predictions, they are difficult to adapt to long input documents, and their training requires a large amount of labeled data. In this paper, we develop FiD-Ex, which addresses these shortcomings for seq2seq models by: 1) introducing sentence markers to eliminate explanation fabrication by encouraging extractive generation, 2) using the fusion-in-decoder architecture to handle long input contexts, and 3) intermediate fine-tuning on re-structured open domain QA datasets to improve few-shot performance. FiD-Ex significantly improves over prior work in terms of explanation metrics and task accuracy on five tasks from the ERASER explainability benchmark in both fully supervised and few-shot settings.

## introduction while large pre-trained language models @xcite @xcite with hundreds of millions of parameters have made super-human performance possible on various nlp datasets, they lack transparency into their decision making process, which can adversely affect user trust in their predictions. recent works have proposed the use of natural language (nl) rationales @xcite @xcite as a means to either obtain an understanding of the reasoning process of models, or the early jain scholar … of sanskrit. sanskrit belongs to the indo -european family of languages. it is one of the three ancient documented languages that likely arose from a common root language now referred to as ... … tasha oohed in awe. i said, "frodo's been visiting you, eh ?" malaquez said, "your pet ?" "hardly. he lives around here somewhere. i suppose he was attracted to the commotion up the hill." ... figure 1 : example questions, answers and corresponding passages from the boolq and multirc datasets from the eraser benchmark @xcite . annotated rationales are highlighted. note that rationales can be multi-sentence and non-contiguous. as a human-readable snippet for users to verify predictions @xcite . figure 1 presents examples of extractive textual rationales for two qa tasks from the eraser benchmark (deyoung et al., 2020) foot_0 . recently, @xcite show that sequence to sequence (seq2seq) models outperform previous methods at generating textual rationales for various explainability benchmarks. however, seq2seq models can fabricate rationales even for wrong predictions, are hard to scale to datasets involving several, long evidence documents, and, require large amounts of expensive rationale annotated data for training. in this paper, we introduce fid-ex, to alleviate these problems and enhance seq2seq models to achieve significant gains in rationale generation performance. @xcite find that models that generate free-form nl explanations can tailor them to convincingly justify incorrect model predictions, for example, generating "there is no dog in the image" to justify an no prediction on the image of a dog. although recent seq2seq models @xcite obtain state of the art performance on rationale generation benchmarks, they are vulnerable to having similar behaviours and can hallucinate new facts by tapping into stored world knowledge in the language model parameters. in order to retain their effectiveness and yet, alleviate the problem of explanation fabrication, fid-ex introduces the novel use of sentence markers into pre-trained seq2seq models. training seq2seq models to decode sentence marker tokens instead of explanation tokens not only guarantees the production of unaltered rationales but also significantly improves explanation metrics on five datasets (section 7). fine-tuning pre-trained models on data-rich intermediate tasks before fine-tuning on classification end tasks has recently been shown to improve endtask performance @xcite , more so in the few-shot setting. we find that this method also extends to seq2seq models, for explanation generation. we fine-tune pretrained seq2seq models to extract supporting evidence for existing open-domain qa datasets such as natural questions @xcite and @xcite , which then improves downstream performance on rationale extraction benchmarks. this approach is motivated by the similarity of the process of gathering supporting facts for qa, to that of rationale extraction for classification tasks. while earlier works on rationale generation @xcite are limited by the input passage size of pre-trained models and resort to input-passage truncation, fid-ex uses the fusion-in decoder (fid) approach @xcite , that separately encodes chunks of long passages and fuses them in the decoder, which further improves performance. ## related work deep learning models typically function as black boxes offering very little insight into their decision making mechanics. to expose model understanding at various depths, researchers have proposed various structural probing @xcite @xcite and behavioral probing methods @xcite @xcite , as well as input saliency maps to highlight the most important tokens/sentences in the input for each prediction @xcite @xcite , and input token relationships @xcite . alongside, there is work on producing textual rationales @xcite , which are snippets of nl to help explain model predictions. models may take a pipelined approach, where rationales are first selected as the sole inputs to the prediction stage, either in a supervised @xcite or an unsupervised @xcite @xcite fashion. alternatively, rationales can also serve as post-hoc supporting evidence, produced after the model prediction, as a snippet to help users verify the prediction @xcite . in this work, we improve upon seq2seq models to produce the latter kind of nl explanations, along with model predictions. in addition to extractive nl rationales obtained from subsequences of the input text, there is recent work on generating abstractive textual explanations for nlp tasks such as commonsense qa @xcite and nli @xcite . @xcite train language models to transparently output their world knowledge as nl tokens, which is then consumed by a light-weight classifier. @xcite use a generative seq2seq t5 model to produce nl explanations token-by-token for the extractive eraser benchmark, in order to take advantage of multi-task training, i.e., training for task prediction alone, or jointly with explanations if available. unlike strict input attribution based is sanskrit the first language of the world ? ## modeling in this section, we develop fid-ex, which improves upon seq2seq approaches to jointly produce nl rationales along with model predictions. we illustrate our method using the boolq dataset from the eraser explainability benchmark, which comprises of questions with passages and boolean answers (see figure 1 ), together with human annotated rationales (details in section 4). formally, given an input query q and an input passage p comprising sentences @xmath0, our goal is to produce a prediction y and rationale sentences {e k } k ## datasets in this section, we discuss the open-domain qa datasets and our pre-processing steps to prepare them for ift, as well as, the eraser rationalizing datasets that we use for evaluation. ## evaluation metrics we report exact match accuracy (em) in terms of exact token match between the predicted class label and the true label, which is equivalent to traditional classification accuracy. to evaluate the explanation quality, we report the following: rationale f1 (rf1) is an f1 score over the set of predicted explanation sentences as compared to the set of gold explanation sentences, computing set intersection based on exact sentence match. token f1 (tf1) is a token level f1 score between the predicted explanation sentence tokens and the gold explanation sentence tokens, in terms of sets of token positions, by first mapping tokens to token positions in the input passage. this is computed exactly as in @xcite , using spacy for tokenization. when using sentence markers, we map the markers back to the original sentences before computing tf1. intersection over union (iou f1) as described in deyoung et al. ( 2020 ), is computed by first matching up each predicted rationale with a gold rationale, and then computing f1. iou is similar to rf1, except that it does not use exact match. a prediction and gold sentence match if the size of the overlap of their token positions divided by the size of the union of the token positions is higher than a threshold (we use 0.5). for our models, iou f1 is very similar in magnitude to rf1. other metrics we do not use human evaluation scores since @xcite found them to be much higher than the automated metrics, and therefore, hard to interpret, in addition to being expensive and noisy. also, since we aim to provide users with evidence for model predictions, causal faithfulness metrics such as comprehensiveness and sufficiency (deyoung et al., 2020), do not apply. ## implementation details we use the fid (izacard and grave, 2020) model architecture with t5-base (220m params). we use 1024 input sub-word tokens per context for mul-tirc and 512 for the rest. we use a maximum context size of 10 for boolq and evi, and 6 for movies. we use data distributed training on machines with 8 32-gb gpus with a batch size of 8 per gpu. we train all models for 20,000 steps using adam (kingma and ba, 2014), with learning rates chosen from {1e -4 , 1e -5 } based on dev performance and use linear decay. we compute dev metrics every 500 steps and select the model with the best tf1 score. we use greedy decoding for the prediction and the explanation. the above settings are used, both for ift as well as for end-task fine-tuning. for segmenting wikipedia passages into sentences for nq, we use punkt @xcite for english from nltk. for our evaluation datasets, we used the pre-segmented and pre-tokenized input passages provided by eraser. ## results and discussion we compare the performance of different variants of our fid-ex model using all evaluation metrics on five eraser datasets, in sentence markers the addition of sentence markers leads to a large improvement in explanation metrics on all datasets as compared to generating raw tokens, demonstrating the capabilities of pre-trained seq2seq models to select scattered input markers; for the single context case, boolq tf1 improves by 4.1%, movies by 28.2%, evi by 15.2%, multirc by 4.1% and fever by 12.7%. additionally, it also provides the desirable guarantee of being extractive by eliminating the problem of fabricated rationales that seq2seq models are susceptible to (appendix b presents examples of fabrication). furthermore, while wt5 @xcite yielded a tf1 of 0 on multirc when trained on less than 10,000 examples, fid-ex obtains a tf1 of 69.8% with just 2,000 examples owing to the use of sentence markers. increased passage size using fid's multiple context encoders instead of the input truncation methods of prior work, helps significantly improve performance. when also using sentence markers, boolq tf1 improves by 7.2%, movies by 8.4% and evi by 21.1%. this is accompanied by task em gains of 8.5% in movies and 12.1% in evi. input passages in multirc and fever are not long enough to benefit significantly from increased passage size. the gains from increasing passage size are orthogonal to the gains by sentence markers, i.e., explanation metrics improve with additional context with or without using sentence markers ( ## conclusion in this paper, we develop general methods to improve the performance of large pre-trained seq2seq models for jointly producing nl rationales and answer predictions. specifically, we introduce sentence markers into seq2seq models to tackle explanation fabrication, we enable larger input passage sizes using the fusion-in-decoder architecture, and we infuse knowledge by fine-tuning on restructured qa datasets. we show that a universal model can perform favourably compared to the best task-specific fine-tuned models. our methods improve the state of the art on rationale extraction metrics and task accuracy on multiple eraser benchmarks while reducing the extent to which seq2seq models fabricate explanations to justify incorrect predictions, thereby improving the reliability and verifiability of the generated rationales.