Investigating the Benefits of Free-Form Rationales (2022)

Abstract: Free-form rationales aim to aid model interpretability by supplying the background knowledge that can help understand model decisions. Crowdsourced rationales are provided for commonsense QA instances in popular datasets such as CoS-E and ECQA, but their utility remains under-investigated. We present human studies which show that ECQA rationales indeed provide additional background information to understand a decision, while over 88% of CoS-E rationales do not. Inspired by this finding, we ask: can the additional context provided by free-form rationales benefit models, similar to human users? We investigate the utility of rationales as an additional source of supervision, by varying the quantity and quality of rationales during training. After controlling for instances where rationales leak the correct answer while not providing additional background knowledge, we find that incorporating only 5% of rationales during training can boost model performance by 47.22% for CoS-E and 57.14% for ECQA during inference. Moreover, we also show that rationale quality matters: compared to crowdsourced rationales, T5-generated rationales provide not only weaker supervision to models, but are also not helpful for humans in aiding model interpretability.

## introduction free-form rationales designed to explain decisions by providing additional world knowledge or commonsense reasoning, are key for interpretability @xcite @xcite in natural language processing tasks. 1 free-form rationales come with the promise of being easily interpretable by humans, in contrast to other kinds of explanations, such as extractive ## do crowdsourced rationales aid human interpretability? free-text rationales purportedly improve human user interpretability by explaining a model's decisions in natural language. we seek to discover which characteristics of the rationales aid users: q1 do rationales provide additional background knowledge for understanding decisions? e.g., the rationale: 'air cannot stay in any object that has a hole in it' provides additional knowledge for understanding why the answer to 'what would not be true about a basketball if it had a hole in it but it did not lose its general shape?' should be 'full of air'. q2 do rationales provide explicit clues to leak the correct answer? for comqa, this might initially seem like a helpful rationale, without really being so. 4 e.g., given a rationale: 'mexico is one of the largest coffee production country.', one can guess the correct answer should be 'mexico', when given the options 'mildred's coffee shop', 'mexico', 'diner', kitchen' or 'canteen', without looking at the question 'in what spanish speaking north american country can you get a great cup of coffee?'. ## related work rationales serve interpretability in that they can reveal the "reasoning" behind model decisions, and can be roughly categorized into two broad categories: extractive and free-form rationales. extractive rationales provide supportive evidence in a grounded context, such as textual highlights within an input document @xcite , sufficient to make a prediction on its own without relying on the rest of the input. analogous to our work, there has been a line of work that studies the value of extractive rationales as additional training signals to improve model performance @xcite or human interpretability @xcite . we use free-text rationales, on the other hand, which employ free-form natural language to fill in the commonsense reasoning or knowledge gaps. such rationales have been used for language and vision tasks @xcite but have far less adoption in nlp @xcite . concurrent to our work, @xcite provide a formal framework for free-text explanation utility for models, and a synthetic dataset for the same. in addition to the datasets used in this work, e-snli @xcite provides free-text rationales for the natural language inference task. rationale generation models can be roughly categorized into supervised @xcite @xcite @xcite and unsupervised @xcite . for supervised models, @xcite and narang et al. (2020) finetune t5 to generate extractive and free-form rationales separately. for unsupervised models, @xcite propose a differential training framework to create models that output faithful rationales without supervision. instead of directly generating rationales, @xcite use t5 to complete contrastive explanation prompts which explicitly contrast different possible answers @xcite . recent work has relied on few-shot prompting to generate explanations @xcite . our work follows a supervised approach to generate rationales, and further uses both generated and crowdsourced rationales as training signals to produce output labels. ## conclusion we investigated the utility of free-form rationales from both a human and a modeling perspective. centering our analysis on commonsense qa datasets, we find that humans perceive rationales with more background knowledge as more useful than those which simply combine the question and the answer. we provided a detailed qualitative analysis of cos-e and ecqa rationales, and found that even small amounts of carefully written rationales are helpful as additional training signals for task models. our work highlights the importance of inspecting the quality of crowdsourced rationales before using them for additional supervision. we also found that generated rationales are not as useful as crowdsourced rationales for human interpretability or for model supervision. our investigations shed light on fundamental assumptions about human interpretability in collecting and generating rationales, and call for further deeper investigation into the utility of free-form rationales.