M easuring Association Between Labels and Free-Text Rationales (2021)

Abstract: In interpretable NLP, we require faithful rationales that reflect the model’s decision-making process for an explained instance. While prior work focuses on extractive rationales (a subset of the input words), we investigate their less-studied counterpart: free-text natural language rationales. We demonstrate that *pipelines*, models for faithful rationalization on information-extraction style tasks, do not work as well on “reasoning” tasks requiring free-text rationales. We turn to models that *jointly* predict and rationalize, a class of widely used high-performance models for free-text rationalization. We investigate the extent to which the labels and rationales predicted by these models are associated, a necessary property of faithful explanation. Via two tests, *robustness equivalence* and *feature importance agreement*, we find that state-of-the-art T5-based joint models exhibit desirable properties for explaining commonsense question-answering and natural language inference, indicating their potential for producing faithful free-text rationales.

## introduction interpretable nlp aims to better understand predictive models' internals for purposes such as debugging, validating safety before deployment, or revealing unintended biases and behavior @xcite . these objectives require faithful rationalesexplanations of the model's behavior that are accurate representations of its decision process @xcite . one way towards faithfulness is to introduce architectural modifications or constraints that produce rationales with desirable properties @xcite @xcite . for example, pipeline models (figure 2 ) were designed for information extraction (ie) tasks for which a rationale can be extracted as a subset of figure 1 : a categorization of interpretable nlp on an illustrative faithfulness spectrum. two predominant forms of explanation exist that align with two predominant classes of nlp tasks. unlike models for ie tasks, the desirable properties of interpretable models for reasoning tasks have not been explored. we investigate architectures and tests for explaining reasoning tasks. the input and is sufficient to make a prediction on its own, without the rest of the input @xcite . such models approach faithfulness by construction @xcite . there is a growing interest in tasks that require world and commonsense "knowledge" and "reasoning", such as commonsense question-answering (commonsenseqa; @xcite and natural language inference (snli; @xcite . here, extractive rationales necessarily fall short-rationales must instead take the form of free-text natural language to fill in the reasoning or knowledge gap @xcite . 2 in we study two distinct model classes: self- ## shortcomings of free-text pipelines we first analyze "faithful-by-construction" pipeline models (@xmath0;@xmath1: quality of generated rationales ( §3.1) and appropriateness of the sufficiency assumption ( §3.2). ## analyzing necessary properties of joint models despite their popularity and widespread use, the extent to which self-rationalizing models exhibit faithful rationalization has not been studied. to illustrate this point, we reference narang et al. (2020): it is not infeasible that a large, overparameterized model trained on both gold-rationale emulation and a labelling task can learn to do both equally well, without having to rely on shared information in its parameters. therefore, rationales from @xmath2. at minimum, rationales must be implicitly or explicitly tied to the model's prediction. we present two metrics to analyze the association between the mechanisms that produce labels and rationales in a multi-task, @xmath3: robustness equivalence ( §4.1) and feature importance agreement ( §4.2). these experiments serve as a necessary sanity check for the reliability of @xmath4. ## conclusion after demonstrating the weaknesses that pipeline models exhibit for free-text rationalization tasks, we propose two measurements of label-rationale association in self-rationalizing models. we find that on three free-text rationalization datasets for commonsenseqa and snli, models based on t5 exhibit high robustness equivalence and feature importance agreement, demonstrating that they pass a necessary sanity check for generating faithful free-text rationales. future work can expand analysis to more properties. we believe this research direction to be important moving forward due to the advantages of large multi-task explanation models, and as a complement to development of interpretable architectures that can be fickle and task-specific. although our measurements address only necessary and not sufficient properties, by viewing faithful interpretability as a spectrum, we make a step to quantitatively situate common models on it.