SELFEXPLAIN : A Self-Explaining Architecture for Neural Text Classifiers (2021)

Abstract: We introduce SelfExplain, a novel self-explaining model that explains a text classifier’s predictions using phrase-based concepts. SelfExplain augments existing neural classifiers by adding (1) a globally interpretable layer that identifies the most influential concepts in the training set for a given sample and (2) a locally interpretable layer that quantifies the contribution of each local input concept by computing a relevance score relative to the predicted label. Experiments across five text-classification datasets show that SelfExplain facilitates interpretability without sacrificing performance. Most importantly, explanations from SelfExplain show sufficiency for model predictions and are perceived as adequate, trustworthy and understandable by human judges compared to existing widely-used baselines.

## introduction neural network models are often opaque: they provide limited insight into interpretations of model decisions and are typically treated as "black boxes" @xcite . there has been ample evidence that such models overfit to spurious artifacts @xcite @xcite and amplify biases in data @xcite . this underscores the need to understand model decision making. prior work in interpretability for neural text classification predominantly follows two approaches: (i) post-hoc explanation methods that explain predictions for previously trained models based on model internals, and (ii) inherently interpretable models whose interpretability is built-in and optimized jointly with the end task. while post-hoc methods @xcite @xcite are often the only option 1 code and data is publicly available at https:// github.com/dheerajrajagopal/selfexplain ## selfexplain let m be a neural c-class classification model that maps @xmath0, where x are the inputs and y are the outputs. selfexplain builds into m, and it provides a set of explanations z via highlevel "concepts" that explain the classifier's predictions. we first define interpretable concepts in §2.1. we then describe how these concepts are incorporated into a concept-aware encoder in §2.2. in §2.3, we define our local interpretability layer (lil), which provides local explanations by assigning rel-evance scores to the constituent concepts of the input. in §2.4, we define our global interpretability layer (gil), which provides global explanations by retrieving influential concepts from the training data. finally, in §2.5, we describe the end-to-end training procedure and optimization objectives. ## dataset and experiments experimental settings: for our selfex-plain experiments, we consider two transformer encoder configurations as our base models: (1) roberta encoder @xcite ) -a robustly optimized version of bert @xcite . classification results : we first evaluate the utility of classification models after incorporating 6 https://cogcomp.seas.upenn.edu/data/qa/qc/ gil and lil layers in we also observe that the hyperparameter k did not make noticeable difference. additional ablation experiments in model accuracy xlnet-base 93.4 selfexplain-xlnet + lil 94.3 selfexplain-xlnet + gil 94.0 selfexplain-xlnet + gil + lil 94.6 roberta-base 94.8 selfexplain-roberta + lil 94.8 selfexplain-roberta + gil 94.8 selfexplain-roberta + gil + lil 95.1 ## explanation evaluation explanations are notoriously difficult to evaluate quantitatively @xcite . a good model explanation should be (i) relevant to the current input and predictions and (ii) understandable to humans @xcite @xcite . towards this, we evaluate whether the explanations along the following diverse criteria: from selfexplain, we extracted (i) most relevant local concepts: these are the top ranked phrases based on r(nt) 1:j from the lil layer and (ii) top influential global concepts: these are the most influential concepts q 1:k ranked by the output of gil layer as the model explanations to be used for evaluations. ## analysis does selfexplain's explanation help predict model behavior? in this setup, humans are presented with an explanation and an input, and must correctly predict the model's output @xcite @xcite . we randomly selected 16 samples spanning equal number of true positives, true negatives, false positives and false negatives from the dev set. three human judges were tasked to predict the model decision with and without the presence of model explanation. we observe that when users were presented with the explanation, their ability to predict model decision improved by an average of 22%, showing that with selfexplain's explanations, humans could better understand model's behavior. ## conclusion in this paper, we propose selfexplain, a novel self-explaining framework that enables explanations through higher-level concepts, improving from low-level word attributions. selfex-plain provides both local explanations (via relevance of each input concept) and global explanations (through influential concepts from the training data) in a single framework via two novel modules (lil and gil), and trainable end-toend. through human evaluation, we show that our interpreted model outputs are perceived as more trustworthy, understandable, and adequate for explaining model decisions compared to previous approaches to explainability. this opens an exciting research direction for building inherently interpretable models for text classification. future work will extend the framework to other tasks and to longer contexts, beyond single input sentence.