P roto TE x: Explaining Model Decisions with Prototype Tensors (2022)

Abstract: We present ProtoTEx, a novel white-box NLP classification architecture based on prototype networks (Li et al., 2018). ProtoTEx faithfully explains model decisions based on prototype tensors that encode latent clusters of training examples. At inference time, classification decisions are based on the distances between the input text and the prototype tensors, explained via the training examples most similar to the most influential prototypes. We also describe a novel interleaved training algorithm that effectively handles classes characterized by ProtoTEx indicative features. On a propaganda detection task, ProtoTEx accuracy matches BART-large and exceeds BERTlarge with the added benefit of providing faithful explanations. A user study also shows that prototype-based explanations help non-experts to better recognize propaganda in online news.

## introduction neural models for nlp have yielded significant gains in predictive accuracy across a wide range of tasks. however, these state-of-the-art models are typically less interpretable than simpler, traditional models, such as decision trees or nearest-neighbor approaches. in general, less interpretable models can be more difficult for people to use, trust, and adopt in practice. consequently, there is growing interest in going beyond simple "black-box" model accuracy to instead design models that are both highly accurate and human-interpretable. while much research on white-box explainable models focuses on attributing parts of the input (e.g., word sequences) to a model's prediction @xcite @xcite @xcite , there is much debate around their faithfulness and reliability @xcite @xcite * both authors contributed equally. 2020). additionally, while such local explanations (if faithful) can be extremely useful in more intuitive tasks such as sentiment classification, that may not be the case for difficult tasks where human judgments may require a high degree of training or domain expertise. in such cases, understanding how models make their decisions for a particular input based on its training data can be insightful especially for engaging with users to develop an intuition on the model's decision making process. in this paper, we propose prototype tensor explainability network (prototex) foot_0 to faithfully explain classification decisions in the tradition of case-based reasoning @xcite . our novel white-box nlp architecture augments prototype classification networks @xcite with large-scale pretrained transformer language models. through a novel training regime, the network learns a set of prototype tensors that encode latent clusters of training examples. at inference time, classification decisions are entirely based on similarity to prototypes. this enables model predictions to be faithfully explained based on these prototypes, directly via similar training examples (i.e., those most similar to top-matched prototypes). we build upon the state-of-the-art nlp neural architectures to augment their accuracy with faithful and humaninterpretable explanations. figure 1 shows an example of prototex on the task of propaganda detection @xcite . another contribution of prototex concerns effective modeling of positive vs. negative classes in the presence of asymmetry. in a typical binary classification (e.g., sentiment detection), the presence of positive vs. negative language can be used to distinguish classes. however, with a task such as web search, what most distinguishes relevant vs. irrelevant search results is the presence vs. absence of relevant content. having this absence (rather than presence) of certain features most clearly dis-figure 1 : prototex architecture along with a use case demonstration. pink/green dots denote training examples, which are clustered around positive prototypes (blue) and a single negative prototype (black). dotted lines represent distances. in this use case, the user gives prototex an input, which produces a prediction while retrieving a set of highest ranked examples that directly influenced model decision. in this diagram, by using "overly aggressive or grandstanding" the input creates propaganda via exaggeration @xcite . prototex learns to identify sentences that contain propaganda phrases. in example 1, using "russian" to describe manafort (former american political consultant) constitutes propaganda, so does using "witch hunt" in example 2. exposure to similar examples helps users build an intuition towards the language used in propaganda. while our model is largely agnostic to the prediction task, we evaluate prototex on a sentence-level binary propaganda detection task @xcite . recent work on explainable fact-checking @xcite has provided explanations via attention @xcite , rule discovery @xcite , and summarization @xcite , but not prototypes. better explanations could enable support for human fact-checkers @xcite . we show that prototex provides faithful explanations without reducing classification accuracy, which remains comparable to the underlying encoder, bart-large @xcite , superior to that of bert-large @xcite , and with the added benefit of faithful explanations in the spirit of case-based reasoning. furthermore, to the best of our knowledge, we are the first work in nlp that examines the utility of global casebased explanations for non-expert users in model understanding and downstream task accuracy. ## related work explainable classification unlike post-hoc analysis approaches for explainability @xcite , prototype clas-sification networks @xcite @xcite are white-box models with explainability built-in via case-based reasoning @xcite rather than extractive rationales @xcite @xcite . they are the neural variant of prototype classifiers @xcite , predicting based on similar known instances. contemporary work @xcite also stressed the importance of "global" explainability through training examples, yet in their approach, the similar training examples are not directly integrated in the decision itself; in contrast, we do so via learned prototypes to provide more transparency. our work builds on @xcite , which we lay out in section 3.1. later work @xcite enables prototype learning of partial images. in nlp, @xcite retrieved prototype examples from the training data for edit-based natural language generation. hase and bansal (2020) used a variant of @xcite 's work to examine among other approaches; unlike our work, they used feature activation to obtain explanations similar to post-hoc approaches, and did not handle the absence of relevant content. evaluating explainability explainability is a multi-faceted problem. hci concerns include: a) for whom are we designing the explanations? b) what goals are they trying to achieve? c) how can we best convey information without imposing excessive cognitive load? and d) can explainable systems foster more effective human+ai partnerships @xcite @xcite @xcite ? on the other hand, algorithmic concerns include generating faithful and trustworthy explanations @xcite , local vs. global explanations, and post-hoc vs. self-explanations @xcite . explainability evaluation methods @xcite include measuring faithfulness (jacovi and goldberg, 2020), enabling model simulatability @xcite , behavioral testing @xcite , and evaluating intelligent user interactions @xcite . human+ai fake news detection while explainable fact-checking @xcite could better support human-in-the-loop factchecking @xcite , studies rarely assess a human+ai team in combination @xcite . in fact, hu-man+ai teams often under-perform the human or ai working alone @xcite , emphasizing the need to carefully baseline performance. propaganda detection @xcite constitutes a form of disinformation detection. because propaganda detection is a hard task for non-expert users and state-of-the-art models are not accurate enough for practical use, explainability may promote adoption of computational propaganda detection systems @xcite . ## methodology we adopt prototype classification networks @xcite first proposed for vision tasks as the foundation for our prototype modeling work (section 3.1). we design a novel interleaved training procedure, as well as a new batching process, to (a) incorporate large-scale pretrained language models, and (b) address within classification tasks where some classes can only be predicted by the absence of characteristics indicative of other classes. ## human evaluation prototex is designed to provide faithful case we obtain 540 user-responses, based on 20 testset examples, balancing gold labels and model predictions to include 5 examples from each group: true-positives, false-negatives, true-negatives, and false-positives. to simplify propaganda definitions for non-experts, we pick only four types of propaganda and we provide participants with definitions and examples for each type: appeal to authority, exaggeration or minimisation, loaded language, and doubt. we select these categories because they cover the majority of the examples in the test set. for each example, we select the top-5 prototypes that most influenced the model's prediction. we then represent each prototype by the closest training example in the embedding space. as with case-base reasoning, we explain model decisions to participants by showing for each test example the five training examples that best represent the evidence (prototypes) consulted by the model in making its prediction. participants are primed that the model is wrong in 50% of the cases (to prevent over-trust). ## conclusion prototex is a novel approach to faithfully explain classification decisions by directly connecting model decisions with training examples via learned prototypes. prototex builds upon the state of the art in nlp. it integrates an underlying transformer encoder with prototype classification networks, and uses a novel, interleaving training algorithm for prototype learning. on the challenging propaganda detection task, prototex performed on-par in classification as its underlying encoder (bart-large), and exceeded bert-large, with the added benefit of providing faithful model explanations via prototypes. our pilot human evaluation study shows that additional input provided by pro-totex contains relevant information for the task and can improve the annotation performance, provided sufficient model accuracy. we further demonstrate that explanations help non-expert users better understand and simulate model predictions.