Contrastive Explanations for Model Interpretability (2021)

Abstract: Contrastive explanations clarify why an event occurred in contrast to another. They are inherently intuitive to humans to both produce and comprehend. We propose a method to produce contrastive explanations in the latent space, via a projection of the input representation, such that only the features that differentiate two potential decisions are captured. Our modification allows model behavior to consider only contrastive reasoning, and uncover which aspects of the input are useful for and against particular decisions. Our contrastive explanations can additionally answer for which label, and against which alternative label, is a given input feature useful. We produce contrastive explanations via both high-level abstract concept attribution and low-level input token/span attribution for two NLP classification benchmarks. Our findings demonstrate the ability of label-contrastive explanations to provide fine-grained interpretability of model decisions.

## introduction explanations in machine learning attempt to uncover the causal factors leading to a model's decisions. methods for producing model explanations often seek all causal factors at once-making them difficult to comprehend-or organize the factors via heuristics, such as gradient saliency @xcite . however, it remains unclear what makes a particular collection of causal factors a good explanation. studies in social science establish that human explanations, conversely, are typically "contrastive" @xcite : they rely on the causal factors that explain why an event occurred instead of an alternative event @xcite . such explanations pro-he has 25 years of experience. dr. ismaili is affiliated with medical center of arlington. his specialties include oral and maxillofacial surgery. he speaks english. he has 25 years of experience. dr. ismaili is affiliated with medical center of arlington. his specialties include oral and maxillofacial surgery. he speaks english. he has 25 years of experience. dr. ismaili is affiliated with medical center of arlington. his specialties include oral and maxillofacial surgery. he speaks english. he has 25 years of experience. dr. ismaili is affiliated with medical center of arlington. his specialties include oral and maxillofacial surgery. he speaks english. figure 1 : illustrative example of a biography from bios @xcite with explanations for the occupation label highlighted in yellow. explanations without an explicit contrast (1) are potentially misaligned with human expectations of what is being explained, making them confusing for human interpretability. contrastive explanations (2, 3) prune the space of all causal factors to 'intuitively' relevant ones, aiding finer-grained understanding, and can vary based on the contrast decision (e.g. accountant, surgeon). mote easier communication by pruning the space of all causal factors, reducing cognitive load for the explainer and explainee, as illustrated in figure 1. this reduction is deeply relevant: since explanations of opaque ml and nlp models are approximations of complex statistical processes, humans interpret these decisions subjectively by assuming some contrast decision. our work seeks to design explanations that are explicitly contrastive, thereby revealing finegrained aspects of model decisions, while being more representative of human comprehension ( §2). we introduce a novel framework for deriving contrastive explanations applicable to any neural classifier ( §3). our method operates on the input representation space, and produces a latent, contrastive representation. we accomplish this by project- ## contrastive explanations explanations can be considered as answers to the question: "why p ?" where p , the fact, is the event to be explained. consider the question (q1; fig. 2 ): why was it decided to hire [person x]? the explanation behind decision p (the hiring decision) comprises the causal chain of events that led to p ; a 'reasonable' answer to the question may cite that person x has relevant degrees or professional experience. however, explaining the complete causal chain (a1; fig. 2 ) is both burdensome to the explainer and cognitively demanding to the explainee @xcite . for instance, the hiring event above is caused by person x's application for the role-yet a reasonable explainer will likely omit this factor from the explanation for simplicity, thus reducing the cognitive load. but which factors should be omitted, and which should not? the theory of contrastive explanations provides a solution common in human explanations, which inherently answer the question: "why p , rather than q?" @xcite , where q (the foil) is some alternative event. 2 therefore, the explanation to the hiring decision might answer (q2; fig. 2 ): why was it decided to hire [person x], rather than not hiring them? since the decision not to hire the candidate can also be traced back to the fact that they applied to the role, the explanation can be simplified by omitting this factor (a2; fig. 2 ); this illustrates contrastive causal attribution ( §3.3) in the explanation process. similarly, given a different foil (q3; fig. 2 ): why was it decided to hire [person x], rather than hiring [person y]? the explainer might find it unnecessary to include attributes (such as a "relevant college degree") common to both person x and person y (a3; fig. 2 ). for the rest of the paper, we will refer to explanations which are not explicitly contrastive, such as a1 in fig. 2 , as non-contrastive explanations. implications for model explanations: model explanations can benefit from explicit contrastiveness in two ways. model decisions are complex and noisy statistical processes-'complete' explanations are difficult @xcite . contrastive explanations make model decisions easier to explain by omitting many factors, given the relevant foils. this reduces the burden of the explaining algorithm to interpret and communicate the 'complete' reasoning. additionally, humans tend to inherently, and often implicitly, comprehend (model) explanations contrastively. however, the foil perceived by the explainee may not match the one truly entailed by the explanation (e.g., @xcite ; contrastive explanations make the foil explicit, and are therefore easier to understand. ## contrastive explanations framework we present our framework for producing contrastive explanations. we describe the preliminaries for our framework ( §3.1), and outline an interventionist approach for causal attribution ( §3.2). next, we introduce our projection-based method to produce latent contrastive representations ( §3.3) and a measure of the resulting behavioral change ( §3.4). ## case study i: analyzing nli we apply our contrastive framework to the natural language inference (nli) task @xcite , on two datasets: multinli @xcite and @xcite . given a premise sentence, the nli task classifies if a hypothesis sentence entails, contradicts or is neutral to the premise. our experiments are based on a roberta-large model ( liu et al., 2019) ## case study ii: analyzing bios we apply our framework on the bios dataset (de-arteaga et al., 2019) containing individuals biographies, labeled with their professions and binary gender foot_11 (see ## related work the interventionist approach to causality in our work follows several recent works in nlp @xcite @xcite @xcite , and is justified by accumulating empirical evidence for the inability to draw causal interpretation from statistical associations alone @xcite @xcite . our contrastive interventions follow an amnesic operation, similar to @xcite who assess the causality of concepts, by adversarial removal of information guided by causal graphs. while we share the amnesic method, we focus on contrastive explanation, while they focused on the influence of concepts on model performance. contrastive explanations are a relatively new area in nlp. recently, @xcite proposed to derive highlights containing the portion of the input which flips the model decision; others propose similar flips via minimal edits @xcite and conditional generation @xcite . these can be viewed as other interventions orthogonal to our work, since our contrastive framework can be used to understand such interventions. additionally of interest are adversarial perturbations @xcite , which are usually implemented as gradient-based interventions. in contrast, our work relies on the identification of erasureusing linear algebra-of linear subspaces that are associated with a given concept. subspace-based interventions have the advantage of being more interpretable and controlled when compared with gradient-based interventions, which, albeit expressive, are quite opaque, not to mention of unclear efficacy @xcite . @xcite propose a model-agnostic contrastive explanation scheme based on shapley values. they offer a local explanation, unlike our global method. in addition, our approach employs behavioral interventions, while @xcite do not. others have raised concerns regarding feature importance methods based on shapley-values @xcite ; the implicit foil of such methods can be unintuitive to human explainees. in computer-vision, many have studied the generation of counterfactual explanations or counterfactual data points. freiesleben (2020) provide a unifying theoretical framework around the relationship between adversarial examples and counterfactual explanations. @xcite proposed a method that provides natural language counterfactual explanation of image classification decisions. they have relied on a model that proposes potential counterfactual evidence, followed by a verifier that is based on human-provided image description. as their method relies on pre-existing explanation model and human descriptions, there is no guarantee the explanation it provides are related to the model's reasoning process. @xcite used gans to generate examples representing minority groups, to improve fairness measures. this work, like other works in vision, relies on the continuous input, which is not present in naturallanguage applications. for more information, see @xcite for a survey of counterfactual explanations. ## conclusion we introduce a novel framework for producing contrastive explanations for model decisions, via a projection of the input representation to a contrastive space for the prediction and an alternate label. we also propose a measure of the degree of contrastive behavior, following a contrastive intervention. our experiments with english text classification benchmarks on bios and nli demonstrate our framework's ability to rank model decisions, as well as features responsible for the decision, contrastively. our quantitative and qualitative evaluations show the fine granularity of contrastive explanations, which could be useful for debugging model behavior. our framework is general enough to extend other (interventionist) explanation methods to produce contrastive explanations. contrastive explanations in nlp and ml are relatively novel; future research could explore variations of interventions and evaluation metrics for the same. this paper presented a formulation designed for contrastive relationships between two specific classes; future work that contrasts the fact with a combination of foils could explore a formulation involving a projection into the subspace containing features from all other classes.