Rather a Nurse than a Physician - Contrastive Explanations under Investigation (2023)

Abstract: Contrastive explanations, where one decision is explained *in contrast to another*, are supposed to be closer to how humans explain a decision than non-contrastive explanations, where the decision is not necessarily referenced to an alternative. This claim has never been empirically validated. We analyze four English text-classification datasets (SST2, DynaSent, BIOS and DBpedia-Animals). We fine-tune and extract explanations from three different models (RoBERTa, GTP-2, and T5), each in three different sizes and apply three post-hoc explainability methods (LRP, GradientxInput, GradNorm). We furthermore collect and release human rationale annotations for a subset of 100 samples from the BIOS dataset for contrastive and non-contrastive settings. A cross-comparison between model-based rationales and human annotations, both in contrastive and non-contrastive settings, yields a high agreement between the two settings for models as well as for humans. Moreover, model-based explanations computed in both settings align equally well with human rationales. Thus, we empirically find that humans do not necessarily explain in a contrastive manner.

## introduction in order to build reliable and trustworthy nlp applications, it is crucial to make models transparent and explainable. some use cases require the explanations not only to be faithful to the model's inner workings but also plausible to humans. we follow the terminology from @xcite and define plausible explanations as model-based rationales that have high agreement with human rationales, and faithful explanations as the input tokens most relied upon for classification. both qualities (plausibility and faithfulness) can be estimated via metrics, i.e., are not binary. recently, various contrastive explanation approaches have been proposed in nlp @xcite , 2021; @xcite where an explanation for a model's decision is provided in contrast to an alternative decision. figure 1 shows an example for text classification of professions, where the contrastive setting is phrased as: "why is the person [...] a dentist rather than a surgeon?". contrastive explanations are considered closer to how humans would argue and thus considered more valuable for humans to understand the model's decision @xcite @xcite . in particular, the latter has been shown in previous evaluation studies. so far however, it has not been empirically investigated whether contrastive explanations are indeed closer to how humans would come to a decision, i.e., whether human rationale annotations are more similar to contrastive explanations than to non-contrastive explanations. in this work, we initially compare human gaze and human rationales collected from a subset of the sst2 dataset @xcite @xcite , a binary sentiment classification task, with contrastive and non-contrastive model-based explanations using the layer-wise relevance propagation (lrp, @xcite framework. we find no difference between non-contrastive and contrastive model-based rationales in this binary setting. the analysis on dynasent @xcite yields similar results. we thus further explore the potential of contrastive explanations, collecting human rationale annotation for both settings, contrastive and non-contrastive, on a subset of the bios dataset @xcite for a five-way medical occupation classification task, and compare them to model-based explanations. we find that human annotations in both settings agree on a similar level with model-based rationales, which suggests that similar tokens are selected. contrastive human explanations seem to be more specific (fewer words annotated), but agreement between the two settings varies across classes. based on these results, we conclude that humans do not necessarily explain in a contrastive manner by default. moreover, model explanations computed in a non-contrastive and contrastive manner do not differ while both align equally well with human rationales. we observe similar findings in another single-label multi-class animal species classification dataset, dbpedia animals. note -human rationales ̸ = reasoning: as part of this work, we collect human rationales in the form of highlighting supporting evidence in the text to decide for the gold label; we show an example in figure 1 . these human rationales should be understood as proxies for how humans (annotators) explain (rationalize) a given outcome posthoc, which shall not be conflated with how humans reason, came to a decision, ad-hoc. in other words, human rationales can be only be seen as a filtered aftermath of human reasoning. hence, our observations are only suggestive of how humans explain decisions they are provided with, rather than how they come to make these decisions. the latter could possibly be examined by analyzing physiological signals, e.g., brain stimuli or gaze (eye-tracking), pre-hoc in relation to rationales. ## methodology non-contrastive explanations typically compute the most relevant features for a target class label l, e.g., by computing the gradients with respect to the logit (score) of the top-predicted class. contrastive explanations can in extension be obtained by considering the difference in evidence for the target class l and some foil class (f ̸ = l). to compute contrastive and non-contrastive explanations, we use the following methods: layer-wise relevance propagation since naive computation of gradients in transformer models has been shown to result in less faithful explanations @xcite , we build contrastive explanations in the framework of 'layer-wise relevance propagation' (lrp) for transformer models. to compute explanations that accurately reflect the model predictions, the handling of specific nonlinear model components is needed, which includes the layer normalization and attention head modules, that can be treated via carefully detaching nodes of the computation graph as part of the forward pass. for non-linear activation functions, i.e., gelu, we propagate relevance proportionally to the observed activations @xcite . gradient × input we further compute contrastive and non-contrastive explanations via 'gradient × input' @xcite , which can be seen as a special case of lrp without the use of specific propagation rules. in our setting this results to no specific treatment of non-linear computations, i.e., detaching of non-conserving modules. gradient norm in addition, computing the norm of the gradient @xcite directly has been also considered in the context of contrastive explanations by @xcite . to obtain contrastive explanations, we define the evidence to be explained as the difference: ## results we first show results for the binary sentiment classification datasets, sst2 and dynasent, before we dive into the extensive analysis of the human and model-based rationales of the bios dataset. ## discussion and conclusion in this work, we have compared both human and model rationales in contrastive and non-contrastive settings on four english text classification datasets. we find that human rationale annotations agree on a similar level within than across contrastive and non-contrastive settings but fewer tokens are selected in the contrastive settings (on average 4 vs. 8). this suggests that there is not per se a difference in token selection for the two settings but tokens are selected more carefully in the contrastive setting. the agreement varies across classes, indicating that for more challenging labels the token selection is not as straightforward as for classes that share a specific vocabulary. we further compare human rationales with model-based explanations and find no difference in agreement between the contrastive and noncontrastive setting for both models and humans on the bios dataset. on the binary sentiment classification tasks, we see similar agreement scores between noncontrastive human rationales and model explanations than for the 5-class classification task on bios. the numbers need to be compared carefully as the annotation task was different across the two datasets. for the sentiment classification task, no labels were given a-priori and we only analyze the samples where the true label agrees with the label assigned by the annotator. furthermore, the sentiment classification task is much more subjective than the occupation classification task. annotators might select tokens differently when they first had to assign a label, i.e., first assess the sentence before deciding to which class it most likely belongs. including human gaze into the analysis shows lower agreement with the model explanations in comparison to the human annotations. prior work has shown that human gaze correlates to a higher degree with attention mechanisms @xcite . in general, human gaze could be considered an alternative to human rationales when evaluating model explanations as they provide more information and the task, i.e., reading the text, might be more intuitive than assigning rationales afterwards. when comparing model-based explanations with each other, we find them to highly correlate between contrastive and non-contrastive settings. in general, our results did not show that contrastive explanations are by default more class-specific in selecting relevant tokens than non-contrastive explanations. our analysis suggests that contrastive explanations are more class-specific, i.e., focus on specifc terms for classes that share a joint set of features (similar tokens) like dentist and surgeon in the bios dataset, similar to human rationales. in line with previous work, we have seen that noncontrastive explanations are not necessarily class discriminative and that contrastive explanations can be more class specific but overall share similar features for similar classes @xcite . while we have observed a strong correlation between modelbased contrastive and non-contrastive explanations, a qualitative analysis of text samples has in parallel provided sensible examples where contrastive explanations do provide more class-specific information that deviates from the relevant features selected by non-contrastive methods. our findings suggest that contrastive explana-tions are in particular useful in generative models. in contrast to the limited differences observed in the context of few-label classification settings typically investigated, our study provides additional evidence supporting the benefit of the contrastive setting for more complex tasks. this is further supported by findings in self-explaining language models where contrastive prompting leads to explanations that are preferred by humans over noncontrastive explanations @xcite . the subtle differences between contrastive and non-contrastive explanations may provide important signal for improving ml models. previous work has shown how non-contrastive explanations provide useful information for debugging and removing undesired model behavior @xcite . in extension, contrastive and noncontrastive explanations could be useful during training to improve robustness of models and avoid shortcut learning behavior by regularizing the model to focus on more class-specific features.