Let the CAT out of the bag: Contrastive Attributed explanations for Text (2022)

Abstract: Contrastive explanations for understanding the behavior of black box models has gained a lot of attention recently as they provide potential for recourse. In this paper, we propose a method Contrastive Attributed explanations for Text (CAT) which provides contrastive explanations for natural language text data with a novel twist as we build and exploit attribute classifiers leading to more semantically meaningful explanations. To ensure that our contrastive generated text has the fewest possible edits with respect to the original text, while also being fluent and close to a human generated contrastive, we resort to a minimal perturbation approach regularized using a BERT language model and attribute classifiers trained on available attributes. We show through qualitative examples and a user study that our method not only conveys more insight because of these attributes, but also leads to better quality (contrastive) text. Quantitatively, we show that our method outperforms other state-of-the-art methods across four data sets on four benchmark metrics.

## introduction explainable ai (xai) has seen an explosion of interest over the last five years, not just in research @xcite , but also in the real world where governments @xcite and industry have made sizeable investments. the primary driver for this level of interest has been the inculcation of deep learning technologies @xcite , which are inherently black box, into decision making systems that affect billions of people. trust thus has become a central theme in relying on these black box systems, and one way to achieve it is seemingly through obtaining explanations. although many feature-based @xcite @xcite ., * 1 work done while at ibm research. given this surge of interest and its importance in recourse, in this paper, we propose a novel method contrastive attributed explanations for text (cat) which provides contrastive explanations for natural language data, a modality that has received comparatively less attention when it comes to these type of explanations. we show that our method produces fluent contrasts and possesses an additional novel twist not seen in prior works. as such, our method also outputs a minimal set of semantically meaningful attributes that it thinks led to the final contrast. these attributes could be subtopics in a dataset, different from the class labels, that characterize a piece of text or the attributes could even be obtained from a different dataset. our approach is to leverage these attributes by building models (viz. classifiers) for them and then using these classifiers to guide the search for contrasts in the original task. regarding the motivating biased chatbot example above, learning attributes that lead to contrasts can be particularly useful. gender bias, for instance, is input gyc mice cat contrast attributes movie studios to sue illegal film file traders movie studios to sue juveniles psychiatrically movie studios to sue illegal film -investors movie studios to sue illegal film tax traders +entertainment, +politics, -crypt. search providers seek video find challenges search providers seek video find videos, seek opportunities find challenges healthcare providers seek to find challenges +medicine, -wndws, -crypt. hard to locate because contrasts can modify gender on individual chats in different ways (removing pronouns, replacing gendered words with neutral, etc.). such bias can be more easily identified if the different modifications all affect a (latent) attribute that is gendered such as motherhood, which our cat explanations should highlight. to better understand what cat offers, consider the examples provided in a key novelty is that cat provides additional information in terms of characteristic attributes it thinks the contrast sentence belongs to (indicated by + sign), while also indicating characteristics it thinks are no longer applicable (indicated bysign). our method picks a few relevant attributes that guide generation of the contrast; the attributes themselves provide additional insight into the functioning of the black box model. this is confirmed through two separate user studies conducted with a total of 75 participants for which the results are reported in section 4.4. users found it easier to predict the class of the input sentence given our explanation over gyc, mice, mice with no fine tuning (mice-nft), and an ablation of our method, called cat with no attributes (cat-na), and moreover, users qualitatively preferred our method in terms of understandability, sufficiency, satisfiability and completeness on a five point likert scale. it is important to note that there are various ways to generate text using different language models (gpt-2, bert, t5, etc.) and even different techniques on how infilling might be performed (forward or bi-directional). the key idea of guiding the generation of contrasts through attributes can be considered for other recent methods, whether the contrast is learned through differentiable optimization @xcite or through combinatorial search @xcite . we note that author provided implementations of gyc and mice require models that use specific text embeddings and are not easily adaptable to other embeddings. our method cat, however, is easily adaptable, and we thus compare against the methods gyc and mice using the embeddings they are respectively implemented for in order to get stringent and fair comparisons for cat with previous state-of-the-art methods. this means we must compare against gyc and mice on different text classification models, and hence require two separate user studies, one comparing cat with gyc and the other comparing cat with mice (and we suspect this is why no comparisons are found in the literature). as such, our contributions are as follows: 1) cat introduces the idea of using attributes to drive the generation of text contrastives. this contribution is both conceptual as it brings new insight to the user as well as methodological as it leads to userpreferred contrasts as seen in the user study. 2) the cat implementation is easily adaptable to classifiers with different embeddings. 3) we qualitatively evaluate cat through examples on four different datasets from different domain tasks. 4) we quantitatively evaluate cat over other methods in terms of flip rate, content preservation, fluency, levenstein distance and efficiency. 5) we demonstrate the utility of cat through two user studies that ask users to determine a model's prediction on an instance given an explanation; cat is compared with gyc, mice, mice with no fine tuning, and an ablation of our method cat-na. ## related literature regarding the explanations of machine learning predictions on text data, a recent survey @xcite ) considered 50 recent explainability works for natural language processing, and moreover only methods that "justify predictions" rather than understanding "a model's behavior in general". our intention is also to explain individual predictions. little work has been done for global explainability with respect to text classification; @xcite suggests using various "representative" local predictions to get a global understanding of model behavior. the vast majority of explainability techniques found by @xcite fall under local explainability. local methods can be divided among post-hoc methods that explain a fixed model's prediction and self-explainable methods where the model is itself understandable; our focus is on the former. one large group of explainability methods are feature based where the explanation outputs some form of feature importance (i.e., ranking, positive/negative relevance, contributions, etc.) of the words in text @xcite @xcite @xcite . other types of local post-hoc explanations include exemplar based @xcite @xcite that output similar instances to the input. amongst local methods our focus is on contrastive/counterfactual methods @xcite @xcite that modify the input such that the class changes and explains the prediction as "if the sample were modified by x, the prediction would have been y instead," where x is a change to the input and y is a new class. such explanations are complementary to the other methods discussed above. they offer different types of intuition and thus should be used in conjunction with rather than instead of counterfactuals. @xcite learn contrasts using latent attributes, but specifically for color images. their approach does not readily translate to text as humans do not perceive minor unrealistic parts of a generated image, whereas any nuance in text is easily noticed. thus, generating (fluent) contrasts for text models is inherently much more challenging. the most relevant comparisons to our work are gyc @xcite and mice @xcite . gyc builds on the text generation of @xcite by learning perturbations to the history matrix used for language modeling that are also trained to reconstruct input from the generated counterfactuals. a diversity regularization term ensures that this does not result in counterfactuals that are identical to the input. a more recent work mice masks and replaces important words selected by @xcite in order to flip the prediction. mice requires fine-tuning their language model to each dataset, which is a significant overhead especially given the fact that we are simply generating local explanations versus our cat framework. not to mention in many real applications (sufficient) data may not be available to fine tune explanations @xcite . both gyc and mice works are targeted, i.e., the user must decide what class the counterfactual should be in as opposed to cat which automatically decides the contrast class. other recent methods are polyjuice @xcite and a contrastive latent space method @xcite . the former is a human-in-theloop method requiring supervision about the type of modification to be performed to the text such as negation, word replacement, insertion, deletion, and is not catered towards explaining a specific classifier by automatically finding the appropriate edits. the latter does not generate contrastive text but rather highlights (multiple) words in the input text that are most likely to alter the prediction if changed, where again the target class has to be provided. further, @xcite assume access to encodings from the second-to-last layer of the model being explained, and is thus not a black box method like cat. our focus being automated contrastive explanation generation, we compare with gyc and mice, i.e., methods designed towards explaining a classifier, where a valid contrast is also generated. the value of cat versus gyc and mice comes from the output of (hidden) subtopics that are added/removed from the original text to create the contrast, giving important intuition that is missing from these other methods. this is confirmed through two user studies we conduct and qualitative examples we provide. the attribute classifiers built to predict these subtopics also aid in creating better contrasts. moreover, as will be evident, such attribute classifiers can be used across datasets, thus precluding the need for each dataset to contain such attributes. furthermore, topic models or autoencoders could be used to divulge such attributes. ## proposed approach we now describe our contrastive attributed explanations for text (cat) method. contrastive explanations, convey why the model classified a certain input instance to a class p, and not another class q. this is achieved by creating contrastive examples (also called contrasts) from input instance which get predicted as q. contrastive examples are created by minimally perturbing the input such that the model prediction changes. in the case of text data, perturbations can be of three types: (1) inserting a new word, (2) replacing a word with another, and (3) deleting a word. in addition to keeping the number of such perturbations small, constrastive explainers also try to maintain grammatical correctness and fluency of the contrasts @xcite . as an example, take the case of a black box model trained on the ag news dataset that predicts which category a certain news headline falls under. given a headline, "many technologies may be a waste of time and money, researcher says" which is predicted as sci-tech, a contrastive explainer will try to explain why this headline wasn't predicted as, say, business by generating a contrastive example,"many technologies jobs may be a waste of time and money, researcher says" which is predicted as business. observe that a single word replacement achieves a prediction change. such contrastive explanations can help users test the robustness of black box classification models. we observed that even with constraints for minimal perturbation and fluency on a given black box model and an instance, there are multiple contrastive examples to choose from and, very often, many are less informative than others. for example, another possible contrast is, "many technologies may be a waste of time investment and money , researcher says" which also gets predicted as business. however, this particular explanation is not as intuitive as the previous one as "money" is a form of "investment" and the nature of the sentence has not changed in an obvious sense with the word "technologies" still present in the sentence. to alleviate this problem, we propose to construct and use a set of attribute classifiers, where the attributes could be tags/subtopics relevant to the classification task obtained from the same or a related dataset used to build the original classifier. attribute classifiers indicate the presence/absence of a certain subtopic in the text and confidence scores from these classifiers could be used as a regularization to create a contrast. we thus prefer contrasts which change attribute scores measurably as opposed to those contrasts which do not. however, at the same time, we want a minimal number of attribute scores to change so as to have crisp explanations. hence, our regularization not only creates more intuitive contrasts, but also provides additional information to the user in terms of changed subtopics which, as confirmed through our user study in section 4.4, provide better understanding of the model behavior. the important steps in our method are depicted in figure 1 . formally, given an input text @xmath0, and a text classification model f (•) which predicts @xmath1, we aim to create a perturbed instance x such that the predictions f (x) = f (x ) and x is "minimally" different from x. we use a set of m attribute classifiers ζ i : @xmath2, ∀@xmath3, . . . , m}, which produce scores indicative of presence (higher scores) or absence (lower scores) of corresponding attributes in the text. we say that attribute i is added to the perturbed sentence if word-level levenshtein distance between original and perturbed instance d lev (x , x), which is the minimum number of deletions, substitutions, or insertions required to transform x to x is used to keep the perturbed instance close to the original. the naturalness (fluency) of a generated sentence x is quantified by the likelihood of sentence x as measured by the language model used for generation; we denote this likelihood by p lm (x ). for a predicate φ, we denote 1 φ the indicator of φ, which takes the value 1 if φ is true and 0 otherwise. given this setup, we propose to find contrastive examples by solving the following optimization problem: regarding generalizability of our approach, as already noted, the attribute classifiers can be derived from other sources of data and are not necessarily dependent on the data and model being explained. furthermore, other methods could be used to obtain attributes; unsupervised methods such as lda, vaes, gans could be leveraged to ascertain semantically meaningful attributes. the attribute classifiers that appear in the loss function of eq. 1 could be replaced by disentangled representations learned by vaes @xcite or by topic models. hence, cat is generalizable beyond annotated datasets. ## conclusion in this paper, we proposed a contrastive explanation method, cat, with a novel twist where we construct attribute classifiers from relevant subtopics available in the same or different dataset used to create the black box model, and leverage them to produce high quality contrasts. the attributes themselves provide additional semantically meaningful information that can help gain further insight into the behavior of the black box model. we have provided evidence for this through diverse qualitative examples on multiple datasets, a carefully conducted user study and through showcasing superior performance on four benchmark quantitative metrics. in the future, it would be interesting to test cat on other applications with the appropriate attribute classifiers. another useful direction is generating counterfactually-augmented data (cad) using cat. if used to generate multiple contrasts, cat could offer attributional/topic diversity. since diversity in types is key to producing robust models, cat could potentially to a large degree alleviate "the lack of perturbation diversity that limits cad's effectiveness" (n. joshi, 2022).