Generating Realistic Natural Language Counterfactuals (2021)

Abstract: Counterfactuals are a valuable means for understanding decisions made by ML systems. However, the counterfactuals generated by the methods currently available for natural language text are either unrealistic or introduce imperceptible changes. We propose CounterfactualGAN: a method that combines a conditional GAN and the embeddings of a pretrained BERT encoder to model-agnostically generate realistic natural language text counterfactuals for explaining regression and classification tasks. Experimental results show that our method produces perceptibly distinguishable counterfactuals, while outperforming four baseline methods on fidelity and human judgments of naturalness, across multiple datasets and multiple predictive models.

## introduction the increase of machine learning (ml) applications in high-stakes domains has led to a proliferation of explainable ai (xai) and interpretable ml approaches, aimed at making models (global explanations) or individual decisions (local explanations) more understandable @xcite . output explanations explain individual decisions by understanding the (local) behavior around the output @xcite . however, in practice individuals may not always have access to the models they want explained (e.g. because of intellectual property) @xcite . to overcome this access problem, model-agnostic approaches (sometimes called post-hoc approaches @xcite ) only require access to the model outputs for provided instances, with the added benefit of being applicable to explain any model for a type of ml task @xcite . prominent model-agnostic output explanations are local surrogate models @xcite , feature importances @xcite , example-based explanations @xcite and counterfactual explanations @xcite . counterfactual explanations express what might have happened instead @xcite : certain values in an input instance are perturbed (e.g. the age of a defendant) while keeping other values the same, in order to observe how that influences the output (e.g. they would not have been convicted). each of the output-changing perturbations is a counterfactual, where the difference between the counterfactual and the original instance provides insights into how the inputs affect the outputs, and can be used to pinpoint fairness issues and to reach a desired output @xcite . as these counterfactuals are a valuable means to understand the behavior of a system, in recent years the same technique has been applied for explaining ml decisions-mainly for structured data (e.g. @xcite @xcite ) and image data (e.g. @xcite @xcite ). unlike structured and image data, counterfactuals for natural language text data have largely been disregarded. for text classification, @xcite proposed the removal of words as a means to measure their contribution to the output. this paradigm was later adopted for constructing model-agnostic local surrogate models @xcite and to determine which words have to be necessarily present for a classification decision @xcite . yet, this paradigm fails to create realistic counterfactuals (as illustrated in figure 1 for sentiment analysis). realisticness is an important property for how humans create and accept counterfactuals @xcite and may help prevent misleading explanations produced by model-agnostic explanation methods @xcite . recently, human-inthe-loop approaches were proposed @xcite to support explainees in forming realistic counterfactuals. in this work, we propose counterfactualgan: a method able to model-agnostically generate realistic, targeted counterfactuals for natural language text regression and classification without explainee intervention. our method (i) generates realistic counterfactuals for the text-domain-ensuring dataset-specific realisticness by adversarially training on a training set-, (ii) uses a single model to provide counterfactuals for any instance and any (classification or regression) target of a black-box model, (iii) generates counterfactuals with a single pass after training, and (iv) does not require explainee intervention to do so. ## realistic counterfactuals for natural language text, we propose a new strategy to create plausible counterfactuals in an automated manner, by applying a realisticness constraint to the generated counterfactuals. this approach has several benefits over the strategies tackling implausibility in section 2.1: (i) no domainspecific assumptions have to be made by users, (ii) the perturbation path between an original instance and its counterfactual is not constrained and (iii) counterfactuals are not restricted to only training instances. what realisticness entails may depend on the type of language expected within a certain context, e.g. realistic movie reviews typically differ in use of language and grammatical correctness from tweets. therefore, we deem an instance realistic if it is indistinguishable from other instances (expected) in a dataset from a specific domain. definitions. let us assume we are able to provide inputs to a black-box ml model f : @xmath0, and have an instance @xmath1. f (•) was trained on a dataset containing n labeled or unlabeled instances (represented by @xmath2, y) = {(x i , y i )} n @xmath3, respectively). instance x is represented as a feature vector @xmath4, x 2 , ... , x n ), where each x denotes a feature value. in our work, x i is either a word-level token or is absent. a second index j (x i,j ) is used sometimes when tokens are represented by a probability distribution over a vocabulary of m tokens (including a special token indicating absence). a perturbed instance @xmath5. a valid perturbation transforms the feature values in x such that @xmath6. for example, valid perturbations are word replacements or removing a word by setting it to absent. we estimate instance realisticness by determining if it is indistinguishable from the original data distribution p(x). in practice we estimate this indistinguishability with a discriminator model g x : @xmath7, 1] (trained on x) indicating the likelihood that an instance x could have come from p(x). a low score indicates out-of-distribution instances. we define the set of counterfactuals (cfs) of the instance x as cf f (x) = { @xmath8., 2020), i.e. all instances with an output different to x. targeted counterfactuals (tcfs) are instances that change the fact (the current output @xmath9, to the foil, an output of interest @xmath10. instead of requiring the output f ( x) of perturbed instance x to be exactly equal to y , we generalize the assumption f ( x) = y and obtain: ## method: counterfactualgan we operationalize model-agnostic, realistic, targeted counterfactuals for text regression and classification with our proposed method counterfactual-gan. our method works across predictive models by crafting counterfactuals in the form of a string. to ensure counterfactual realisticness, we combine insights from pretrained language models (lms) and generative adversarial networks (gans). ## experiments counterfactualgan was evaluated against four baselines using a quantitative validation and a human evaluation in the form of a user experiment. ## conclusion in this paper, we proposed counterfactualgan: a counterfactual generation method providing realistic counterfactuals to explain natural language text regression and classification black-box models, using a combination of pretrained lms and a stargan to craft counterfactuals. experimental results showed that our counterfactual generation method outperforms baselines across predictive counterfactualgan greatly improves natural language counterfactuals' quality, potentially having a profound effect on the explanation quality of model-agnostic xai methods using perturbations to form explanations (e.g. local surrogates @xcite and counterfactual explanations @xcite ). for future work, we intend to (i) assess for which xai methods and in which contexts realisticness is most beneficial, (ii) determine what level of perceptibility is optimal for human-understandable explanations, and (iii) extend counterfactualgan to other languages than english and more ml task types (e.g. multi-label classification).