Text Counterfactuals via Latent Optimization and S hapley-Guided Search (2021)

Abstract: We study the problem of generating counterfactual text for a classifier as a means for understanding and debugging classification. Given a textual input and a classification model, we aim to minimally alter the text to change the model’s prediction. White-box approaches have been successfully applied to similar problems in vision where one can directly optimize the continuous input. Optimization-based approaches become difficult in the language domain due to the discrete nature of text. We bypass this issue by directly optimizing in the latent space and leveraging a language model to generate candidate modifications from optimized latent representations. We additionally use Shapley values to estimate the combinatoric effect of multiple changes. We then use these estimates to guide a beam search for the final counterfactual text. We achieve favorable performance compared to recent white-box and black-box baselines using human and automatic evaluations. Ablation studies show that both latent optimization and the use of Shapley values improve success rate and the quality of the generated counterfactuals.

## introduction deep neural networks have achieved state-of-theart performances for many natural language processing (nlp) tasks @xcite . when applying such models in real world applications, understanding their behavior can be challenging -the ever increasing complexity of such models makes it difficult to understand and debug their predictions. a human can explain why an example belongs to a specific concept class by constructing a counterfactual of an example that is minimally altered but belongs to a different class. contrasting the original example with its counterfactual highlights the critical aspects signifying the concept class. we study a similar approach to understand deep nlp models' classification criteria. given a classifier and an input text, our goal is to generate a counterfactual by making a set of minimal modifications to the text that change the label assigned by the classifier. additionally, our goal is to understand the model's behavior when processing naturally occurring inputs, hence we wish to generate grammatically correct and semantically plausible counterfactuals. automatic generation of text counterfactuals has been studied in different settings. @xcite considered counterfactual story rewriting which aims to minimally rewrite an original story to be compatible with a counterfactual event. @xcite used a fine-tuned gpt-2 model to generate general purpose counterfactuals that are not tied to a particular classification model. @xcite aim to generate plausible-sounding counterfactuals that flip a classification model's decision for financial texts. related, textual adversaries also aim to change the model prediction (with modifications resembling natural text). the difference is that adversaries further aim to escape human detection (not changing a human's classification), whereas counterfactuals do not have such requirement. another line of related work is style transfer @xcite @xcite , which aim to modify a given text according to a target style. it differs from adversary or counterfactual generation in that it seeks to fully change all style-related phrases, as opposed to minimally perturbing a text to change a classifier's decision. white-box approaches have been widely used to generate adversaries or counterfactuals for vision tasks where the continuous inputs can be optimized to alter model predictions @xcite @xcite . such optimization based approaches are difficult to apply to language due to the discrete nature of text. we circumvent this difficulty by directly optimizing in the latent space of the input towards the desired classification. we then exploit the language generation capability of pre-trained language models, available for most state-of-the-art nlp models such as bert @xcite or roberta @xcite , to generate semantically plausible substitutions from the optimized latent representations. we further introduce shapley values to estimate the combinatoric effect of multiple simultaneous changes, which are then used to guide a beam search to generate the final counterfactual. leveraging pre-trained language models to generate alternative texts has been a popular black-box approach in the recent literature on text adversaries @xcite @xcite . our work presents a first attempt to combine the strength of white-box optimization and the power of pre-trained language models. while shapley values have been widely studied for the problem of feature importance @xcite and data valuation @xcite , this is the first effort demonstrating their usefulness for text generation. we compare our method to several white-box and black-box baselines on two different text classification tasks. automatic and human evaluation results show that our method significantly improves the success rate of counterfactual generation, while reducing the fraction of input tokens modified and enhancing the semantic plausibility of generated counterfactuals. we also show through ablation studies that both counterfactual optimization of the latent representations and shapley value estimates contribute to our method's strong performance. ## proposed method problem statement. we are given a text classification model, m , an initial input token sequence, @xmath0, ..., x n-1 }, with vocabulary v . model m outputs a classification score ŷ = m (x) ∈ (0, 1), representing p (@xmath1. based on the score, a class label @xmath2, 1} is assigned. we seek to generate a counterfactual of x, which is defined as a set of tokens, @xmath3, ..., x n-1 }, that differs from x in no more than c max percent of locations, is grammatically plausible, and leads to a different classification, y . here c max is an input parameter for maximum changes allowed, and smaller c max imposes stronger restrictions. note that our setup assumes binary classification, but can be easily extended to multi-class scenario to generate either targeted (with specified y ) or un-targeted counterfactuals (with unspecified y ). ## empirical evaluation to evaluate our proposed method, we consider two different text classification tasks: sentiment classification and natural language inferences. ## human and qualitative evaluation human evaluations. for human evaluations, we choose to compare closs against bert-attack and hotflip o, the two baselines performing the best in perplexity and flip rate respectively. we randomly selected 100 original texts from imdb for evaluation with the restriction that all three methods must successfully flip the classification changing 15% or less of the original tokens. additionally, we exclude texts with more than 50 tokens to ease the burden on evaluators. using the bert classifier, we apply bert-attack, closs and hotflip to generate counterfactuals for each input. eight human evaluators are each assigned 25 original texts and asked to rank (ties allowed) the three counterfactuals in order of grammatical correctness. each input is evaluated by two evaluators, the inter-evaluator agreement per pairwise comparison is 75.4%. human evaluators ranked closs competitively with bert-attack and hot-flip, assigning average ranks of 1.54 to closs, 1.68 to bert-attack and 2.50 to hotflip. the difference between closs and hotflip is statistically significant (one-sided sign test, p-@xmath4.0001). qualitative analysis of generated text. inspecting the generated coutnerfactuals, we observe some interesting patterns, summarized below. see appendix (tables 5 and 6 ) for specific examples. for the imdb dataset, closs often changes one or two sentiment words while the rest of the input still supports the original prediction. this suggests that the model may be triggered by a few sentiment words, ignoring most input. identifying such critical substitutions will allow us to inspect the patterns of these "triggers" to reveal the weakness of the classifier. we also observe that when the model misclassifies, it often takes little change to correct the model, which helps debug the mistake. sometimes closs introduces synergistic changes where each change's capacity to influence the classification seems contingent on the other. finally, closs sometimes distorts sentimentphrases into non-words to remove their impact on classification, possibly making up for the lack of ability to remove words. for the qnli dataset, unsurprisingly, we note that changing from entailment to non-entailment is far easier than the opposite (see figures 5(c , d ) in appendix), and often requires changing only a few words shared by the question and context. conversely, closs can sometimes change non-entailment to entailment by introducing some shared word(s). this suggests that the model relies heavily on overlapping words to decide entailment. more detailed analysis can be found in the appendix, including how closs's changes are distributed among part of speech tags (figure 4 ) and a failure analysis for closs (a.7, ## conclusion we are motivated by how humans use counterfactuals to explain the concept of a class and seek to automatically generate counterfactual text input as a means to understand a deep nlp model and its definition of class. we assume full white-box access to the given model and perform optimization in the latent space to maximize the probability of predicting a target class. we then map from the optimized latent representation to candidate token substitutions using a language model. a key novelty of closs is using shapley values to estimate the potential of a token substitution in changing the model's prediction when used in combination with other substitutions. the shapley value is then used to guide a breadth-first beam search to generate the final counterfactual. through both automatic and human evaluations, we show that closs achieves highly competitive performance both in terms of the success rate of generating counterfactuals as well as the quality of the generated counterfactuals. our approach has several limitations. as a whitebox approach, we require full access to the model, which can be restrictive in practical applications. our approach currently only considers substitutions, excluding deletions and insertions. finally, our method is only applicable to models that are based on pre-trained language models. future work will adapt closs to adversarial and black box settings. we also hope to improve the efficiency of closs via more efficient shapley value estimation @xcite . mukund sundararajan and amir najmi. 2020. the many shapley values for model explanation. ke wang, hang hua, and xiaojun wan. 2019. controllable unsupervised text attribute transfer via editing entangled latent representation. tongshuang wu, marco tulio ribeiro, jeffrey heer, and daniel s. weld. 2021. polyjuice: automated, general-purpose counterfactual generation. linyi yang, eoin m. kenny, tin lok james ng, yi yang, barry smyth, and ruihai dong. 2020. generating plausible counterfactual explanations for deep transformers in financial text classification. yuan zang, fanchao qi, chenghao yang, zhiyuan liu, meng zhang, qun liu, and maosong sun. 2020. word-level textual adversarial attacking as combinatorial optimization. in proceedings of the 58th annual meeting of the association for computational linguistics, pages 6066-6080, online. association for computational linguistics.