Making a (Counterfactual) Difference One Rationale at a Time

1 Introduction . Research in neural model interpretability has been cast as important and received significant recent attention [ 21 ] . Within the field of natural language processing ( NLP ) , rationales have been a popular method for providing interpretability in the form of extracted subsets of text [ 10 ] . Rationale models typically consist of two cooperating modules where one module , the `` rationale selector '' , selects the rationale from a source document , and the other module , the `` classifier '' , acts on only the selected rationale without seeing the rest of the document . There is interpretability through sparsity and exclusivity . A common approach for training these rationale models is based on the maximum mutual information criteria ( MMI ) [ 7 ] . With the MMI criteria , rationale selectors seek the subset of text that carries the most information about the target label . Often , sparsity and coherency constraints are used to keep the rationales interpretable . Within many datasets , however , spurious patterns and co-varying aspects can cause the rationale selector to pick up on patterns that do not capture a desired relationship between input text and target labels . As a result , the rationalized model can have undesirable behaviour like predicting a hotel is very clean because it is in a convenient location . Nonsensical rationales or 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . explanations might decrease trust in the model , and in some cases , suggest the model might generalize poorly [ 26 ] . In this work , we propose a general counterfactual data augmentation ( CDA ) [ 22 ] approach to aid rationale models trained with MMI . We show that theoretically our CDA approach can effectively improve the performance of rationale selectors by lowering the mutual information between spurious signals and aspects of interest . Empirically , we show that models trained on our CDA datasets learn higher quality rationales than those trained on the original dataset when both use the same MMI criteria . More importantly , the most significant advantage of our CDA approach is that it does not require human intervention . We use rationales from an initial , noisy model and replace them with new text that changes the target label using a generative neural model . In this way , our CDA approach is completely hands off and does not need input from human experts or crowd workers . We first show that in the extremely ideal scenario where the initial rationale selector is perfect , our CDA approach can eliminate the mutual information between spurious signals and the target label . Next , we show in the common , realistic scenario where the rationale selector is noisy and imperfect , our CDA approach can still yield gains . Finally , the effectiveness of our CDA approach is compared against several baselines including an improved MMI-based rationale schema [ 19 ] on two common multi-aspect review datasets , TripAdvisor [ 29 ] and RateBeer [ 24 ] . Multi-aspect datasets are our main focus as they are guaranteed to contain spurious patterns and co-varying aspects ; we primarily used MMI-based baselines because the fundamental goal our CDA approach is to reduce the mutual information between spurious signals and aspects of interest . 2 Counterfactual Data Augmentation and Multi-Aspect Datasets . 2.1 Definitions and Notations . We use upper-case letters to denote random variables , X and Y , and lower-case letters to denote samples from these variables , x and y. I ( X , Y ) marks the mutual information between X and Y . Mutual information is defined as the reduction in uncertainty of a random variable due to knowledge in another random variable , I ( X , Y ) = H ( Y ) − H ( Y |X ) , where H ( X ) and H ( Y |X ) are the Shannon entropy and conditional entropy respectively [ 8 ] . 2.2 Problem Formulation . This work follows the same rationale concept introduced in [ 19 ] . Specifically , one neural module extracts text from a document and another neural module classifies the extracted text . Later it has been shown that the rationalization criteria aims `` to maximize the mutual information between selected features and the response variable '' [ 7 ] defined as : max G I ( XM ; Y ) subject to M ∼ G ( X ) ( 1 ) where M denotes a binary rationale mask over the input produced by a rationale selector G. That is , the goal of the rationale selector is to select the subset of features in X that are most informative of the label Y under some constraints defined by the selector G. In this work , our selector constraints affect the size and coherency of the rationales . Here we consider a multi-aspect dataset , D with features X and labels Y : X is a set of features , or a sequence of words in NLP , and Y is a vector , possibly one dimensional , of numerical scores . In a multi-aspect dataset , a single document can discuss multiple attributes of a single object . For example , a single beverage review might discuss its appearance , taste , and smell . We assume some subset of the features , X1 , belong to the target aspect label , Y1 , while other features , X2 , are spurious or non-causal . These features could belong to other aspects or be artifacts of the dataset . In the following , for simplicity , we will use < X , Y1 > or < X1 , X2 , Y1 > interchangeably based on the context . Our goal is to estimate or model the score for aspect Y1 by using the function f given only X1 , as follows : Y1 = f ( X1 ) . There is one such model corresponding to each < X1 , Y1 > pair . In this work , we use multi-aspect datasets , and model each aspect individually . This simulates the more common case where a dataset has a single output label of interest and all signals in the dataset that do not pertain to that label are considered spurious or belonging to other , not-estimated aspects . When predicting Y1 , our ideal model would focus onX1 and ignore all other features , X2 . Specifically , for a given sample < x , y > , our selector would select a subset of x such that x1 = xM . A rationale selector might fail to extract x1 effectively for two primary reasons : first , Chen et al . [ 7 ] showed it is intractable to find a solution to Eqn 1 and thus they derive a variational approximation with Monte Carlo based gradient estimates . Second , datasets can contain artifacts such that spurious patterns might contain significant mutual information with the target label . Along the same lines but even more concerning , other aspects in a dataset might be highly correlated with the target label . For example , beers that smell good usually taste good as well . Since the goal of the selector is to select features that maximize I ( XM , Y1 ) , we can assist the selector in finding X1 by lowering the mutual information between the other spurious features and the desired label , I ( X2 , Y1 ) . Drawing on ideas from [ 22 ] , we do this through a general counterfactual data augmentation ( CDA ) scheme where , in the counterfactual dataset , superscript c , we flip the label of the document from Y1 to Y c1 and replace the text selected by the rationale selector , X1 , with an inference , Xc1 , generated by a class conditioned masked language model ( MLM ) using Y c 1 and X2 described as : Y c1 ← 1− Y1 ; Xc1 ← arg max X1 p ( X1|1− Y1 , X2 ) ( 2 ) In our generated counterfactual dataset < Xc1 , X2 , Y c 1 > , X c 1 is the newly generated counterfactual , X2 is the original spurious feature set , and we assign Y c = 1− Y1 . We will show , in the augmented dataset which is the concatenation of datasets < X1 , X2 , Y1 > and < Xc1 , X2 , 1 − Y1 > , we are lowering the mutual information I ( X2 , Y1 ) . We will use superscript a for the augmented dataset : Da = < Xa1 , X a 2 , Y a 1 > . 2.3 Lowering I ( X2 , Y1 ) , an idyllic case Take a dataset of beer reviews where each document contains a description of the taste and smell of a beer as well as a numerical score for only the smell aspect . The task is to estimate the smell score while using the smell text as the rationale . Figure 1 demonstrates our process . An example of a concise document in this dataset might be This beer smells great . It tastes terrific . In this document x1 is the phrase smells great and x2 is tastes terrific . Both have positive sentiment but our only label for this document is y1 = 1 for the smell sentiment . Our counterfactual document could be This beer smells awful . It tastes ter- rific and our label becomes y1 = 0 . We can see that in the augmented dataset the phrase tastes terrific maps to both positive and negative labels and therefore p ( Y1|X2 ) = p ( Y1 ) . Finally , I ( X2 , Y1 ) is 0 and I ( X1 , Y1 ) is unchanged . With perfect knowledge of the ground truth rationales and the process Xc1 ← p ( X1|1− Y1 , X2 ) , we can craft counterfactual documents and therefore a counterfactual dataset that perfectly eliminates I ( X2 , Y1 ) while preserving I ( X1 , Y1 ) . However , the challenge is that ground truth rationales are not provided in the training data . Therefore , it is important for us to show that even when rationales selected by the initial rationale selector are noisy and imperfect , we can still lower I ( X2 , Y1 ) in the augmented dataset and benefit subsequent models trained with MMI . 2.4 Dealing with a Noisy Initial Selector . Here we are working on the augmented dataset : Da = < Xa1 , X a 2 , Y a > . To completely eliminate I ( Xa2 , Y a 1 ) , our CDA approach requires a perfect rationale selector . We were originally motivated by improving a poor rationale selector , so here we track what happens to both I ( Xa2 , Y a 1 ) and I ( X a 1 , Y a 1 ) when the rationale selector is not perfect . Our new goal is to reduce I ( Xa2 , Y a 1 ) more than we reduce I ( Xa1 , Y a 1 ) . We analyze the procedure in the worst case scenario in order to determine a lower bound on our algorithm ’ s benefits under some assumptions . In an extremely erroneous case , we say that for a given < x1 , x2 , y1 > , our initial rationale selector mistakenly selects x2 when aiming for x1 . When creating the corresponding counterfactual document , we still have yc1 = 1 − y1 , but we modify the document according to xc2 ← arg maxX2 p ( x2|1− y1 , x1 ) instead of the process defined by Eqn 2 . Going back to our concise example , this would be the counterfactual document This beer tastes good . It also smells bad . In this extremely erroneous case , we have decreased I ( X1 , Y1 ) while I ( X2 , Y1 ) remains unchanged . Thus , we define the worst case scenario as reducing I ( X1 , Y1 ) at some error rate α and keeping I ( X2 , Y1 ) constant at the same rate . If we say that this error happens to all samples with a rate α , we can analyze conditions that must be present in the original dataset so that our CDA approach is beneficial . Let ’ s first define ∆Ia as the change in mutual information from the original to the augmented dataset . ∆IaXi , Yj = I ( Xi , Yj ) − I ( X a i , Y a j ) ( 3 ) In order for our CDA approach to be beneficial , we need to decrease I ( X2 , Y1 ) more than I ( X1 , Y1 ) . That is : ∆IaX2 , Y1 −∆I a X1 , Y1 > 0 ( 4 ) For samples in the original dataset where the initial selector results in an error , X1 would map to 1− Y1 for the corresponding counterfactuals . Furthermore , the augmented dataset would map X1 to both Y1 and 1− Y1 for these original erroneous samples and corresponding counterfactuals , and thus , X1 is no longer informative of Y1 on such samples . That is , in the augmented dataset and with proportion α , p ( Y a1 |Xa1 ) = p ( Y a1 ) . For the remaining 1 − α portion of non-error samples , CDA can successfully capture p ( Y1|X1 ) in the original dataset and p ( Y c1 |Xc1 ) in the counterfactual dataset . To quantify the conditional distributions of the augmented dataset , we make two assumptions : first , we assume that we can successfully generate the counterfactuals , and thus , we can assume p ( Y c1 |Xc1 ) = p ( Y1|X1 ) ; second , we assume that the erroneous samples happen randomly across the original dataset , and the marginal distributions of the original and augmented datasets are the same . Based on these two assumptions , we have : p ( Y a1 |Xa1 ) = αp ( Y1 ) + ( 1− α ) p ( Y1|X1 ) ( 5 ) For X2 , the success and failure cases are reversed . p ( Y a1 |Xa2 ) = ( 1− α ) p ( Y1 ) + αp ( Y1|X2 ) ( 6 ) We can now expand Eqn 4 using the definition I ( X , Y ) = H ( Y ) −H ( Y |X ) . 0 < −H ( Y1|X2 ) +H ( Y a1 |Xa2 ) +H ( Y1|X1 ) −H ( Y a1 |Xa1 ) ( 7 ) Using the definition , H ( Y |X ) = −E log p ( Y |X ) , we can expand this further to 0 < −E log p ( Y1|X1 ) + E log p ( Y a1 |Xa1 ) + E log p ( Y1|X2 ) − E log p ( Y a1 |Xa2 ) ( 8 ) Eqn 8 describes the conditions that must be met in our original dataset in order to yield gains from our CDA procedure for some error rate α . It is impossible to calculate this relation directly because it will require exact knowledge of our ground-truth rationales . In order to shed some light on this , Figure 2 shows the efficacy of the CDA approach when we approximate X1 and X2 with binary variables , p ( Y1|X1 ) = 34 , p ( X1 ) = p ( X2 ) = p ( Y1 ) = 1 2 . When X1 and X2 are equally informative of Y1 , the benefits of CDA decrease linearly with the error rate , and intuitively , our error rate must be less than 50 % to see gains . When X2 is more informative than X1 , we have a higher error budget to see any benefit , and when X1 is more informative than X2 , our error budget is smaller . According to this analysis , we have a higher error budget when spurious signals offer the same or more information about the target label . When the spurious signals carry much less information , the initial selector must have a low error rate in order to benefit from the CDA approach .

Summary: A rationale in NLP is a subset of an original input text on which a classifier operates. Extracting such a rationale is a popular approach for more interpretability because by definition we know the classifier only had access to this reduced text. However, rationales can be nonsensical and can consist of spurious patterns in the dataset.  To improve the coherence of rationales and to improve generalization, this paper proposes a technique to automatically create counterfactual data. This counterfactual data is created specifically to reduce mutual information between the true target label and irrelevant parts of the original text with regards to the true target label. For example, in “This beer smells great. It tastes terrific.” both sentences have positive sentiment but only the first is relevant when predicting whether the beer has a good smell. To help a model to automatically learn to distinguish, a helpful counterfactual example would be “This beer smells awful. It tastes terrific”.  Contributions  -	A method to automatically generate counterfactual examples to improve rationale extraction. -	An analysis of how good an initial rationale extractor should be to be able to expect performance improvements using the proposed method of generating counterfactual examples.