Making a (Counterfactual) Difference One Rationale at a Time
Authors: Mitchell Plyler
Michael A Green
Min Chi
Venue: NeurIPS 2021 Poster (2021)
Keywords: interpretability
rationales
counterfactual
data augmentation
natural language processing
information theory

Abstract: Rationales, snippets of extracted text that explain an inference, have emerged as a popular framework for interpretable natural language processing (NLP). Rationale models typically consist of two cooperating modules: a selector and a classifier with the goal of maximizing the mutual information (MMI) between the "selected" text and the document label. Despite their promises, MMI-based methods often pick up on spurious text patterns and result in models with nonsensical behaviors. In this work, we investigate whether counterfactual data augmentation (CDA), without human assistance, can improve the performance of the selector by lowering the mutual information between spurious signals and the document label. Our counterfactuals are produced in an unsupervised fashion using class-dependent generative models. From an information theoretic lens, we derive properties of the unaugmented dataset for which our CDA approach would succeed. The effectiveness of CDA is empirically evaluated by comparing against several baselines including an improved MMI-based rationale schema on two multi-aspect datasets. Our results show that CDA produces rationales that better capture the signal of interest.

Making a (Counterfactual) Difference One Rationale at a Time
Mitchell Plyler
Michael Green
Department of Computer Science North Carolina State University mlplyler@ncsu.ee edu
Laboratory for Analytic Sciences
magree228ncsu. edu
Min Chi
Department of Computer Science
North Carolina State University mchi@ncsu. edu
Abstract
Rationales, snippets of extracted text that explain an inference, have emerged as a popular framework for interpretable natural language processing (NLP). Rationale models typically consist of two cooperating modules: a selector and a classifier with the goal of maximizing the mutual information (MMI) between the "selected" text and the document label. Despite their promises, MMI- based methods often pick up on spurious text patterns and result models with nonsensical behaviors. this work, we investigate whether counterfactual data augmentation (CDA), without human assistance, can improve the performance of the selector by lowering the mutual information between spurious signals and the document label. Our counterfactuals are produced an unsupervised fashion using class-dependent generative models. From an information theoretic lens, we derive properties of the unaugmented dataset for which our CDA approach would succeed. The effec- tiveness of CDA is empirically evaluated by comparing against several baselines including an improved MMI-based rationale schema [19] on two multi-aspect datasets. Our results show that CDA produces rationales that better capture the signal of interest.
Introduction
Research in neural model interpretability has heen cast as important and received significant recent attention (2T] Within the field of natural language processing (NLP), rationales have been popular method for providing interpretability in the form of extracted subsets of text [10]. Rationale models typically consist of two cooperating modules where one module, the "rationale selector". selects the rationale from a source document, and the other module, the "classifier", acts on only the selected rationale without seeing the rest of the document. There interpretability through sparsity and exclusivity.
A common approach for training these rationale models is based the maximum mutual information criteria (MMI) [7] With the MMI criteria, rationale selectors seek the subset of text that carries the most information about the target label. Often, sparsity and coherency constraints rationales interpretable. Within many datasets, however, spurious patterns and co-varying aspects can cause the rationale selector to pick up on patterns that do capture a desired relationship between input text and target labels. As a result, the rationalized model can have undesirable behaviour like predicting hotel is very clean because is in convenient location. Nonsensical rationales or
used to keep the
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

explanations might decrease trust in the model, and in some cases, suggest the model might generalize poorly [26]
In this work, we propose general counterfactual data augmentation (CDA) (22) approach to aid rationale models trained with MMI. We show that theoretically our CDA approach effectively improve the performance of rationale selectors by lowering the mutual information between spurious signals and aspects of interest. Empirically, we show that models trained on our CDA datasets learn higher quality rationales than those trained on the original dataset when both use the same MMI criteria. More importantly, the most significant advantage of our CDA approach is that it does nor require human intervention. We use rationales from an initial, noisy model and replace them with new text that changes the target label using a generative neural model. In this way, our CDA approach is completely hands off and does not need input from human experts or crowd workers.
We first show that in the extremely ideal scenario where the initial rationale selector is perfect, our CDA approach can eliminate the mutual information between spurious signals and the target label. Next, we show in the common, realistic scenario where the rationale selector is noisy and imperfect, our CDA approach can still yield gains. Finally, the effectiveness of our CDA approach is compared against several baselines including an improved MMI-based rationale schema [9] on two common multi-aspect review datasets, TripAdvisor [29] and RateBeer [24]. Multi-aspect datasets are our main focus as they are guaranteed to contain spurious patterns and co-varying aspects; we primarily used MMI-based baselines because the fundamental goal our CDA approach is to reduce the mutual
information between spurious signals and aspects of interest.
2
Counterfactual Data Augmentation and Multi-Aspect Datasets
2.1 Definitions and Notations
We use upper-case letters to denote random variables, X and Y, and lower-case letters denote samples from these variables, x and y. [(X, Y) marks the mutual information between and Y. Mutual information is defined as the reduction in uncertainty of a random variable due knowledge in another random variable, T(X, Y) H(Y)- ((Y))), where H(X) and (Y| are the Shannon entropy and conditional entropy respectively (8)
2.2 Problem Formulation
This work follows the same rationale concept introduced in 19] Specifically, one neural module extracts text from document and another neural module classifies the extracted text. Later it has been shown that the rationalization criteria aims "to maximize the mutual information between selected defined as:
features and the response variable" 7
max I(XM;Y) subject to M ~ G(X) G
(1)
where M denotes a binary rationale mask over the input produced by a rationale selector G. That is, the goal of the rationale selector is to select the subset of features in X that are most informative of the label Y under some constraints defined by the selector G. In this work, our selector constraints affect the size and coherency of the rationales.
Here we consider a multi-aspect dataset, D with features and labels X a set of features, or a sequence of words in NIP, and Y is vector, possibly one dimensional, of numerical scores. In multi-aspect dataset, a single document can discuss multiple attributes of a single object. For example, single beverage review might discuss its appearance, taste, and smell. We assume some subset of the features, X,, belong to the target aspect label, Y1, while other features, X2, are spurious or non-causal. These features could belong to other aspects or be artifacts of the dataset. In the following, for simplicity, we will use <X, Y1> or <X,, X2, Y₁> interchangeably based on the context.
Our goal is to estimate or model the score for aspect Yi by using the function f given only X1, as pair. In this work,
follows: Y, = (.) There is one such model corresponding to each < X,, Y >> we use multi-aspect datasets, and model each aspect individually. This simulates the more common case where a dataset has single output label of interest and all signals in the dataset that do not pertain to that label are considered spurious or belonging to other, not-estimated aspects.

---

When predicting Y1, our ideal model would focus on X1 and ignore all other features, X2. Specifically, for a given sample <x. y>, our selector would select a subset of x such that 21 2 M- A rationale selector might fail to extract 21 effectively for two primary reasons: first, Chen et al. f showed it is intractable to find a solution to Eqn [ad thus they derive a variational approximation with Monte Carlo based gradient estimates. Second, datasets can contain artifacts such that spurious patterns might contain significant mutual information with the target label. Along the same lines but even more concerning, other aspects in a dataset might be highly correlated with the target label. For example, beers that smell good usually taste good as well.
Since the goal of the selector is to select features that maximize I [[MM Y1), we can assist the selector in finding X1 by lowering the mutual information between the other spurious features and the desired label, (X2,, Yi). Drawing on ideas from (22]. we do this through a general counterfactual data augmentation (CDA) scheme where, in the counterfactual dataset, superscript c, we flip the label of the document from Y1 to Yf and replace the text selected by the rationale selector, X1, with an inference, Xf, generated by class conditioned masked language model (MLM) using Yf and X₂ described as:
-- Xi + arg max m -Y, X1
(2)
In our generated counterfactual dataset < Xi, X2, Yf >, Xf is the newly generated counterfactual,
X2 is the original spurious feature set, and we assign y = dataset which the concatenation of datasets < X1,, X2, Y1 > and Xf, 1 Y1 >, we are lowering the mutual information (X2,, Y1). We will use superscript a for the augmented dataset: D
Yi. We will show, in the augmented
=<ff Xg, Y!²
2.3 Lowering 1(x2, Y1), an idyllic case
Take dataset of beer reviews where each document contains a description of the taste and smell of beer as well as a numerical score for only the smell aspect. The task is to estimate the smell score while using the smell text the rationale. Figure I demon- strates our process. An example of concise document in this dataset might be This beer smells great. It tastes terrific. In this document 21 is the phrase smells great and 22 is tastes terrific. Both have positive sen- timent but our only label for this docu- ment yı 1 for the smell sentiment. Our counterfactual document could be This beer smells awful. It tastes ter- rific and our label becomes /ı O. We can see that in the augmented dataset the phrase tastes terrific maps both positive and negative labels and therefore (x2) = p(Y)). Finally, I (X2, Y1) is and I(X,, Y1) is unchanged.
D with I(X,, ((2,, Yi)
<1>r..beer smells great. lastes terrilic...
De with I(X,, Y1), I(22, Yi)=0 =0 0 <te...bee amells grest. tastes terifi... <Oe..beer amells sudul. tastes terrific...
Y 1-11-y Vi argmax p(X,]1 X2)
DP with 1(1,, Y1), (X2, Y1) <Oer..beer smells awful. tastes temffic...
Figure 1: Toy example to demonstrate our approach. In the augmented dataset D the mutual information, 1, between the smell score and the smell text is preserved while the mutual information between the smell score and the taste text is eliminated.
With perfect knowledge of the ground truth rationales and the process Xf < p(X,1 - ,,, can craft counterfactual documents and therefore a counterfactual dataset that perfectly eliminates [(2,, Yi) while preserving I(Xi, Yi). However, the challenge is that ground truth rationales are not provided in the training data. Therefore, it is important for us to show that even when rationales selected by the initial rationale selector are noisy and imperfect, we still lower 7(X,, Y1) in the augmented dataset and benefit subsequent models trained with MMI.
2.4 Dealing with a Noisy Initial Selector
Here we are working on the augmented dataset: D D== X2, Y To completely eliminate T(Xg, Yf), our CDA approach requires a perfect rationale selector. We were originally motivated by improving a poor rationale selector, so here we track what happens to both T(X3, Y?)) and T(Xf, Yf) when the rationale selector is not perfect. Our new goal is to reduce 1(Xg, Yf) more than reduce
3

---

[Xy, YP). We analyze the procedure in the worst case scenario in order to determine a lower bound on our algorithm's benefits under some assumptions.
In an extremely erroneous case, we say that for a given <31, 22. yı>, our initial rationale selector mistakenly selects 22 when aiming for I1. When creating the corresponding counterfactual document, we still have vf = 1- 1/1, but we modify the document according to .. < arg max p(rz|| 21) instead of the process defined by Eqn2/ Going back to our concise example, this would be the counterfactual document This beer tastes good. It also smells bad. In this extremely erroneous case, we have decreased T(X,, Yi) while 7X2, Y1) re- mains unchanged. Thus, we define the worst case scenario as reducing T, Y1) at some error rate a and keeping I (X2, Yi) constant at the same rate.
+0.30
0.25
-0.10
0.20
0.35
0.30
0.05
0.00
-0.05
-0.30
If we say that this error happens all samples with rate a, we can analyze conditions that must be present in the original dataset so that our CDA approach is beneficial. Let's 's first define A as the change in mutual information from the original to the augmented dataset.
0.4 0.6
0.8
1.0
Figure 2: CDA benefits (y-axis) as func- tion of error rate Each line represents (X)] = (() cc
A X,,Y, Y)) - I(X;, YJ)
(3)
In order for our CDA approach to be beneficial, we need to decrease I (X2, Y1) more than 1(X, N)). That is:
A - - > 0
(4)
For samples in the original dataset where the initial selector results in an error, X, would map to 1 Yn for the corresponding counterfactuals. Furthermore, the augmented dataset would map Xi to both Vi and 1 - Yi for these original erroneous samples and corresponding counterfactuals, and thus, X1 is no longer informative of Y1 on such samples. That is, in the augmented dataset and with proportion a, p(Y = p(Y)). For the remaining 1 a portion of non-error samples, CDA can successfully capture p(Y]]X;) in the original dataset and p( (Y{ | in the counterfactual dataset. To quantify the conditional distributions of the augmented dataset, we make two assumptions: first, we assume that we can successfully generate the counterfactuals, and thus, we can assume p(Y [x);; second, we assume that the erroneous samples happen randomly across the original dataset, and the marginal distributions of the original and augmented datasets are the same. Based on these two assumptions, we have:
p( (YTlx]) = ax((Y) + (1 - ())((i(X1))
(5)
For X2, the success and failure cases are reversed.
p p(
(1 - a)p((i) + (x(Y)
(6)
We now expand Eqn using the definition I(X,Y) = H(Y) H(][)).
0 xx + HYT" xx) + H(Y[X,,) HHY Using the definition, HY = E log (Y X), we can expand this further to
0 < E log + E log p (Y X + E log p(K - E log pYP Xx2)
(8)
Eqn 8 describes the conditions that must be met in our original dataset in order to yield gains from our CDA procedure for some error rate a. is impossible to calculate this relation directly because it will require exact knowledge of our ground-trui rationales. In order to shed some light on this, Figure shows the efficacy of the CDA approach when we approximate X,, and X2 with binary p(X2) = p(Y))
When X, and X₂ are equally informative of Y1, the benefits of CDA decrease linearly with the error rate, and intuitively, our error rate must be less than 50% to gains. When X₂ is more informative than X1, we have a higher error budget to see any benefit, and when X1 is more informative than X2, our error budget is smaller. According to this analysis, we have a higher error budget when spurious signals offer the same or more information about the target label. When the spurious signals carry much less information, the initial selector must have a low error rate in order to benefit from the CDA approach.
variables, x
3. p(

---

With the datasets used in this work, we can gain insight by approx- imating X, and X2 with guessable bigrams. Here is another concise example: X1 is a binary variable that indicates the occurrence of the phrase no lacing and X2 is binary variable for the phrase light bod- ied. For a beer's appearance aspect, no lacing is strong indicator for a negative score, and for its palate aspect, light bodied a strong negative indicator. As beer's appearance is highly correlated with its palatability, both phrases indicate low appearance score. Taking Y1 the appearance score, Xı the occurrence of no lacing, and light bodied as X2, we can numerically use Eqn 8 to examine how CDA helps for varying enror rates in Figure 3 CDA is beneficial whenever the curve is above zero. In the fully correlated dataset, "Correlated", described in section [. the information carried by light bodied is closer to that of no lacing than it in the decorre- lated dataset. As consequence, in the correlated dataset, have a higher error budget and more opportunities for our CDA.
no lacing, light bodied
0.002
Comeated Decomnlatedd
A K.
-0.002
0.0
0.1
0.3
0.6
Figure 3: CDA benefits when approximating X, and X2 as the occurrence of bi-grams
Architecture and Implementation
3.1 Rationale Framework
The original rationale framework, as viewed in this work, was introduced by [19] It was not our goal to change the core rationalization algorithm, so we re-implemented the algorithm and updated some details. At high level, our rationale framework is the same in that we use one network select the rationale and another network to classify the text. The rationale framework can be visualized by the blue portion of Figure
The original implementation 19 used RNNs for both networks, REINFORCE 30 for dealing with the discontinuity introduced by the binary rationale mask, and variable percentage rationales. In this work, we use transformers for both networks because of their effectiveness over RNNs in NLP 28). the simpler traaighttthrough method rationales because it eliminates sparsity related hyperparameters. Our fixed percentage rationales differ from Chen et al. 77 in that we use the top-K tokens during training and inference whereas Chen et al. 7 use an iterative re-sampling approach with Gumbel-softmax reparameterization [15).
5 instead of REINFORCE, and we use fixed percentage
The classifier is trained only to make quality predictions against the labels while using the rationale. This is the cross-entropy between the labels and the classifier's 's prediction, Ly- We follow 19 by using a coherency regularizer for the rationale selector: I.p = + Σ T....TTi. Im, - m where T is the total number of tokens in a document, m is the rationale mask, and X, is the hyper-parameter used to tune coherency. This encourages the rationales to be contiguous. For the datasets evaluated in this work, coherency coherency regularizer and the cross-entropy between the labels and the classifier's prediction. This is Lg L + Ly-
useful inductive bias. The loss for the rationale selector L, is the
3.2
Counterfactual Predictor
The CDA process described earlier, Eqn requires us to generate new documents with the text and label flipped for just one aspect. That is, we are sampling a new doc- ument from [[X]]X2, 1- Y1). The key challenge that we are not provided with ground truth rationales or coun- terfactuals with which to learn how this data generation process works. This can be viewed under the lens of unsu- pervised style transfer for which there is significant prior work in the NLP domain (20 23 Our method for gener- ating counterfactual documents leverages many ideas from these works, and our main contribution here is connecting these ideas to the rationale framework, We incorporate the generation process because goal is to lower the mutual
Rationale Selector
Classifier
X X M
Lag.
CF Predictor
Discriminator
X
---X" auth
Figure 4: CF Predictor training flow.
rationale framework in the counterfactual
5

---

information between the spurious signals and the target label. An off aff-the-shelf style transfer method might focus on signals other than that selected by our initial rationale selector.
Figure 4 shows the CF Predictor's ' training flow. At its core, the CF Predictor leverages class- dependent Masked-language models (MLM) (9. We replace the original document's rationale with an inference from a MLM and leave the rest of the document unchanged. The MLMs are trained to produce documents with the desired class through reinforcement learning (RL) [2], and they are trained to produce realistic documents through adversarial learning [T. We use straight-thruugh B to propagate gradients through token selection during training. For dataset with binary labels, we
train two separate MLMs: one for generating class-O documents and another for generating class-1 documents (31] The loss for the counterfactual predictor. LCFP is defined as:
Lerp = LRI. - AALA
where RL is the classifier loss after passing the counterfactual through the rationale selector and the classifier. is the cross-entropy between the desired label, all ones or zeros, and the predicted label. LA is the loss component from adversarial training. Following [1] and (32], our discriminator, D seeks to distinguish generated documents from the originals, so the discriminator's 's loss, LD. is LA where LA is the cross-entropy between real-fake labels and the prediction given by the discriminator. AA and ARL balance the adversarial and classification losses. We include X when training the discriminator to hamstring it relative to the predictor. We found that this generally helped us avoid mode collapse commonly seen when adversarially training generative models. For LA, we mask the contribution of original documents without the desired label. When training the class-1 counterfactual predictor, we show the discriminator real documents with Y1 = and counterfactual documents, X where the original documents' labels were Y1 0.
During training, the counterfactual is produced in one step. All words not included in the selected rationale from the original document, XM, are kept in the counterfactual document. The kept tokens are X X As- All words in the selected rationale are replaced by the CF Predictor using one prediction from the MLMs in greedy fashion. After training, when producing the counterfactual dataset, the counterfactual documents still keep all non-rationale tokens. We now replace the rationale tokens from left to right using the output of the CF Predictor MLMs. A counterfactual token at position t is decoded according to the following process.
x arg max p( |22 1- 3/1 31,2f,o... .......)
(10)
11,t
We found this to be a good trade-off between greedy decoding and a more expensive beam search. Greedy decoding could generate frequent, repeated tokens while beam search could be an unnecessary expense for generating documents that reflect the target distribution, but do not necessarily need to pass the bar of human readers.
Figure 5illustrate an original document and its counterfactual generated by our CF predictor. Notice smell aspect. The inclusion of the original and its counterfactual document in the au augmented dataset successfully decreases the mutual information between the non-smell text and the smell label.
here the initial selector was successful in identifying the
4 Experiments
4.1 Datasets
We conduct experiments using datasets from two sources. This first source contains reviews compiled by Wang et al. (29 from TripAdvisor.com. We use the training, dev, and test sets curated by Bao et al. [ and used for rationalization by Chang et al. [5]. The label is binary, and we focus on the location aspect. There are 198 test samples with numan-annotatee rationales.
The second source consists of reviews collected by McAuley et al. 24 from RateBeer. Each review paragraph of text with five numerical scores in the range of to 5 for the appearance, smell, palate, taste, and overall aspects of the beer. We focus on the appearance, smell, and palate aspects. We created training and dev datasets for each aspect that follow the source distribution. Following [5) we binarize this data so that all reviews with a score 2 3 are class 1 and all reviews with a score < 2 are class O. We then balance the dataset between classes Additionally, we use procedurally decorrelated datasets created by Lei et al. [19, and binarize these datasets well. We now have two

---

Label
Document
negative
presentation : 12 oz clear bottle. label metallic with an emblem of a buck. inkjetted on the neck 'mm not sure if this is out of date, or was produced then appearance pours clear, somewhere between an amber and brown color average head but it fades pretty quickly to leave just a hint of lace. smell a bit of grain fairly mild. this is sort of generic heer aroma, without much to pick out specifically. tastes a bit like a light marzen overall impression : is n't it sort generie beer with a hit more flavor than your average macro sort of like a yuengling. nothing to write home ahout but it 's drinkable enough
little bit of sweetness and malt this
positive
presentation : 12 oz clear bottle. label metallic with an emblem of a buck. . inkjetted on the neck 'mm not sure if this is out of date, or was produced then appearance pours clear, somewhere between an amber and brown color average head but it fades pretty quickly to leave just a hint of lace. smell caramel malt and toffee dominate the nose with bit of raisin and generic beer aroma, without much to pick out specifically taste little bit of sweetness and malt this tastes a bit like a light marzen. overall impression: it is n't bad it is sort of generic beer with a bit more flavor than your average macro sort of like yuengling. nothing to write home about, but it 's drinkable enough.
Figure 5: A subset of the augmented dataset for beer smell. The rationale is bold in the original document (top). The replaced words are bold in the counterfactual document (bottom). 21 and y/1 have changed from original to counterfactual. Ground truth rationales and counterfactuals are not provided in any training data.
datasets for each RateBeer aspect: a "correlated" set and a "decorrelated" set. Appendix Section A.2 shows correlation matrices before and after decorrelating the data as well as additional dataset details. In the correlated datasets, the correlations and presumably the mutual information between aspects much higher than in the decorrelated datasets. This makes the rationalization task more difficult. Additionally, McAuley et al. (24) provides about 1,000 holdout samples where the aspect specific text is annotated by human experts.
4.2 Baselines
Three MMI-based baselines are used. First, "MMI" is the original Rationalization scheme 19] that has been updated and re-implemented to use the most recent NLP models as described in section This model is trained on the original, unaugmented dataset.
Second, Factual Data Augmentation (FDA) is used to verify that our gains are not only due to data augmentation but due to the CDA procedure. FDA augments the original dataset with new samples generated by the CF predictor models, but instead of flipping the label and passing it to the 1 y component, we pass the sample to the component of the CF predictor with the same label as the original document during inference. The new samples are produced using the following process:
VI +
of + arg max ((X1111,11)) T2)
(11)
21
The third baseline, simple substitution using antonyms (ANT), does not use neural models to augment counterfactual data. The counterfactual generated by replacing words in the rationale with antonyms
from WordNet (25] 4 Note that we only accept antonyms that are in the vocabulary used by the
models and antonyms that have the same part of speech as labeled by NLTK
4.3 Experiment Settings and Assumptions
We train the rationale selector and the classifier together, early stop based on the selector cost, freeze the selector, and finally fine-tune the classifier on the original dataset. For all of the data sets and models, we use the dev set for early stopping (more details in Appendix Section A.3). Our MLM transformers (8 were pretrained on unlabeled data from the TripAdvisor and RateBeer datasets separately. For the TripAdvisor dataset, we pretrain on all data that does not appear in the location aspect's train, dev, or annotated dataset from [5]. For the RuteBeer datasets, we pretrain on data that does not appear in any train, dev, or annotated dataset from [19 and the correlated datasets. We used a masking rate of 10% and masked tokens were treated as described in ( For the rationale models, the transformers are initialized from models pre- pre-trained with random masking. We found
7

---

Hotel Location
we only stayed one night the location was great and the premises was beautiful the room was typical hotel room but was decent, clean and spacious enough for our needs, were given first floor room next to the pool. was relatively quiet. the staff were informative and friendly- the daily parking was a total rip- off at $ 20/day but i would think that the rates are similar anywhere else you stay on the island. all in all it was good enough stay.
Correlated - Smell
pours two thick fingers of
froth
dark brown body good
with caramel and brown sugar sweetness arising to greet it the taste blends the sweetness with dash of hopped bitterness well. well carbonated this is sturdy. . very drinkable beer that has joined me for much leisure time. wo n't apologize for that ; nor will you after a couple brown ales.
Figure 6: Examples from the annotated sets. Hotel-Location (top) and Correlated-Smeel (bottom). Human annotations are underlined. CDA rationales are in blue MMI rationales are in red Overlaps between CDA and MMT are magenta.
that contiguous masking was better for pre-training the CF predictor. This contiguous masking similar to that from Joshi et al. [66, but masked tokens are treated the same as Devlin al.
[9] and there is only one contiguous span. The rationale selector and classifier are initialized from the
transformers. For the rationale selectors, following [5], we set the rationale percentage to 10% for all datasets. The CF predictor components and the GAN discriminator initialized from the contiguous-maskinn
transformers. These models all have a vocabulary with 2 tokens, 8 layers, 8 attention heads, and hidden dimension of 256. Appendix Section A.4 shows server configurations and more details on our experiment setup.
Models are selected and reported based on the best performance on the dev set across grid search. All methods are evaluated using the same grid search when training the rationale model. The model is selected before fine-tuning the classifier with a frozen rationale selector. The initial selector used to train the counterfactual predictors was the selected MMI model with fine-tuned classifier. The parameters and checkpoints for the CF Predictor models are tuned and chosen to maximize the accuracy of the training documents' predicted label as compared to the target label (measured by the original rationale model) and to maximize the entropy in the inserted counterfactual tokens. The CF Predictor model is chosen from a grid search, using only the training dataset, across A and ARL.
For the TripAdvisor dataset, we repeat the experiment three times: training an initial rationale selector, training a counterfactual predictor, generating a counterfactual dataset, and training new rationale model. Additionally, we train two additional rationale models with different random seeds and the selected hyperparameters. Consequently, there are nine rationale models for each of the four methods, and we report the mean and standard deviation across those nine models. In the RateBeer datasets, we repeated the same experiments two times and then trained two additional models with different random seeds and the selected hyperparameters for each rationale model. As result, we have total of six runs for each method for the RateBeer datasets. All models are Tensorflow MI Our software is publicly released
4.4 Results
We evaluate the rationale models by the precision (Rat. Prec.) as compared to human annotations. This token-level metric is taken as the mean across samples in the annotated set. We also report the accuracy of the classifier on the development set (Dev. Acc.).
As shown in Tables 2 and 3 our CDA approach outperforms all baselines on experiments. As expected, CDA's performance over other methods is most pronounced on the correlated RateBeer datasets compared to on the decorrelated ones. Only on the decorrelated appearance dataset, CDA performs second to MMI. This may be because on the decorrelated appearance dataset the spurious aspects have very low mutual information with the target label, appearance, and therefore the error margin for the initial selector is very tight and CDA is expected to yield low gains as discussed in Section 2.4
"gichub.com/mlppyye///C..... for Rationales
8

---

The baseline augmentation schemes (FDA, ANT) produced mixed results. The FDA scheme should not have changed the mutual information between the spurious aspects and the target label. Whenever it performed worse than MMI alone, it most likely introduced noise into the dataset. While the ANT scheme might be effective in lowering the mutual information between the spurious aspects and the target label, the generated counterfactual does not likely model p(r1|1 U1, 22) so we expect it to also reduce (X1,,)) Yi).
Empirically, we have demonstrated that models trained on the CDA augmented data tend to outperform models trained on the original datasets. This lends credence to the idea that our scheme is indeed lowering the mutual information between the spurious aspects and the target la- bel. The dataset and method pairs with very high variance are dragged down by degenerated runs where the rationale selector has little or no skill. These runs were seen when varying the random seed with the selected hyperparame- ters. Removing the degenerate runs can bring the mean rationale precision of the high variance experiments closer to CDA, but we believe these degenerate models seen primarily in the baselines align well with our theory that CDA allows the rationale selector more easily identify X1.
Table I: TripAdvisor Dev. Acc.
Location Rat. Prec.
MMI
78.16 = 5,83
26.14 = 13.25
FDA
80.61 = 4.38
31.36 = 10.38
ANT
69.79 = 2.90
12.25 = 3.64
CDA
78.11 + 6.77
39.76 = 10.48
Case Study: Figure 6 presents case study comparing the rationales selected by MMI and our proposed CDA approach in the TripAdvisor and Correlated Beer datasets respectively. As shown in Figure 6 our CDA models can select the text that aligns better with human annotations and they successfully avoid selecting sentiment-carrying text that is not relevant to the aspect of interest.
Table 2: Correlated RateBeer Results
Appearance
Smell
Palate
Dev. Acc.
Rat. Prec.
Dev. Acc.
Rat. Prec.
Dev. Acc.
Rat. Prec.
MMI
76.37
3.05
56.28 17.22
79.49 = 4.53
45.39 + 16.45
82.67 + 0.99
34.98 + 8.79
FDA
75.73
3.64
41.02 = 5.09
76.43 = 5.08
41.67 = 23.73
77.60 = 6.06
32.35 = 14.41
ANT
62.09
8.76
43.25 16.51
57.70 = 9.87
11.72 + 7.16
61.93 + 7.59
8.62 + 12.31
CDA
73.30 + 2.93
67.79 = 10.63
81.82 1 0.69
61.84 1 6.76
80.65 1 0.64
41.24 + 1.72
Table Decorrelated RateBeer Results
Appearance
Smell
Palate
Dev. Acc.
Rat. Prec.
Dev. Ace.
Rat. Prec.
Dev. Acc.
Rat. Prec.
MMI
81.33 + 0.93
86.92 = 7.35
77.99 + 5,46
79.71 + 10.56
68.64 + 10.09
46.39 + 29.21
FDA
80.67
1.32
81.97 6.00
79.66 + 1.23
83.62 + 3.52
77.26 + 1.90
66.03 + 3.76
ANT
71.41
- 9.97
67.67 = 27.68
58,81 + 6.31
9.32 + 10.76
57.75 + 7.98
10.83 + 11.15
CDA
80.24 = 1.07
82.82 8.60
79.18 1.43
86.66 + 2.92
76.81 0.85
67.79 + 3.20
Related Work
The rationale framework used in this work was introduced in 19 and connected to the MMI criteria by Chen et al. [7]. Much of the follow-up work introduced modifications to the learning algorithm to overcome spurious signals picked up by MMI. Yu et al. [33) sought to limit the signal left in the complement of the rationales. The concept of class-wise rationalization was introduced in Chang et al. [5], and Chang et al. [6] introduced rationale algorithm that seeks invariant rationales across environments. These works modify the rationale framework and the MMI criteria to help the rationale selector find desirable signals. In our work, we keep the MMI criteria unchanged, but instead seek to diminish the undesirable signals through counterfactual data augmentation.
Data augmentation generally and counterfactual data augmentation (CDA) specifically has been a popular technique in recent natural language processing (NLP) work. Lu et al. [2] introduced counterfactual data augmentation general methodology and showed their rule-based scheme

---

could mitigate gender biases commonly seen in word embeddings and NLP models. Other works used CDA to train better named entity recognition (NER) models in the medical text domain [4]. More work [17] continued the ideas of CDA by working with sentiment analysis and natural language inference tasks. They constructed their CDA datasets using crowd workers and showed models trained on the augmented datasets generalized better -fmmai. Follow-up work [18] showed how their crowd worker interventions reduce spurious signals when viewed through simplified causal modeling lens. The causal lens and generative counterfactual data augmentation have also shown recent, positive results in the computer vision domain 227
Conclusions
This work presents counterfactual data augmentation method for lowering the mutual information between spurious signals and a target label a dataset. We derive theoretical conditions that indicate when our approach will be beneficial to the dataset. Empirically, we show rationale models trained with MMI on our counterfactually augmented datasets result in rationale selectors with improved behaviours. Furthermore, our analysis shows that the benefits of our scheme are proportional to the error rate of the original rationale selector. This suggests we could perform counterfactual data augmentation iteratively where achieve better rationale selectors after each iteration that could be used as the initial selector in the next round of data augmentation.
Our method does not make any changes to the core rationale algorithm. There has been more work in the literature on rationale algorithms since Lei et al. [19] and Chen et al. 77,, but they were not included in this study because they do not necessarily follow the maximum mutual information criteria on which our theory is based. From a practical standpoint, is likely our CDA method could connect with other rationalization strategies because the CF Predictor is trained using frozen rationale model. This could be an important area for future study.
7 Broader Impact
This work focuses on methods for training interpretable NLP models. While interpretability is typically a desired trait, defining exactly what is interpretable is not always clear [21]. Rationale models have been shown to not always be faithful [14] [13, and in the worst case, they can degenerate in such a way where the rationale selector makes the decision instead of the classifier [33] The
degenerated case is likely to produce rationales that nonsensical to human interpretation.
Section 2.4 presents some theoretical proof on how CDA affects the mutual information between text and the target label with different initial selector error rates. Some conclusions drawn in Section 2.4 rely on a simplified model with binary variables. Furthermore, the simplified model makes the assumption that the generated text follows the same distribution as the original dataset. While this is the goal of the adversarial learning strategy that we implemented, we found that the generated text had lower entropy than the rationales they replaced. This is likely due to both the arg max in Eqn and LRL.
We have demonstrated positive results for improving rationales for sentiment analysis sentiment classifiers. Generalizing the approach to other NLP tasks is less straightforward. First, the naive approach to generalizing to K classes in the classification setting suggests having K CF Predictor models and generating K- counterfactual documents per data sample. This would be a significant increase in computation and resources. Second, for the task of natural language inference, CDA has been successful with human annotators [77. It might be possible train CF Predictor for each type of hypothesis. It is less clear how to train CF Predictor to modify premise.
Our method uses data augmentation to lower the mutual information between some signals in documents and a label of interest. When the initial rationale selector aligns with human judgment, models trained the augmented data tend better align with human judgement. The method can be thought of as amplifying signals relative to others. We are changing the properties of the dataset. For sensitive tasks, one should check for undesired biases and fairness concerns before and after CDA.
Acknowledgments: We sincerely thank Jascha Swisher and Stephen Shauger for their insights and help. We thank the Laboratory for Analytic Sciences for supporting the work, and we also thank the reviewers for their time and helpful advice.
10

---

References
[1] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo. Zhifeng Chen, Craig Citro. Greg Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, lan Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg. Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. URL https: /wwp. tensorflow org/ Software available from tensorflow.org.
[2] Yujia Bao, Shiyu Chang, Mo Yu, and Regina Barzilay. Deriving Machine Attention from Human Rationales. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. pages 1903- 1913, Brussels, Belgium, October 2018. Association for Computational Linguistics. doi: 10.18653/ v1/D18- 1216. URLLiittps://accaaatoooooo org/D18 1216
[3] Yoshua Bengio, Nicholas Léonard, and Aaron Courville, Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv: arXiv:1308.344 1308,3432 [es], August 2013. URL http: //arxiv. farxiv.org/sblll....... abs/ 1308. 3432 arXiv: 1308.3432.
[4] Steven Bird, Ewan Klein, and Edward I Loper. Natural language processing with Python: analyzing text with the natural language toolkir." "O O'Reilly Media, Inc.". 2009.
Shiyu Chang, Yang Zhang, Mo Yu, and Tommi Jaakkola. A Game Theoretic Approach to Class- wise Selective Rationalization. Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https:/ http::://proceedings. www.::s....rppp..... cttppa.ccppppeeee wtwp:/pp.ccppppppeeeeeeeeee /aahh/
html
[6] Shiyu Chang, Yang Zhang. Mo Yu, and Tommi Jaakkola. Invariant Rationalization. In Proceedings of 37th International Conference on Machine Learning, pages 1448-1458. PMLR, November 2020. URL https://proceediie
mlr, wttpss/vee//e///ee/..... http///v11l/aaaagc.... html
ISSN: 2640-3498.
[7] Jianbo Chen, Song, Martin Wainwright, and Michael Jordan. Learning to Explain: An Information- Theoretic Perspective on Model Interpretation. In Proceedings of the 35th International Conference on Machine Learning. pages 883-892. PMLR, July 2018. URL httpt:::////cceeettiieee wwtpssesss.eeeemmeee.....e.eeeee
mlr. press/ maaass/v800 /
chen18j .html ISSN: 2640-3498.
[8] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory (Wiley Series in Telecommmiccation and Signal Processing). Wiley- Intersintrseeeecee USA, 2006. ISBN 0471241954.
[9] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume (Long and Short Papers), pages 4171 1171-48 141114466.660 Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: INN19 1423. WRRLLtttps::/www.cllle org/ wwtp.//wwww..ggataaolllggrg.. anthology/
N19 1423
[10] Jay Young Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong. Richard Socher, and Byron C. Wallace. ERASER: A Benchmark Evaluate Rationalized NI.P Models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4443-4458, Online, July 2020. Association for Computational Linguistics. doi: 10.
URI.
attps://aclanttoology wrg/2020 acl-mainn 408
[11] Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets. In Advances in Neural Information Processing Systems, volume 27, Curan Associates, Inc., 2014, URL htt.t:htt///// htttpp::/pproceedingg wtw.eeedddnnnnnn.nnneeeeeeeeeeee httppippp.cpppppp/pppppppp tttstbaatttttttttttttttt.ta Abstract.html
[12] Zhiting Hu, Zichao Yang. Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. Toward Controlled Generation of Text. In Proceedings of the 34th International Conference on Machine Learning, pages 1587-1596, PMLR, July 2017. URL httpht::/proceeediiiieee mlr 2640-3498.
http:///iinn.......eeee///////// wwtpps//e/////eee..eepp html ISSN:
[13] Alon Jacovi and Yoav Goldberg. Aligning faithful interpretations with their social attribution. Transactions of the Association for Computational Linguistics, 9:294-310 310, 2021.
11

---

[14] Sarthak Jain, Sarah Wiegreffe. Yuval Pinter, and Byron C. Wallace. Learning to Faithfully Rationalize by Construction. In Proceedings of the 58th Annual Meeting of the Association for Compulational Linguistics, pages 4459-4473, Online, July 2020, Association for Computational Linguisties. doi: 10. 10.18653/1/2000. acl-main Uttpttss::/aaa http::://ccclee anthocllnnttogggyyy org/ acl-main maain.409)
[15] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax, Sth International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017 Conference Truck Proceedings. OpenReview.net, 2017. URL URLhtttss:t
rkE3y85ee
[16] Mandar Joshi, Danqi Chen, Yinhan Liu, Danicl S. Weld, Luke Zettlemoyer, and Omer Levy. SpanBERT: Improving Pre-training by Representing and Predicting Spans. Transactions of the Association for Computational Linguistics, 8:64-77, 2020. doi: 10.1 16.116// a a000000 URI.https: tatttp://aclanthooloe
org/ org/2020.r tac1-115
[17] Divyansh Kaushik, Eduard H. Hovy, and Zachary Chase Lipton. Learning the difference that makes A difference with counteffactalll-aagmmmn data. In 8th International Conference on Learning Rep- resentations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https://openrevieu http//oorumndd-SlllSOPP.-
[18] Divyansh Kaushik, Amrith Sethur, Eduard H. Hovy, and Zachary Chase Lipton. Explaining the efficacy of counterfactually augmented data. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL petttps:/oppenrreee net/
forum?id-His, idd-HidddHiiii i QKWsUcV
[19] Tao Lei, Regina Barzilay, and Tommi Jaakkola. Rationalizing Neural Predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 107-117, Austin, Texas, 2016. Association for Computational Linguistics. doi: 10.18653//1/D11-1000 URL http: // //aclweb.or org/ http//aoggg//ie----ooe
[20] Juneen Li, Robin Jia, He Hc, and Percy Liang. Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Compitational Linguistics: Human Language Technologies, Volume I (Long Papers), pages 1 1865-1874, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: (0.18653/1/N118-116
[21] Zachary C. Lipton. The Mythos of Model Interpretability: In machine learning, the concept of inter- pretability is both important and slippery. Queue. 16(3):31-57. June 2018. ISSN 1542-7730. doi: 10. (0.1145/923388.322 0241886,3.444400
httpttsss:/// :/doii.///...../ org/10. 10. 1145/3236386.1. 3241340
[22] Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta, Gender Bias in Neural Natural Language Processing. In Vivek Nigam, Tajana Ban Kirigin, Carolyn Talcott, Joshua Guttman. Stepan Kuznetsov, Boon Thau Loo, and Mitsuhiro Okada, editors, Logic, Language, and Security: Essays Dedicated to Andre Scedrov on the Occasion of His 65th Birthday, Lecture Notes in Computer Science, pages 189-2022 Springer International Publishing, Cham, 2020. ISBN 978-3-030-62077-66 doi: 00007/978-3-000-60 62007-6016661 14. URL TRRLhhttss::. htttts:///oo/: :/doi.. org 1ttp//1.11001 00800077777700 3- 030 62077 14.
[23] Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W Black, and Shrimai Prabhumoye. Politeness Transfer: A Tag and Generate Approach. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1869- 1881, I, Online, July 2020. Association for Computational Linguistics. doi: 18653/v112022 ool-main.161 URL https: httpts::/aaaaaalllllllooooo anthology org/ 22000. acl -ainn 169
[24] Julian McAuley, Jure Leskovec, and Dan Jurafsky. Learning attitudes and attributes from multi-aspect reviews. In Mohammed Javeed Zaki, Arno Siebes, Jeffrey Xu Yu, Bart Goethals, Geoffrey Webb, and Xindong Wu, editors, 12th IEEE International Conference on Data Mining, ICDM 2012, Brussels, Belgium, December 10-13, 2012, pages 1020-1025. IEEE Computer Society, 2012. doi: 10.1 19.1109/00..2012... URL hattts:://dot org/ 10. 1ww11119/IICD /IO09IIICMM... 2012. 110
[25] George A. Miller. Wordnet: A lexical datahase for english. Commun, ACM, 38( 1):39-41, November 1995. ISSN 0001-0782. doi: 10. 15/219977...19944 URL http::t//oi..oor/////// 1ttp///0..11/1/220 1115//11977..11974444 2119748
[26] Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi-Velez. Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations. Proceedings of the Twenty-Sixth Interna- tional Joint Conference on Artificial Intelligence, pages
Melbourne, Australia, August 2017. International Joint Conferences on Artificial Intelligence Organization. ISBN 378-0-9999241--9 1-0-33 doi: 10.24963//jeai.201177 2017/371. httht:t:::/wwe http://.waa.ooooorrrroeee///////
12

---

[27] Axel Sauer and Andreas Geiger. Counterfactual generative networks. In 9th International Conference Learning Representattinss ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
URL
[28] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is All you Need. Advances in Neural Information Processing Systems, volume 30, Curran Associates, Inc., 2017. URL
neur cttpriipp.ccpppprr///ceppppppp
http:///hn.n//eee2//4//////eeaaa btw.e/beoooocc11caaaaaaaaaaaaaaa Abstract. html
[29] Hongning Wang. Yue Lu, and Chengxiang Zhai. Latent aspect rating analysis on review text data: a rating regression approach. In In Proceedings of KDD '10, pages 783 792. ACM, 2010.
[30] R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229-2560
2004.
[31] Xing Wu, Tao Zhang. Liangjun Zang. Jizhong Han, and Songlin Hu. Mask and Infill: Applying Masked Lan- guage Model for Sentiment Transfer. In Proceedings of the Twenty-Eightt International Joint Conference Artificial Intelligence, pages 5271-5277, Macao, China, August 2019. International Joint Conferences Artificial Intelligence Organization. ISBN 978-0-9992411 1-4-1. doi: 10.24963/jeci.201999 2019//733. URL
ijcai. http////.rr.//pocede////////////
[32] Lantao Yu. Weinan Zhang, Jun Wang, and Yong Yu. Seggan: Sequence generative adversarial nets with policy gradient. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, 17,I'17, page 2852-2858. AAAI Press, 2017.
[33] Mo Yu, Shiyu Chang, Yang Zhang, and Tommi Jaakkola. Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing (EMNLP-JJCNLP), pages 4092-4101, Hong Kong, China, 2019. Association for Computational
Linguistics. doi: 10. 8653/v1/D19- 1420. URL hhttps://watt http://wwwwwl.l...oo/ org/
wttp:/ebb.rr//atoooogg/////////e aath//nogoooooaeaeeeeo wttpooggyyyDDeeeeeg 1420
[34] Xiangji Zeng, Yunliang Li, Yuchen Zhai, and Yin Zhang. Counterfactual Generator: Weakky-Supprvisee Method Named Entity Recognition. In Proceedings of the 2020 Conference on Empiricul Methods
in Natural Language Processing (EMNLP), pages 7270-7280, Online, November 2020. Association for Computational Linguistics. URL https://wwa ac wweb...ww.... http///nntooooyy/2000
emnlp main. main.590
13


Reviews:
[{"review_id": "dKBUFurw7z", "reviewer": "Reviewer_Sfbi", "summary": "This paper aims to provide a novel rationalization scheme for text classification models in NLP. The authors use the standard setup of a selector and classifier model wherein the selectors selects specific words from the input text and classifier makes a prediction on the selected rationale. Standard issues in such models is that the selector may end up picking on spurious correlations (or \"artifacts\") when picking the rationale if they are correlated with the predicted quantity, instead of true predictive features. The core contribution of the authors is to augment the dataset in such a manner that the correlation between the artifacts and the label is broken.\n\nThe correlation is broken by generating counterfactual examples -- examples where predictive features are *flipped* along with the label but the spurious features are kept the same. Since we don't know a priori which features are predictive vs spurious, the authors hope that the dataset contains slightly less information about the label than the predictive features and provide an analysis in a simple binary feature case, wherein the show when the model may increase the mutual information for predictive features vs rationale error rate of initial selector. \n\nThe modeling part use transformer based models all the way. The setup learns the selector and prediction, in addition to a counterfactual generator (which replaces the rationale words with words predictive of the opposite class), and a discriminator to make sure the words are indeed from same distribution as input text.\n\nExperiments are done on two sentiment analysis multi-aspect datasets and the generated rationales are shown to be more accurate than baseline MMI training. Some qualitative analysis seem to indicate better rationales are produced in terms of coherency and correctness.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While both rationalization and counterfactual augmentation have been performed before, this paper provides first known implementation to correcting for spurious correlation via counterfactuals. The authors provide relatively well researched section for related work although primarily focuses on MMI based rationalization (https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00367/98620/Aligning-Faithful-Interpretations-with-their) . In addition, comparisons to other baselines like Invariant Rationalization has not been made (It is unclear why that is). \n\nThe basic idea behind the paper is reasonably clear from the first reading, although some details can be improved. For example, in line 146, it is unclear what $X^{er}_1$ is. In addition, equations 5 and 6 are not immediately obvious from preceding discussion and can use a few more lines of derivation. Also it seems the derivation assumes that predicted label is binary (the counterfactual label is 1 - Y). What if the tasks is multi class -- how will the derivation generalize ? The details for experiments are reasonably sufficient that I can reproduce the results (assuming the code is also provided). \n\nIn terms of significance, the spurious correlation problem is of course an issue in rationalization literature with not many solutions. the authors provide a novel strategy to solve the same and the results in terms of rationale accuracy is better compared to standard training. Is the improvement because the model is better at not picking up the spurious correlation -- I don't know. An interesting experiment would be to see if the model trained with MMI pick up on rationales from other aspect more than the specific aspect in the correlated setting. \n\nThe biggest issue with the paper appears to be that the experiments are mostly run on two datasets, both belonging to sentiment analysis domain only. While results show improvements in these two datasets, it is unclear if they will generalize beyond these datasets or beyond sentiment analysis domain. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Please include a bit more talk about how you believe the method would generalize beyond simple sentiment analysis (for example, NLI, etc)", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "bIVkAIY0mNF", "reviewer": "Reviewer_SnqY", "summary": "This paper addresses an important problem of correcting the spurious rationales that often get learned due to maximizing the mutual information. Instead, it proposes “counterfactual data augmentation” that lowers the mutual information between spurious signals and the document label. The counterfactual factual data augmentation is an unsupervised method. The paper finally shows strong results on two datasets.\n", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. An important problem to solve and the proposed solution is interesting.\n2. The trade-off between lowering the MMI and preserving the fidelity of the rationales is nicely dealt with.\n3. Did you use Eraser (April 2020) for evaluating rationales? The precision and F1 metrics look the same but it will be important to confirm that. Maybe you could use that benchmark to show results for two datasets.\n4. Check typos such as the incorrect orientation of inverted commas (Line 279, 285, etc.)", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "aUu0B3xbaO", "reviewer": "Reviewer_5Z3P", "summary": "A rationale in NLP is a subset of an original input text on which a classifier operates. Extracting such a rationale is a popular approach for more interpretability because by definition we know the classifier only had access to this reduced text. However, rationales can be nonsensical and can consist of spurious patterns in the dataset.\n\nTo improve the coherence of rationales and to improve generalization, this paper proposes a technique to automatically create counterfactual data. This counterfactual data is created specifically to reduce mutual information between the true target label and irrelevant parts of the original text with regards to the true target label. For example, in “This beer smells great. It tastes terrific.” both sentences have positive sentiment but only the first is relevant when predicting whether the beer has a good smell. To help a model to automatically learn to distinguish, a helpful counterfactual example would be “This beer smells awful. It tastes terrific”.\n\nContributions \n-\tA method to automatically generate counterfactual examples to improve rationale extraction.\n-\tAn analysis of how good an initial rationale extractor should be to be able to expect performance improvements using the proposed method of generating counterfactual examples.\n", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n-\tThe proposed idea is intuitive and interesting.\n-\tExperimental results show improvements over baselines (but also see point 2 under weaknesses).\n\nWeaknesses\n-\tSome parts of the paper are difficult to follow, see also Typos etc below.\n-\tIdeally other baselines would also be included, such as the other works discussed in related work [29, 5, 6]. \n\nAfter the Authors' Response\nMy weakness points after been addressed in the authors' response. Consequently I raised my score.\n- All unclear parts have been answered\n- The authors' explained why the chosen baseline makes the most sense. It would be great if this is added to the final version of the paper.\n\nQuestions\n-\tDo you think there is a way to test beforehand whether I(X_1, Y_1) would be lowered more than I(X_2, Y_1) beforehand?\n-\tOut of curiosity, did you consider first using Aug and then CF.CDA? Especially for the correlated palate result it could be interesting to see if now CF.CDA can improve.\n-\tDid both CDA and MMI have the same lambda_RL (Eq 9) value? From Figure 6 it seems the biggest difference between CDA and MMI is that MMI has more discontinuous phrase/tokens.\n\nTypos, representation etc.\n-\tLine 69: Is X_2 defined as all features of X not in X_1? Stating this explicitly would be great.\n-\tLine 88: What ideas exactly do you take from [19] and how does your approach differ?\n-\tEq 2: Does this mean Y is a value in [0, 1] for two possible labels? Can this be extended to more labels? This should be clarified. \n-\t262: What are the possible Y values for TripAdvisor’s location aspect?\n-\tThe definitions and usage of the various variables is sometimes difficult to follow. E.g. What exactly is the definition of X_2? (see also first point above). When does X_M become X_1? Sometimes the augmented data has a superscript, sometimes it does not. In line 131 the meaning of x_1 and x_2 are reverse, which can get confusing - maybe x’_1 and x’_2 would make it easier to follow together with a table that explains the meaning of different variables?\n-\tSection 2.3: Before line 116 mentioned the change when adding the counterfactual example, it would be helpful to first state what I(X_2, Y_1) and I(X_1, Y_1) are without it.\n\nMinor points\n-\tLine 29: How is desired relationship between input text and target labels defined?\n-\tLine 44: What is meant by the initial rationale selector is perfect? It seems if it were perfect no additional work needs to be done.\n-\tLine 14, 47: A brief explanation of “multi-aspect” would be helpful \n-\tFigure 1: Subscripts s and t should be 1 and 2?\n-\t184: Delete “the” ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There is a broader impact section which discusses the limitations and dangers adequately.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "1ZE92ITU-m", "reviewer": "Reviewer_Z6bp", "summary": "In this paper, the authors discuss selecting rationales to improve classifier performance in an extract then classify setting. The motivation behind this paper is that prior work on rationale selection often seeks to identify a subset of tokens in a document that carry the most information about the target label. However, oftentimes such rationale selectors pick up on spurious correlations that are merely correlated with the label due to confounding and do not express the desired relationship with the label. So even though the desired predictor would rely on “the causal variable”, in this case, predictors often end up relying on the “non causal” factors.\n \nTo address this issue, the authors suggest assisting the rationale selector by lowering the mutual information between non causal features and the label (since a rationale selector will try to identify features that have the maximum mutual information with the label). To do so, the authors propose a framework in which they first use a rationale extractor (similar to that introduced by Lei et al.) to identify rationales for the label of interest. A class conditional masked language model then makes inference on these tokens (while keeping the other tokens constant) to produce new documents with a counterfactual class label. These replacements are performed by the MLM in one step (in a greedy fashion). A discriminator seeks to distinguish between the original document and the generated counterfactual document. This process is run iteratively until convergence. The core idea here is that when the original dataset will be augmented with the counterfactual documents, it should lead to a lower mutual information between the “non rationales” that remained unchanged during counterfactual generation and the label. As the authors recognize, one drawback of automatically generating counterfactual examples is that we assume access to a perfect rationale selector. However, since the motivation of the paper relies on a poor rationale selector which may also reduce the mutual information between causal features and the label, they conjecture that reducing the mutual information between spurious features and the label more than reducing mutual information between causal features and the label could allow us to improve the model performance. The authors conduct experiments on Beer reviews from RateBeer and Tripadvisor reviews. Compared to three baselines, the proposed method appears to generally perform better on the downstream classification metric, and these gains are attributed to better rationale extraction. I think overall the paper represents a valuable contribution but I have some concerns as I lay out below. I will be happy to update my score post rebuttal.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- I think overall this paper is well motivated and presents an interesting framework that could be extended to various problems (since it does not require human annotations).\n- I particularly like the choice of datasets here, along with the extensive analysis as it relates to the error rate associated with the rationale extractor and how it may impact the mutual information between a causal feature and the label and the non causal features and the label.\n- I think the results are quite convincing on each dataset (more on Tripadvisor than Beer Review), however, I would have liked to see experiments with more than one rationale extraction framework, especially since multiple methods have been proposed since Lei et al. Additionally, since the counterfactuals are generated based on sampling, I would have liked to see multiple runs of the experiments and means and standard deviations reported in the tables rather than just a single number. This would especially help with the Beer Review results since in many cases certain baselines achieve comparable performance to the proposed method. There are some other outstanding questions as well that the authors haven’t answered. Lastly, the authors say they set the rationale percentage to 10% for all datasets, but don’t offer a justification for this number. What happens if it is 20% or 30%? I imagine the percentage tokens that are relevant for a particular dataset would be dependent on the dataset to some extent (see [1]).\n- The paper’s presentation needs to be improved, especially as it relates to Section 2. The authors also appear to use causal terms such as “spurious” to mean something different than what they are. For instance, in Lines 68 and 69, they say “We assume some subset of the features, X_1, belong to the target aspect label, Y_1, while other features, X_2, are spurious.” This sentence implies one of two things: (i) either this is a very strong assumption where all features are confounded; or (ii) the authors meant to say non-causal instead of spurious. Currently I am assuming that it is the latter.\n- The paper is lacking a comprehensive qualitative analysis. I think the paper would be considerably better if the authors perform one on generated rationales from various methods (including an error analysis on the same).\n\n[1] Jain, Sarthak, Sarah Wiegreffe, Yuval Pinter, and Byron C. Wallace. \"Learning to Faithfully Rationalize by Construction.\" In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4459-4473. 2020.\n\nAdditional comments:\n- In the last paragraph of Page 2 you talk about two reasons why a rationale selector may not work well. There is in fact a third reason too, where the rationale selector (in end to end models) may in fact end up performing the classification task and the classification layers end up as dummy layers. For instance, if a model just extracts a period as the rationale for positive sentiment and any non-period token as rationale for negative sentiment.\n- For multiple papers, Arxiv versions have been cited even though the papers have been published at peer reviewed conferences/journals. Please update these references.\n\nOther presentation comments:\n- Lines 21 and 22 (‘rationale selector’ and ‘classifier’): In Latex, the opening quotes are ` instead of ‘. Same for Lines 276, 279, and possibly others that I might have missed.\n- Use \\citet when using an intext citation. For instance, in lines 259 and 261, the paper says “reviews collected by [25]” and “rationalization by [5]” respectively. Instead it should read “reviews collected by Wang et al. [25]” and “rationalization by Chang et al. [5]”\n- Line 279: “establish our gains” -> “establish that our gains”\n\n\n---------------------------------------------------------------------------------------------\nI thank the authors for taking the time to respond to my review in such detail. I have updated my score to recommend an acceptance but at the same time I would urge the authors to include the additional experiments I have recommended in my review. I think those will make this a very thorough and influential empirical study. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I believe that the broader impact section could use some more work, especially as it relates to the limitations of current work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "9"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper initially received diverging scores, but after the author response, two reviewers updated their reviews and increased their scores. The reviewers are now unanimous in their assessment that the paper tackles an important problem using an interesting and novel approach, and draws a unique connection between explanations and spurious correlations. It also shows good improvements compared to the baseline. 

There are some additional experiments and analyses requested by the reviewers that I urge the authors to add: more extraction methods, more careful experimental settings (multiple runs, report mean and std), other datasets besides the two (the fact there are several versions of them isn't enough), and qualitative analyses. 

Finally, while not mentioned by the reviewers, one should acknowledge that the evaluation only targets plausibility (by comparing with human explanations) and not faithfulness. This is especially concerning considering the issues with select-predict models raised by Jacovi and Goldberg, "Aligning Faithful Interpretations with their Social Attribution". Therefore, the paper should at the very least discuss these limitations, and more ideally, also evaluate faithfulness (e.g., via sufficiency and comprehensiveness metrics; see Wiegreffe and Marasović, "Teach Me to Explain: A Review of Datasets for Explainable NLP", for a good summary of these concepts). 

Despite these issues, the paper makes valuable contributions and I therefore recommend acceptance. 
Decision: Accept (Poster)