Counterfactual Invariance to Spurious Correlations in Text Classification
Authors: Victor Veitch
Alexander D'Amour
Steve Yadlowsky
Jacob Eisenstein
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: causality
spurious correlation
natural language
stress test
domain generalization

Abstract: Informally, a 'spurious correlation' is the dependence of a model on some aspect of the input data that an analyst thinks shouldn't matter. In machine learning, these have a know-it-when-you-see-it character; e.g., changing the gender of a sentence's subject changes a sentiment predictor's output. To check for spurious correlations, we can 'stress test' models by perturbing irrelevant parts of input data and seeing if model predictions change. In this paper, we study stress testing using the tools of causal inference. We introduce counterfactual invariance as a formalization of the requirement that changing irrelevant parts of the input shouldn't change model predictions. We connect counterfactual invariance to out-of-domain model performance, and provide practical schemes for learning (approximately) counterfactual invariant predictors (without access to counterfactual examples). It turns out that both the means and implications of counterfactual invariance depend fundamentally on the true underlying causal structure of the data---in particular, whether the label causes the features or the features cause the label. Distinct causal structures require distinct regularization schemes to induce counterfactual invariance. Similarly, counterfactual invariance implies different domain shift guarantees depending on the underlying causal structure. This theory is supported by empirical results on text classification.

Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests
Victor Veitch Alexander D Amour!, Steve Yadlowsky', and Jacob Eisenstein
Google Research University of Chicago
Abstract
Informally, a 'spurious correlation' is the dependence of model some aspect the input data that an analyst thinks shouldn't matter. In machine learning, these have know-it-when-youse sentence's 's subject changes a sentiment predictor's output. To check for spurious correlations,
character; e.g-, changing the gender of a
'stress test' models by perturbing irrelevant parts of input data and seeing if model predictions change. In this paper, we study stress test- ing using the tools of causal inference. We introduce counterfactual invariance as a formalization of the requirement that changing irrelevant parts of the input shouldn't 't change model predictions. We connect counterfactual invariance out- of-domain model performance, and provide practical schemes for learning (ap- proximately) counterfactual invariant predictors (without access counterfactual examples). It turns out that both the means and implications of counterfactual invariance depend fundamentally on the true underlying causal structure of the data in particular, whether the label causes the features or the features cause the label. Distinet causal structures require distinct regularization schemes to induce counterfactual invariance. Similarly, counterfactual invariance implies different domain shift guarantees depending on the underlying causal structure. This the- ory is supported by empirical results on text classification.
Introduction
Our focus in this paper is the sort of spurious correlations revealed by "poke it and see what happens" testing procedures for machine-learning models. For example, we might test a sentiment analysis tool by changing one proper noun for another ("tasty Mcxican food" to "tasty Indian food"), with the expectation that the predicted sentiment should not change. This kind of perturbative stress testing increasingly popular: it is straightforward to understand and offers a natural way to test the behavior of models against the expectations of practitioners [Rib+20; Wu+19; Nai+18].
Intuitively, models that pass such stress tests are preferable to those that do not. However, fundamen- tal questions about the and meaning of perturbative stress tests remain open. For instance, what is the connection between passing stress tests and model performance on prediction? Eliminating spurious correlation should help with domain shifts that affect the spuri- ous correlation- but how do we make this precise? And, how should we develop models that pass stress tests when our ability generate perturbed examples limited? For example, automatically perturbing the sentiment of a document in a general fashion is difficult.
predictor dependence
The ad hoc nature of stress testing makes it difficult give general answers to such questions. In this paper, we will use the tools of causal inference to formalize what means for models to pass stress tests, and use this formalization answer the questions above. We will formalize passing stress tests as counterfactual invariance, a condition on how a predictor should behave when given
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

certain (unobserved) counterfactual input data. We will then derive implications of counterfactual invariance that can measured the observed data. Regularizing predictors to satisfy these ob- servable implications provides means for achieving (partial) counterfactual invariance. Then, we will connect counterfactual invariance to robust prediction under certain domain shifts, with the aim of clarifying what counterfactual invariance buys and when it is desirable.
An important insight that emerges from the formalization is that the underlying causal structure of the data has fundamental implications for both model training and guarantees. Methods for handing 'spurious correlations' data with a given causal structure need not perform well when blindly translated to another causal structure.
Counterfactual Invariance Consider the problem of learning a predictor that predicts label Y from covariates X. In this paper, we'r re interested in constructing predictors whose predictions are invariant to certain perturbations on X. Our first task formalize the invariance requirement.
To that end, assume that there is an additional variable Z that captures information that should not influence predictions. However, Z may causally influence the covariates X. Using the potential outcomes notation, let (=) ) to denote the counterfactual X we would have seen had Z been set to 2, leaving all else fixed. Informally, we can understand perturbative stress tests as way of producing particular realizations counterfactual pairs X (=). (=²) that differ by an intervention on z. Then, formalize the requirement that an arbitrary change to does not change predictions:
Definition 1.1. A predictor f is counterfactually invariant Z if f(X(2)) f(X(z') almost everywhere, for all 2, x' in the sample space of When Z is clear from context, we'll ll just say the predictor is counterfactually invariant.
2 Causal Structure
Counterfactual invariance a condition on how the predicted label behaves under interventions on parts of the input data. However, intuitions about stress testing are based on how the true label behaves under interventions on parts of the input data. We will see that the true causal structure fundamentally affects both the implications of counterfactual invariance, and the techniques we use to achieve it. To study this phenomenon, we'll use two causal structures that are commonly encountered in applications; see Figure
2.1 Prediction in the Causal Direction
We begin with the case where X a cause of Y.
Example 2.1. We want to automatically classify the quality of product reviews. Each review has number of "helpful" votes Y (from site users). We predict Y using the text of the product review X. However, we find interventions on the sentiment of the text change our prediction; changing "Great shoes!" to "Bad shoes!" changes the prediction.
In the examples this paper, the covariate X is text data. Usually, the causal relationship between the text and Y and Z will be complex- e.g., the relationships may depend abstract, unlabeled, parts of the text such as topic, writing quality, or could enu- merate all such latent variables, con- struct a causal graph capturing the relationships between these variables and Y, Z, and use this causal strue- ture to study counterfactual invari- ance. For instance, if we think that topic causally influences the helpful- influenced by sen-
x
X
X
Xynx
tone. principle,
(a) Causal direction
(b) Anticausal direction
Figure 1: Causal models for the data generating process. We de- composc the observed covariate X into latent parts defined by their causal relationships with and Y. Solid arrows denote causal rela- tionships, while dashed lines denote non-causal associations. The differences between these causal structures will turn out to be key for understanding counterfactual invariance.
ness Y. but
timent Z, then we could build a counterfactually invariant predictor by extracting the topic and

---

predicting Y using topic alone. However, exhaustively articulating all possible such variables is a herculean task.
Instead, notice that the only thing that's relevant about these latent variables is their causal relation- ship with Y and Z. Accordingly, we'll decompose the observed variable X into parts defined by their causal relationships with Y and Z. We remain agnostic to the semantic interpretation of these parts, Namely, we define X as the part of X that is not causally influenced by Z (but may influence Y), X as the part that does not causally influence Y (but may be influenced by Z), and XY is the remaining part that is both influenced by Z and that influences Y. The causal structure is shown in Figure la.
We see there are two paths that lead to Y and Z being associated. The first is when affects XYAZ which, in turn, affects Y. For example, a very enthusiastic reviewer might write a longer, more detailed review, which will in turn be more helpful. The second is when a common cause or selection effect in the data generating process induces association between Z and Y, which we denote with a dashed arrow. For example, if books tend to get more positive reviews, and also people who buy books are more likely flag reviews helpful, then the product type would be common cause of sentiment and helpfulness.
2.2 Prediction in the Anti-Causal Direction
We also consider the case where Y causes X.
Example 2.2. We want to predict the star rating of movie reviews from the text X. However, we find that predictions are influenced by the movie genre Z: e.g., changing "Adam Sandler" (a comedy actor) to "Hugh Grant" (a romance actor) changes the predictions.
Figure 1b shows the causal structure. Here, the observed X is influenced by both Y and Z. Again, we decompose X into parts defined by their causal relationship with Z and Y. Here, Z (and thus X²)) can be associated with Y through two paths. First, if XYAZ is non-trivial, then conditioning on it causes a dependence between Z and Y (because XY is a collider). For example, if Adam Sandler tends to appear in good comedy movies but bad movies of other genres then seeing "San- dler" in the text induces dependency between sentiment and genre. Second, Z and Y may be associated due common cause, or due to selection effects in the data collection protocol this is represented by the dashed line between Z and Y. For example, fans of romantic comedies may tend to give higher reviews (to all films) than fans of horror movies.
2.3 Non-Causal Associations
Frequently, predictor trained predict Y from X will rely on XÝ. even though there is no causal connection between Y and Xt, and therefore will fail counterfactual invariance. The reason is that X serves a proxy for Z, and Z is predictive of Y due to the non-causal (dashed line) association.
There are two mechanisms that can induce such associations. First, Y and Z may be confounded: they are both influenced by an unobserved common cause U. For example, people who review books may be more upbeat than people who review clothing. This leads to positive sentiments and high helpfuiness votes for books, creating association between sentiment and helpfulness. Second, Y and Z may be subject to selection: there is some condition (event) S that depends on Y and Z, such that data point from the population is included in the sample only if S 1 holds. For example, our training data might only include movies with at least 100 reviews. If only excellent horror movies have many reviews (but most rom-coms get that many), then this selection would induce an association between genre and score. Formally, the dashed-line causal graphs mean our sample is distributed according to P(X, Y, Z) J P(X, Y, Z, u S 1)dP(u) where Y, Z are caused by U and are causes of S, and (X, Z) are causally related according to the graph.
In addition to the non-causal dashed-line relationship. there is also dependency induced by between Y and Z by XYAZ. Whether or not each of these dependencies is "spurious" a problem-specifie judgement that must be made by each analyst based on their particular use case. E.g., using genre to predict sentiment may or may not be reasonable, depending on the actual application in mind. However, there is special case that captures a common intuition for purely spurious association.
Definition 2.3. We say that the association between Y and Z is purely spurious if Y 1 | Z.
3

---

That is, if the dashed- fash association did not exist (removed by conditioning on 2) then the part of X that is not influenced by Z would suffice estimate Y.
3 Observable Signatures of Counterfactually Invariant Predictors
We now consider the question of how to achieve counterfactual invariance in practice. The challenge is that counterfactual invariance is defined by the behavior of the predictor on counterfactual data that is never actually observed. This makes checking counterfactual invariance impossible. Instead, we'll derive signature of counterfactual invariance that actually can be measured- enforced- using ordinary datasets where Z (or a proxy) is measured. For example, the star rating of a review as proxy for sentiment, genre labels in the movie review case.
Intuitively, a predictor f is counterfactually invariant if it depends only on X the part of X that is not affected by Z. To formalize this, we need to show that such a X is well defined:
Lemma 3.1. Let X be a X-measurable random variable such that, for all measurable functions f, we have that is counterfactually invariant if and only if (X) is X -measurable. If Z is discrete' then such X z exists.
Accordingly, we'd like to construct a predictor that a function of X only (i.c., is X measurable). The key insight is that we can use the causal graphs to read off set of conditional independence relationships that are satisfied by Z, X Y. Critically, these conditional independence relationships testable from the observed data. Thus, they provide a signature of counterfactual invariance:
Theorem 3.2. If f is a counterfactually invariant predictor:
1. Under the anti-causal graph, f(X) 1 Z Y.
2. Under the confounded), f(X) 1 Z.
graph, if Y and Z are not subject to selection (but possibly
3. Under the causal-directier graph, if the association is purely spurious, Y 1 X X Z. and Y and Z are not confounded (but possibly selected), f(X) 1 Z LY.
Remark 3.3 (Connection to Fairness). This result has an interesting interpretation when Z is pro- tected attribute (e.g., sex or race) that we'd d like to be fair with respect to. There are many ways of formalizing fairness, which are usually mutually incompatible. In the fairness setting, counterfactual invariance is equivalent to counterfactual fairness [Kus+ 17; Gar+19], the condition f(X) 1. Z is equivalent to demographic parity, and the condition f(X) 1 Z Y is equivalent to equalized odds [Meh+19]. Theorem 3.2 then says that counterfactual fairness implies either demographic parity equalized odds, depending on the true underlying causal structure of the problem. Hence, the relationship between disparate fairness notions is clarified by taking the underlying causal structure into account. This also suggests we can take counterfactual fairness the fundamental notion, then use demographic parity in the causal-conte confouudio case and equalized odds otherwise. However, we leave the (ethical) question of whether this is sound strategy to future work.
Causal Regularization Without access to counterfactual examples, we cannot directly enforce counterfactual invariance. However, we can require trained model to satisfy the counterfactual in- variance signature of Theorem 3.2. The hope that enforcing the signature will lead the model to be counterfactually invariant. To do this, we regularize the model to satisfy the appropriate conditional independence condition. For simplicity of exposition, we restrict to binary Y and Z. The (infinite data) regularization terms are
marginal regularization = MMD(P(F(X) 2 = 0), P((() conditional regularization MMD(P(f(X) 0. 0). == 1,Y 0)) (3.2)
(3.1)
+ MMD(((((X)) 2 = 0, P()((
Maximum mean discrepancy (MMD) is a metric on probability measures. 2 The marginal indepen- dence condition is equivalent (3.1) equal 0, and the conditional independence is equivalent (3.2)
'In fact, it suffices that all potential outcomes {Y (2)}x are jointly measurable with respect to a single well-behaved sigma algebra; discrete Z is sufficient but necessary.
"The choice of regularization by MMD is for concreteness. Any technique for enforcing the independence
signatures would do in principle-ee e.g. adversarial methods borrowed from the fairesss literature. The key point here is the observation that different causal structures imply differrnt independence signatures.

---

equal O. In practice, we can estimate the MMD with finite data samples [Gre+12]. When training with stochastic gradient descent, we compute the penalty on each minibatch.
The procedure is then: the data has causal-direction structure and the Y ++ Z association is due to confounding, add the marginal regularization term to the the training objective. If the data has anti-causal structure, or the association is due to selection, add the conditional regularization term instead. In this way, we regularize towards models that satisfy the counterfactual invariance signature.
A key point is that the regularizer we must use depends the true causal structure. The conditional and marginal independence conditions are generally incompatible. Enforcing the condition that is mismatched to the true underlying causal structure may fail to encourage counterfactual invariance, or may throw away more information than is required.
Gap to Counterfactual Invariance The conditional independence signature of Theorem 3.2 is necessary but not sufficient for counterfactual invariance. This is for two reasons. First, counter- factual invariance applies to individual datapoint realizations, but the signature is distributional. In particular, the invariance (ff(X) do(Z 2)) P(f(X) do(Z z²)) for all 2, 2' would also imply the conditional independence signature. But, this invariance is weaker than counterfactual invariance, since it doesn't require access to counterfactual realizations. Second, f(X)
does not imply, in general, that Z is not a cause of s(X). This (unusual) behavior can happen if, e.g., there are levels of that we do not observe in the training data, or there are variables omitted from the causal graph that are common cause of Z and X.
fundamental re- striction of using observational data. The conditional independence signature is in some sense the closest proxy for counterfactual invariance we can hope for. In section 5, we'll 'll see that enforcing the signature does good job of enforcing counterfactual invariance in practice.
Unfortunately, the gap between the signature and counterfactual invariance
Performance Out of Domain
Counterfactual invariance is an intuitively desirable property for predictor to have. However, it's not immediately clear how it relates to model performance as measured by, e.g., accuracy. Intu- itively, eliminating predictor dependence a spurious Z may help with domain shift, where the data distribution in the target domain differs from the distribution of the training data. We now turn to formalizing this idea.
First, we must articulate the set of domain shifts be considered. In our setting, the natural thing is to hold the causal relationships fixed across domains, but to allow the non-causal ("spurious") de- pendence between Y and Z to vary. Demanding that the causal relationships stay fixed reflects the requirement that the causal structure describes the dynamics of an underlying real-world process e.g., the author's sentiment is always cause (not an effect) the text in all domains. On the other hand, the dependency between Y and Z induced by either confounding or selection can vary without changing the underlying causal structure. For confounding, the distribution of the confounder may differ between domains e.g., books are rare in training, but common in deployment. For selec- tion, the selection criterion may differ between domains e.g., we include only frequently reviewed movies in training, but make predictions for all movies in deployment.
We want to capture spurious domain shifts by considering domain shifts induced by selection or confounding. However, there is an additional nuance. Changes to the marginal distribution of Y will affect the risk of a predictor, even the absence of any spurious association between Y and Z. Therefore, we restrict to shifts that preserve the marginal distribution of Y.
Definition 4.1. We say that distributions P, Q are causally compatible' if both obey the same causal graph, PYY) Q(Y), and there is a confounder U and/or selection conditions S, such that P-fP P(X, Y, Z =1)d(P)) and Q Q=f Y, Z
for some P(U), Q(U).
We connect counterfactual invariance and robustness to domain shift.
Theorem 4.2. Let Finvar be the set of all counterfactually invariant predictors. Let be either square error or cross entropy loss. And, let s == argmin fe g Ep Ep(L(Y, ((X)) be the counter- factually invariant risk minimizer. Suppose that the target distribution Q is causally compatūble with the training distribution P. Suppose that any of the following conditions hold:
5

---

1. the data obeys the anti-causal graph
2. the data obeys the causal-direction graph, there is no confounding (but possibly selection),
and the association is purely spurious, Y L X z 7, or
3. the data obeys the causal-directio graph, there is no selection (but possibly confounding), the association is purely spurious and the causal effect of X on Y is additive, i.e., the true data generating process is
Y +(Xx)++++( + where Eig
x =0
(4.1)
for some functions 9.9.
Then, the training domain counterfactually invariant risk minimizer is also the target domain coun- terfactually invariant risk minimizer, f fe EQIL(Y, f(X)))
Remark 4.3. The causal case with confounding requires an additional assumption (additive struc- ture) because, e.g., an interaction between confounder and X can yield a case where X & and Y have a different relationship in each domain (whence, out-of-domain learning is impossible).
This result gives a recipe for finding a good predictor in the target domain even without access to any target domain examples training time. Namely, find the counterfactually invariant risk minimizer in the training domain. In practice, we can use the regularization scheme of section 3 to (approximately) achieve this. We'll see in section 5 that this works well in practice.
Optimality Theorem 4.2 begs the question: if the only thing we know about the target set- ting is that it's causally compatible with the training data, is the best predictor the counterfac- tually invariant predictor with lowest training risk? A natural way to formalize this question study the predictor with the best performance in the worst case target distribution. We define Q = {Q Q causally compatible with P} and the Q-minimax predictor f Fminimax argmin F max LY, [X)]. The question is then: what's the relationship between the counterfactually invariant risk minimizer and the minimax predictor?
Theorem 4.4. The counterfactually invariant risk minimizer not Q-minimax in general. However,
under the conditions of Theorem 4.2, if the association is purely spurious, XY L Y X Z, and P(Z, Y) satisfies overlap, then the two predictors are the same. By overlap we mean that P(Z, Y) is discrete distribution such that for all (z, v). if P(2, > 0 then there is some y' # y such that also P(z, > O.
Conceptually, Theorem 4.4 just says that the counterfactually invariant predictor excludes XYAz. even when this information useful in every domain. In the purely spurious case, XY carries no useful information, counterfactual invariance is optimal.
5 Experiments
The main claims of the paper are:
1. Stress test violations be reduced by suitable conditional independence regularization.
2. This reduction will improve out-of-domain prediction performance.
3. To get the full effect, the imposed penalty must match the causal structure of the data.
Setup To assess these claims, we'll examine the behavior of predictors trained with the marginal conditinal regularization on multiple text datasets that have either causal anti-causal structure. We expect to see that marginal regularization improves stress test and out-of-domain performance on data with causal-confounded structure, and conditional regularization improves these on data with anti-causal structure.
For each experiment, we use BERT [Dev+19] finetuned to predict label Y from the text base model. We train multiple cuusally-regularize models on the each dataset. The training varies by whether
the conditional term. That is, we train identical architectures using CrossEntropy + X Regularizer as the objective function, where we vary À and take Regularizer either the marginal penalty, (3.1), conditional penalty, (3.2). We compare these models' predictions on data with causal and anti-causal structure.
marginal penalty, and by the strength of the regularization
See supplement for experimental details.

---

0.10
leat
0.100
ponalty
0.56
0.075
]
10
0.22
0.050
100
0.00
2003
pensity
0.00
con conditional
0.000
conditional
0.002
0, D00
0.004
0.005
monginal
0.84
0.85
0.87
manginal MMD
conditional MMD
Figure 2: Regularizing conditional MMD improves counterfactual invariance on synthetic anti-causal data. Sufficiently high regularization of marginal MMD also improves invariance, but impairs accuracy. Dashed lines show baseline performance of an unregularized predictor. Left: lower conditional MMD implies that predictive probabilities are invariant to perturbation. Although marginal MMD penalization can result in low conditional MMD and good stress test performance, this comes at the cost very low in-domain accuracy. Right: MMD regularization reduces the rate of predicted label flips on perturbed data, with little affect on in- domain accuracy. Conditional MMD regularization reduces predicted label flips 1.4%, while the best result for marginal MMD is 2.8%.
5.1 Robustness to Stress Tests
First, we examine whether enforcing the causal regularization actually helps to enforce counterfac- tual invariance. We create counterfactual (stress test) examples by perturbing the input data and compare the prediction on these. We build the experimental datasets using Amazon reviews from the product category "Clothing, Shoes, and Jewelry" [NLM19].
Synthetic To study the relationship between counterfactual invariance and the distributional sig- nature of Theorem 3.2, we construct a synthetic confound. For each review, we draw Bernoulli random Z, and then perturb the text X so that the common words "the" and "a" carry information about Z: for example, we replace "the" with the token "thexxxxx" when = We take Y to be the review score, and subsample so Y is balanced. This data has anti-causal structure: the text X is written to explain the score Y. Further, we expect that the Y, 2 association is purely spurious, because "the" and "a" carry little information about the label.
We train the models on data where P(Y Z) = 0.3. We then create perturbed stress-test datasets by changing each example X2) to the counterfactual X1 - 2) (using the synthetic model). By measuring the performance of each model on the perturbed data, we can test whether the distribu- tional properties enforced by the regularizers result in counterfactual invariance at the instance level. Figure shows that conditional regularization (matching the anti-causal structure) reduces checklist failures, measured by the frequency that the predicted label changes due perturbation well as the mean absolute difference in predictive probabilities that is induced by perturbation.
Natural To study the relationship in real data, we use the review data in a different way. We now take Z to be the score, binarized as Z e {I or 2 stars, 4 or 5 stars). We use this 2 as a proxy for sentiment, and consider problems where sentiment should (plausibly) not have causal effect on Y. For the causal prediction problem, we take Y to be the helpfulness score of the review (binarized as described below). This is causal because readers decide whether the review is helpful based on the text. For the anti-causal prediction problem, we take Y to be whether "Clothing" is included as cat- egory tag for the product under review (e.g., boots typically do not have this tag). This is anti-causal because the product category affects the text.
Dataset = anticausal
Dataset causs!
0.100
palle M11
0.040
0.075
conditional marginal
0.035
0.050
0.030
0.025
2.6 7.0
10 128
2.6 7.0 penalty coeff
18
120
penalty coeff
Figure 3: Penalizing the MMD matching the causal structure improves stress test performance on nat- ural product review data. Note that penalizing the wrong MMD may not help: the marginal MMD hurts on the anticausal dataset. Perturbations are generated by swapping positive and negative sentiment adjectives in examples.
We control the strength of the spurious association between and Z. In the anti-causal case, this is done by selection: we randomly subset the data to enforce target level of dependence between Y and The causal-direction case with confounding is more complicated. To manipulate confounding strength, we binarize the number of helpfulness votes V in manner determined by
7

---

the target level of association. We take Y = 1[V > Tz] where Tz is a Z Z-dependent threshold, chosen induce a target association. We choose P(Y 0 Z 0) = P(Y Z = 1) 0.3. We balance Z. by subsampling, which also balances Y.
Now, we create stress test perturbations of these datasets by randomly changing adjectives in the examples. Using predefined lists of postive sentiment adjectives and negative sentiment adjectives, we swap any adjective that shows up on a list with randomly sampled adjective from the other list. This preserves basic sentence structure, and thus creates a limited set of counterfactual pairs that differ on sentiment.
Results for differences in predicted probabilities between original and perturbed data are shown in Figure 3. Each point is trained model, which vary in measured MMD on the test data and on sensitivity to perturbations. Recall that the conditional independence signature of Theorem 3.2 necessary but not sufficient for counterfactual invariance, so it's not certain that regularizing to c
regulann
reduce the MMD will reduce perturbation sensitivity. Happily, we see that regularizing to reduce the MMD that matches the causal structure does indeed reduce sensitivity to perturbations.
Notice that regularizing the causally mismatched MMD can have strange effects. Regularizing marginal MMD in the anti-causal case actually makes the model more sensitive to perturbations!
5.2
Domain Shift
Next, we study the effect of causal regularization on model performance under domain shift.
Natural Product Review We
again use the natural Amazon review data described above.
0.50
Woret Doman accurry
0.90
conditinnl MND
0,004
Accarary
0.88
0.98
0.008
For both the causal and anti- causal data, we create multi- ple test sets with variable spu- rious correlation strength. This done in the manner described above, varying P(Y = 0 0
0.85
0.85
0.012
0.016
0.83
0.020
0.83
penalty type condizional marginal
0.50
0.2
0.6
0.8
0.8
0,800
0,525 0. 0.550
0.875
Paull 1/ 1) 13 PaolY- 012-
8]
Accuracy
Anti-Causal Data: conditional regularization improves domain-shift robustness.
0) P(Y 11 2 = = y. Here, Y is the strength of spurious association. The test sets are oft-oo-domain samples. By design, Y is balanced in each dataset, so these samples are causally compatible with the training data. For both the causal and anti-causal datasets, the training data has P(Y 0 7 - 0) = P(Y = 1| Z = 1) = 0.3. We train classi- fier for each regularization type and regularization strength, and measure the accuracy on each test domain. The results are shown in Figure 4.
0.80
Worst Domain Accurscy
marginal MMD
Accurayy
0.70
0.70
0.004
0.c08
0.012
0.60
0.60
0.016
0.02
pen alty type conditional marginal
0.50
0.50
0.2
0.4
0.6
0.8
0.5
0.6
Pra:("
112-111-2p1pp11 PaulY 01
Accuracy
Causal-Direction Data: marginal regularization improves domain-shift robustness.
Figure 4: The best domain-shift robustness is obtained by using the regularizer that matches the underlying causal structure of the data. The plots show out-of-doman accuracy for models trained on the (natu- ral) review data. In each row, the left figure shows out-of oft-of-domain accu- racies (lines are models). with the X-axis showing the level of spurious correlation in the test data (0.3 is the training condition); the right fig- ure shows worst out-of-domail accuracy versus in-domain test accuracy (dots are models).
regularizzaion
First, the unregularized predictors do indeed learn to rely on the spurious association between senti- ment and the label. The accuracy of these predictors decays dramatically as the spurious assocation moves from negative (0.3) to positive-in in the causal case, the unregularized predictor is worse than chance in the 0.8 domain.
unregularzze
Following section 3, the regularization that matches the underlying causal structure should yield predictor that (approximately) counterfactually invariant. Following Theorem 4.2, we expect that good performance of counterfactually-invann performance in each of the other domains. Indeed, we see that this is so. Models that are regularized to have small values of the appropriate MMD do indeed have better out-of domain performance. a
predictor in the training domain should imply good
regulian
8

---

worse group accurecy
woret RECUPACy
conditional
0.6
0.70
marginal MMD
0.4-
0.65
0.2
0.55
0.50
0.0
0.5
2,0
8.0
32.0
128.0
0.75
0.76
0.77
0.79 overallacturas acturasy
0.78
0.80
0.82 0.52
penalit coeff
overall
Figure 5: Conditional MMD penalization improves robustness in anti-causal MNLI data. Marginal regu- larization does not improve over the baseline unregularized model, shown with dashed lines. Left: Conditional regularization improves minimum accuracy across (Y, ?) groups. When overregularized, the predictor returns the same y for all inputs, yielding a worst-group accuracy of O. Right: Conditional MMD regularization sig- nificantly improves worst (Y, 7) group accuracy (y-axis) while only mildly reducing overall accuracy (z-axis).
Such models have somewhat worse in-domain performance, because they no longer exploit the spurious correlation.
MNLI Data For an additional test on naturally-occuringg confounds, we use the multi- genre natural language inference (MNLI) dataset [WNB18]. Instances are concatenations of two sentences, and the label describes the semantic relationship between them, y E {contradiction, entailment, neutral]. There is well-known confound in this dataset: examples where the second sentence contain negation word (e.g., "not") are much more likely to be la- beled as contradictions [Gur+18] Following Sagawa et al. [Sag+20], we set Z to indicate whether one small set of negation words present. Although is derived from the text X, it can be viewed as proxy for latent variable indicating whether the author intended to use negation in the text. This is an anti-causal prediction problem: the annotators were instructed to write text to reflect the desired label [WNB18].
Following Sagawa et al. [Sag+20], we divide the MNLI data into groups by (Y, Z) and compute the "worst group accuracy" across all such groups. Because this is an anti-causal problem, we predict that the conditional MMD is a more appropriate penalty than the marginal MMD. As shown in Figure 5, this prediction holds: conditional MMD regularization dramatically improves performance on the worst group, while only lightly impacting the overall accuracy across groups.
Related work
Several papers draw connection between causality and domain shifts [SS18; SCS19; Arj+20; Mei18; PBM16; RC+18; Zha+13; Sch+12]. Typically, this work considers a prediction setting where the covariates X include both causes and effects of Y. and it is unknown which is which. The goal to learn to predict Y using only its causal parents. Zhang et al. [Zha+13] considers anti-causal domain shift induced by changing P(Y) and proposes data reweighting scheme. By contrast, counterfactual invariance is an example-wist wise notion of invariance, not an invariance across environments. In particular, learning a counterfactually invariant predictor only requires access to data from a single environment. Theorem 4.2 establishes that counterfactual invariance does imply certain domain generalization guarantee over causally- compally- domains. Note though that the notion of causal compatibility is not generally the same as class of domain shifts previously considered. For example, we have invariant performance in the anti-causa setting, but this is ruled out by Arjovsky et al. [Arj+20].
A related body of work focuses on "causal representation learning" [Bes+19; Loc+20; Sch+21; Arj+20]. Our approach follows this tradition, but focuses on splitting X into components defined by their causal relationships with the label Y and additional covariate Z. Rather than attempting to infer the causal relationship between X and Y, we show that domain knowledge of this relationship is essential for obtaining data generation in -umi-supereised learning and transfer learning has also been studied [Sch+21; Sch+12]. In this paper we focus on different implication of the causal anti-causal distinction.
saatterfacttalll-nvviiiaa predictors. The role of causal vs anti-causal
Another line of work considers the case where either the counterfactuals X(=), X(=') are observed for at least some data points, or where it's assumed that there's enough structural knowledge to (learn to) generate counterfactual examples [RPH21; Wu+21; Gar+19; Mit+20;; WZ19; KCC20; 201,20; TAH20J. Kusner et al. [Kus+17] and Garg et al. [Gar+19] in particular examine notion of

---

counterfactual faimess that can be seen as equivalent to counterfactual invariance. In these papers, approximate counterfactuals are produced by direct manipulation of the features (e.g.. change male to female names), generative models (e.g., style transfer of images), or crowdsourcing. Then, these counterfactuals can either be used as additional training data or the predictor can be regularized such that it cannot distinguish between (z) and X² This strategy can be viewed as enforcing counterfactual invariance directly; an advantage is that it avoids the necessary-but -ut-no--sufffiiinnt nuance of Theorem 3.2. However, counterfactual examples can be difficult to obtain for language data in many realistic problem domains, and it may be difficult to learn to generalize from such examples [HLB20].
The marginal and conditional independencies of Theorem 3.2 have appeared in other contexts, As discussed in remark 3.3, if we think of Z as protected attribute and f as a 'fair' classifier, then counterfactual invariance is counterfactual faimess, the marginal independence is demographic par- ity, and the conditional independence is equalized odds [Meh+ 19]. In another setting, one approach to domain adaptation to seek representations & such that either ((X) [e.g.. MBS13; Bak+13; 16n+16; Tze+ 14] or ((X) Y [c.g., MLM19; Yan+17) 17] have the same distribution in each environ- ment. The connection here is subtler; it is inappropriate to interpret Z as a domain label (we only have access to one domain at train time, and all values of 2 are present in each domain). To clarify the connection, consider introducing an extra variable E that labels the domain. For concreteness, consider the anti-causal case where the spurious association between Y and Z is due to observed confounder U. Now, suppose that E' a cause of U. Then, E labels the distribution of the un- observed confounder, and thus different values of E correspond to different causally compatible domains. Now, if we draw the causal graph we can read off that _= F/Y. That is, we arrive at the same conditional independence criterion. However, it's essentially coincidence that this matches the criterion we' dd use if we' d observed Zhe two variables have totally different causal meanings and interpretations. For example, in the review data case, we might take Z to be review score and E to be the website the review is collected from. The review quality doesn't 't need to be counterfactually invariant to the website, and indeed we wouldn't expect it to be!
Finally, this work can be viewed as part of a recent line of work using counterfactuals to examine the connection between example-win robustness (stress testing) and iistributional-levee robustness (domain shift) [c.g., TAH20; RPH21; Kau+21]. Teney et al. [TAH20] use near counterfactuals with different labels in modified training objective, and show improved performance out-of-domain empirically. Robey et al. [RPH21] use image models to generate counterfactuals, and incorporate the model into training, showing out-of-domain improvement. They assume somewhat different causal structur--articllatiee [Kau+21] I] study Sinear-Gaussial model and argue that out-of-domain performance can be used as a signal to distinguish 'causal' and 'spurious' features.
the precise connection is an interesting future direction. Kaushik et al.
7 Discussion
We used the tools of causal inference to formalize and study perturbative stress tests. A main insight of the paper that counterfactual desiderata can be linked to observationally-eestabe conditional independence criteria. This requires consideration of the true underlying causal structure of the data. Done correctly, the link yields a simple procedure for enforcing the counterfactual desiderata, and mitigating the effects of domain shift.
The main limitation of the paper the restrictive causal structures we consider. In particular, we require that X the part of X not causally affected by Z, is also statistically independent of Z in the observed data, However, in practice these may be dependent due common cause. In this case, the procedure here will be overly conservative, throwing away more information than required. Additionally,
not obvious how to apply the ideas described here to more complicated causal situations, which can occur in structured prediction (e.g., question answering). Extending the ideas to handle richer causal structures an important direction for future work. The work described here can provide template for this research program.
10

---

8 Acknowledgements
Thanks Kevin Murphy, Been Kim, Molly Roberts, Justin Grimmer, and Brandon Stewart for feedback on an earlier version.
9 Funding Transparency
This paper was not supported by any third party funding, and the authors have no competing interests to declare.
References
[Arj+20]
M. Arjovsky. L. Bottou, I. Gulrajani, and D. Lopez-Paz. Invariant Risk Minimization. 2020. arXiv: 1907. 02893 [stat.ML] ML]]
[Bak+13]
M. Baktashmotlagh, M. Harandi, B. C. Lovell, and M. Salzmann, "Unsupervised domain adaptation by domain invariant projection". In: Proceedings of the 2013 IEEE International Conference on Computer Vision. 2013.
[Bes+19]
M. Besserve, A. Mehrjou, R. Sun, and B. Schölkopf. Counterfactuals uncover the mod- ular structure of deep generative models. 2019. arXiv: 1812.03253 [cs. LG].
[Dev+19]
Devlin, M.-W. Chang, Lee, and K. Toutanova. "Bert: pre-training of deep bidirec- tional transformers for language understanding". In: Proceedings of the 2019 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Pupers). 2019.
[Gan+16]
Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marc- hand, and V. Lempitsky. Learn. Res. 1 (2016).
coomain-advesaarin training of neural networks". In: Mach.
[Gar+19] S. Garg, V. Perot, N. Limtiaco, A. Taly, E. H. Chi, and A. Beutel. "Counterfactual fair- in text classification through robustness". In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 2019.
[Gre+12]
A. Gretton, M. Borgwardt,
Rasch, B. Schölkopf, and A. Smola. "A kernel two-sample test". In: The Journal of Machine Learning Research 1 (2012).
[Gur+18] S. Gururangan, S. Swayamdipta, O. Levy. R. Schwartz, S. Bowman, and N. A. Smith. "Annotation artifacts in natural language inference data". In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies. Volume 2 (Short Papers). 2018
(HLB20)
W. Huang, H. Liu, and S. R. Bowman. Counterfactually-Augeeee Data Does Not Yield Better Generalization Than Unaugmented Data. 2020. arXiv: 2010. 04762 [cs. CL].
SNLI Training
[KHL20]
D. Kaushik, E. Hovy, and Z. C. Lipton. Learning the Difference that Makes Difference with saatterffcuuallyyAgmmeeett Data. 2020. arXiv: 1909. 12434 [cs.CL].
[Kau+21]
D. Kaushik, Setlur, Hovy, and C. Lipton. Explaining The Efficacy of Counter- factually Augmented Data. 2021. Xiv: 2010. 02114 [cs. CL].
[KCC20]
V. Kumar, A. Choudbary, and E. Cho. "Data augmentation using pre-trained trans- former models". In: Proceedings the 2nd Workshop on Life-long Learning for Spoken Language Systems. 2020.
[Kus+17]
M. Kusner, J. Loftus, C. Russell, and R. Silva. "Counterfactual faimess".. In: Ad- vances in Neural Information Processing Systems. 2017.
[Loc+20]
F. Locatello, B. Poole, G. Raetsch, B. Schölkopf, O. Bachem, and M. Tschannen. "Weakly-supervised disentanglement without compromises". In: Proceedings of the 37th International Conference Machine Learning. 2020.
[MLM19] J. Manders, T. van Laarhoven, and E. Marchiori. Adversarial Alignment of Class Pre- diction Uncertainties for Domain Adaptation. 2019. arXiv: 1804. 04448 [stat. MM....M...

---

[Mch+19] N. Mehrabi, F. Morstatter. N. Saxena, K. Lerman, and A. Galstyan. A Survey on Bias and Fairness Machine Learning. 2019. arXiv: 1908. 09635 [cs.LG] LG].
[Mei18]
N. Meinshausen. "Causality from distributional robustness point of view". In: 2018 JEEE Data Science Workshop (DSW). 2018.
[Mit+20]
Mitrovic, B. McWilliams, J. Walker, L. Buesing, and C. Blundell. Representation Learning via Invariant Causal Mechanisms. 2020. arXiv: 2010. 07922 [cs LG].
[MBS13]
K. Muandet, D. Balduzzi, and B. Schölkopf. "Domain generalization via invariant fea- ture representation" In: Proceedings of the 30th International Conference on Machine Learning. 2013.
[8] A. Naik, A. Ravichander, N. Sadeh, C. Rose, and G. Neubig. "Stress test evaluation for natural language inference". In: Proceedings the 27th International Conference on Computational Linguistics. 2018.
[NLM19] J. Ni, J. Li, and J. McAuley. "Justifying recommendations using distantly-labeled re- views and fined- fined-grained aspects". In: Empirical Methods in Natural Language Process- ing (EMNLP) (2019).
[PBM16] Peters, Bühlmann, and N. Meinshausen, "Causal inference by using invariant pre- diction: identification and confidence intervals". In: Journal of the Royal Statistical Society, Series B (Statistical Methodology) 5 (2016).
[Rib+20] M. T. Ribeiro, T. Wu, C. Guestrin, and Singh. "Beyond accuracy: behavioral test- ing of nlp models with checklist". In: Proceedings of the 58th Anmual Meeting of the Association for Computational Linguisties. 2020.
[RPH21] A. Robey, G. J. Pappas, and H. Hassani. Model-Basee Domain Generalization. 2021. arXiv: 2102. 1 11436 Setat.ML].
[RC+18] M. Rojas-Carulla, B. Schölkopt. R. Turner, and J. Peters. "Invariant models for causal transfer learning". In: Journal of Machine Learning Research 36 (2018).
[Sag+20]
S. Sagawa, P. W. Koh, T. B. Hashimoto, and P. Liang. "Distributionally robust neural networks for group shifts: on the importance of regularization for worst-case general- ization". In: International Conference on Learning Representations. 2020.
[Sch+12] B. Schölkopf, D. Janzing, J. Peters, F. Sgouritsa, K. Zhang, and J. Mooij. "On causal and anticausal learning". In: Proceedings of the 29th International Coference on Inter- national Conference on Machine Learning. 2012.
[Sch+21]
B. Schölkopf, F, Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio. "Toward causal representation learning". In: Proceedings of the IEEE (2021).
[SCS19]
A. Subbaswamy, B. Chen, and S. Saria. A Universal Hierarchy of Shift-Stable Distribu- tions and the Tradeoff Between Stability and Performance. 2019. arXiv: 1905. 11374 [stat.Mi.
[SS18]
A. Subbaswamy and S. Saria. "Counterfactual normalization: proactively addressing dataset shift and improving reliability using causal mechanisms". In: Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence (UAI), 2018. 2018.
[TAH20]
D. Teney, E. Abbasnejad, and A. van den Hengel. "Learning what makes a differ- ence from counterfactual examples and gradient supervision". In: CoRR (2020). arXiv: 2004. 09034.
[Tze+14]
E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell, Deep Domain Confusion: Maximizing for Domuin Invariance. 2014. arXiv: 1412. 3474 [cs.CV).
[WZ19]
J. Wei and K. Zou. "EDA: easy data augmentation techniques for boosting performance on text classification tasks". In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNI.P-JCCNIPP) 2019.
[WNB18] A. Williams, N. Nangia, and S. Bowman. "A broad-coverage challenge corpus for sen- tence understanding through inference". In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018
12

---

[Wu+19]
T. Wu, M. T. Ribeiro, J. Heer, and D. Weld. "Errudite: scalable, reproducible, and testable enror analysis". In: Proceedings of the 57th Annual Meeting of the Associa- tion for Computational Linguistics. 2019.
[Wu+21]
T. Wu, M. T. Ribeiro, J. Heer, and D. S. Weld. "Polyjuice: generating counterfactuals for explaining, evaluating, and improving models". In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics. 2021.
[Yan+17]
H. Yan, Ding, Li, Q. Wang. Y. Xu, and W. Zuo. Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation. 2017. arXiv:
[cs.CV].
[Zha+13]
K. Zhang, Schölkopf, K. Muandet, and Z. Wang. "Domain adaptation under target and conditional shift". In: Proceedings of the 30th International Conference on Machine Learning. 2013.
13


Reviews:
[{"review_id": "tqZDFW1v3Tu", "reviewer": "Reviewer_U7gK", "summary": "In this paper, the authors study counterfactual invariance in machine learning models---intervening on “non causal” parts of the input should not change model predictions---and its relation to how models generalize out-of-domain. The authors show that counterfactually invariant predictors rely more on “causal” features as opposed to spurious correlations, thus generalizing to unseen distributions where these spurious correlations may not hold. The authors further propose two regularization schemes (one if the underlying data generation mechanism is represented by a causal model and another if it is an anticausal model) using which we can train counterfactually invariant predictors without access to counterfactual examples---as opposed to most prior work in which researchers often seek to either automatically construct or crowdsource counterfactual examples. They show that these strategies are heavily dependent on the underlying causal mechanisms, and that applying the regularization scheme designed for a causal model could lead to worse performance if the underlying data generation mechanism was anticausal and vice versa. The authors offer empirical evidence on text classification tasks (review helpfulness classification and natural language inference) to support their theoretical findings. I have some concerns about the paper (as discussed below) but overall I believe this paper builds on an exciting line of work and offers good theoretical and empirical contributions.", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The paper is well motivated, and studies an important question of how predictive models may rely on causally relevant features or spurious correlations and the impacts on model performance out of domain. \n- This paper is a significant contribution to an emerging line of research. To the best of my knowledge, so far there is only one paper that offers a theoretical explanation for the relationship between counterfactual invariance and out-of-domain generalization [1]. This paper takes a different route but arrives at the same conclusion, advancing our theoretical understanding of counterfactual invariance.\n- There are some gaps between the conditional independence signature proposed in the paper and counterfactual invariance, however, to the authors’ credit they acknowledge that as a limitation of their methodology. However, this is not limited to their work, in fact, this as they correctly point out, this is a limitation of working with observational data.\n- The proposed regularization schemes are dependent on the underlying causal mechanism (causal vs anticausal). From a practical point of view, requiring one to have knowledge of the underlying causal direction between the covariates and the label of interest can be a limiting factor in practical utility of proposed regularization schemes, as a mismatch between the enforced independence condition and underlying causal structure can lead to worse out of domain performance.\n- The paper does not offer a discussion of prior work that provides an explanation for the effectiveness of counterfactual invariance as it relates to out-of-domain generalization. I believe this can be easily fixed in a future version. The paper could in general use a better discussion of related work, especially as it relates to Peters et al. (2016).\n- The empirical investigation also has some limitations: first, I think the experiments section could use some background on dataset selection, processing, etc. Second, the whole focus of this paper is on how counterfactually invariant predictors generalize better to out of domain data. However, despite the presence of multiple datasets (representing plausible distribution shifts) for the tasks of how helpful a review is---check Yelp dataset, you can construct various test sets (businesses, restaurants, etc.)---and for natural language inference, the experiments are restricted only to Amazon reviews (from one category) in the former and MNLI in the latter. These results would be more convincing if the authors show that these results (better ood performance) hold on a battery of out-of-domain datasets.\n\nI'm happy to improve my score if the authors can address these concerns.\n\nThere’s a typo on Line 252: “conditinal” → “conditional”\n\nCuriosity question: Did you try experiments with regularizing the causally mismatched MMD on domain shift?\n\nMissing references:\n\n[1] Divyansh Kaushik, Amrith Setlur, Eduard H. Hovy, and Zachary Chase Lipton. \"Explaining the Efficacy of Counterfactually Augmented Data.\" In International Conference on Learning Representations. 2021.\n\n\n--------------------------------------------------------------------------------------\nI have read the author response, including the clarifications offered and the additional details promised by the authors that would appear in the final draft. I will keep my score as before and recommend a strong acceptance for this paper and encourage the authors to continue pursuing this line of research.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I believe that the authors have provided a sound discussion of the limitations of proposed regularization schemes. But as they also showed in their experiments, regularizing causally mismatched MMD could make the model more reliant on spurious patterns, thus I would like the authors to include a discussion of how this limitation could potentially lead to catastrophic failures in critical applications.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "9"}, "extra_text": {}}, {"review_id": "h64b1f4zb9", "reviewer": "Reviewer_NhDL", "summary": "The paper casts robustness to spurious correlations as a model's dependence on input data that is not stable under stress testing. Stress testing here corresponds to changing part of the input data that a practitioner believes should not change the prediction. The authors term this counterfactual invariance and consider two different causal structures that could've produced the data. Identifying distribution-level implications (as opposed to unit-level) of these causal structures, the authors propose regularization schemes to enforce when learning predictors to ensure that counterfactual-invariance holds (under certain assumptions). The effectiveness of these schemes is shown both theoretically and empirically.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper tackles an important problem (of building models robust to spurious correlations) in an interesting and well-motivated way (using independence implications of the assumed causal graph). The main idea of using this independence as regularization in learning representations is well-motivated. The theory supports the claims of generalizing to arbitrary distribution with the same causal structure (and mechanisms) as the one the training data comes from. This, shown in theorem 4.2, is the most illuminating part of the paper, I think.\n\nEmpirically, the proposed method seems easy to implement (at least when Z is of small cardinality) and the experiments indicate sizeable improvements. Showing the performance changing for different levels of regularization shed some light onto the workings of the method.\n\nCould the authors explain why in theorem 4.2, the proposed counterfactual invariance property only helps guarantee out of performance generalization for certain kinds of correlations, like the guarantee for the causal-direction graph requires no confounding but works under selection? I suspect this is because the relationship between the label and the covariates can change but a simple of example of how the performance changes would be helpful.\n\nFurther, could the authors explain the significance of assuming Z is discrete? It seems like the first lemma depends on it.\n\nFinally, could the authors comment about how one would go about choosing the causal structure over the anti-causal structure (vice versa) for some data at hand?\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "One limitation I can think of for this paper is where the additional variable Z has a large number of categories or is high-dimensional; for example, in the task of paraphrase detection, a possible Z is the number of matching words, length etc. The proposed MMD-based independence method might pose problems because the groups for each possible value-pair of Y, Z may be too small. in the high-dimensional case, most of these groups may have only a single sample. \n", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "WESD0gMSECK", "reviewer": "Reviewer_92Ro", "summary": "This paper presents conditions that are necessary for counterfactual invariant predictions under certain assumptions. They derive these from causal graphs and show the connection to worst-case domain generalization. Experiments on natural language datasets show that the proposed conditions do lead to a decrease in worst-domain error.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "[Update] Increased my score based on the rebuttal. \n\nThe paper presents a nice characterization of causal and anti-causal prediction tasks and the associated independence properties that follow from their respective causal graphs. The examples in Section 2 are useful to understand the main point. Rather than directly regularizing using ground-truth counterfactual examples as in past work, the paper claims to propose \"necessary\" conditions that can be measured by data. The connection to out-of-domain generalization is good for completeness.\n\nMy only reservation is that the paper's key result (the marginal or conditional independence regularizers) is very, very similar to recent line of work in domain generalization once you think of Z as a domain (different domains have perturbations on the non-causal features or their relationship with Y). As a result, it misses an opportunity to address the limitations that have been described for such regularizers. \n\n1) The \"marginal regularization\"  is the domain invariance condition for representation learning in domain adaptation/generalization (DG) and has been proposed by Muandet (2013), [Li et al](https://openaccess.thecvf.com/content_cvpr_2018/papers/Li_Domain_Generalization_With_CVPR_2018_paper.pdf). Further, recent work has also shown the fundamental issues with this regularization (see [Zhao et al.](https://arxiv.org/abs/1901.09453), [Akuzawa et al.](https://arxiv.org/abs/1904.12543)). The simplest case is where they do not work is when the distribution of Y, P(Y) changes across domains. Perhaps this is why authors needs to assume P(Y)=Q(Y) in Def. 4.1, but this seems a very strong assumption. It will be good to address this literature upfront when the regularizer is proposed---that will also help to justify Def. 4.1 more naturally.\n\n2) The \"conditional regularization\" term is the class-conditional domain invariance condition (see, e.g.,  [Li et al. (b)](https://arxiv.org/abs/1807.08479), many works actually use the same metric MMD). Can the authors acknowledge this past work upfront in their manuscript since the regularization term is exactly the same? Again, as with 1, recent work has shown the limitations of this regularizer (see [Mahajan et al.](https://arxiv.org/abs/2006.07500)). That work claims that this regularizer is correct only when the distribution of causal features remains the same across domains. It will be good for the authors to discuss their regularizer in light of these results. There is an interesting discussion here, on whether there is a difference in assumptions between the DG literature and this work, or simply a notational difference.\n\nEvaluation: The experiments on natural language tasks are a good addition to the DG literature which mostly focuses on image classification.  For that reason, it may be difficult to compare to the past work in DG. But perhaps the authors can compare their regularizers to simpler DG methods that can work for language tasks too, like contrastive matching over items with the same class but different Z (see [Motiian et al.](https://openaccess.thecvf.com/content_ICCV_2017/papers/Motiian_Unified_Deep_Supervised_ICCV_2017_paper.pdf)) How do the results compare? It is a bit odd to have no baselines on non-causal regularization approaches that have been proposed before.\n\nOverall, I think this paper adds a nice twist to the standard domain generalization/adaptation regularizers and adapts it for robust learning with confounders. \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"ethical_concerns": "NA", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2 hours"}, "extra_text": {}}, {"review_id": "I3qbSRwaizC", "reviewer": "Reviewer_ZoEc", "summary": "In this work, the authors focus on how to build models that are robust to spurious correlations. The focus of the paper is language datasets. The authors work with two types of DAGs -- a) association of the label with the invariant feature is anti-causal, and b) association of the label with invariant feature is causal. The authors propose a notion of invariance, which they call counterfactual invariance, that they require the predictor to satisfy. Under this notion, the prediction $f$ has to remain invariant under all the potential outcomes that $X$ can have under variations to the spurious feature $Z$. Since the notion of counterfactual invariance is hard to enforce as counterfactuals are not observed, the authors come up with necessary conditions that are then used as proxies to enforce counterfactual invariance. The necessary conditions are different in the two directions causal and anti-causal. Further, the authors analyze the relationship between best counterfactual invariant predictor on train domain and target domain (under appropriate notion of changes allowed in target domain). Finally, the authors establish conditions for min-max optimality of the predictor that is learned. ", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is very nicely written. The application to language models is particularly interesting and novel. I have a few concerns that I highlight below. \n\n1. **Counterfactual invariance notion in fairness**  The authors towards the end in the paper recognize that the notion of counterfactual invariance in an equivalent form already exists [Silva et al.]. Also, from the notion of counterfactual invariance, the necessary conditions that authors identified, which are very well known as demographic parity and equality of odds. Since there is so much overlap with the fairness literature, it is quite important to see the discussion where the notions are introduced and also following the theorem. Also, the authors should explain if they add theory to fairness literature. One would argue that there are papers that establish that equality of odds and demographic parity are not compatible is well-known. I believe that showing how one notion is better suited for one type of DAG and another for another type of DAG might be less well known in fairness literature. Also, another contribution that authors could highlight w.r.t fairness literature might be the connection of counterfactual fairness to demographic parity and equality of odds. \n\n2. **Counterfactual invariance notion in domain generalization** The notion that the authors introduced is very closely related and equivalent under some conditions to the notion of G-invariance in [Robey et al.]. In Robey et al., the authors introduced the notion that the prediction $f(X) = f(G(X,e))$. $G(X,e)$ can be understood as a model that generates the potential outcomes under different environments. I believe a discussion in the same place as to how the notion introduced by the authors and the notion in [Robey et al.] is related is very important. \n\n\n3. **Reconciling [Robey et al.] and the claims of the authors** In [Robey et al.], the authors learn the model $G$. In the terminology used in the current paper, we can say that the authors learned the model to generate the potential outcomes by observing the variations in the observational data. Finally using the model $G$ the authors enforce $G$-invariance. However, in this paper the authors claim that it is necessarily needed to know the causal or anti-causal direction and then use different penalties accordingly.  How to reconcile that we could learn the model for potential outcomes and not have to worry about two different necessary conditions - - demographic parity and equality of odds. A detailed discussion on whether it is really the case we cannot circumvent the need for knowing the causal direction is very important. \n\n\n4. **Examples of**  $X_Z^{\\perp}$ The authors should actually provide mathematical examples in the paper to show how to decompose $X$ in the two DAGs. Currently, I had to look at the proofs to better understand this and I believe some examples of this can really help the reader. \n\n*** \n\n## References\n\n[Silva et al.] Kusner, Matt J., Joshua R. Loftus, Chris Russell, and Ricardo Silva. \"Counterfactual fairness.\" arXiv preprint arXiv:1703.06856 (2017). \n\n[Robey et al.]  Robey, A., Pappas, G. J., & Hassani, H. (2021). Model-Based Domain Generalization. arXiv preprint arXiv:2102.11436.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors have provided the limitations and have stated N/A for negative societal impact. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers have given very positive reviews with positive revisions after the discussion.

Briefly, the notion of counterfactual invariance discussed, its relation to stability against spurious variation and different conditions/regularization depending on the DAG model that is needed to enforce it have been well appreciated. Empirical results on text have been satisfactory as well. 

There remains some minor concerns which authors have promised to address in their response. I encourage the authors to add additional material that they have promised to various reviewer concerns.


Decision: Accept (Spotlight)