Counterfactual Invariance to Spurious Correlations in Text Classification

1 Introduction . Our focus in this paper is the sort of spurious correlations revealed by “ poke it and see what happens ” testing procedures for machine-learning models . For example , we might test a sentiment analysis tool by changing one proper noun for another ( “ tasty Mexican food ” to “ tasty Indian food ” ) , with the expectation that the predicted sentiment should not change . This kind of perturbative stress testing is increasingly popular : it is straightforward to understand and offers a natural way to test the behavior of models against the expectations of practitioners [ Rib+20 ; Wu+19 ; Nai+18 ] . Intuitively , models that pass such stress tests are preferable to those that do not . However , fundamental questions about the use and meaning of perturbative stress tests remain open . For instance , what is the connection between passing stress tests and model performance on prediction ? Eliminating predictor dependence on a spurious correlation should help with domain shifts that affect the spurious correlation—but how do we make this precise ? And , how should we develop models that pass stress tests when our ability to generate perturbed examples is limited ? For example , automatically perturbing the sentiment of a document in a general fashion is difficult . The ad hoc nature of stress testing makes it difficult to give general answers to such questions . In this paper , we will use the tools of causal inference to formalize what it means for models to pass stress tests , and use this formalization to answer the questions above . We will formalize passing stress tests as counterfactual invariance , a condition on how a predictor should behave when given 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . certain ( unobserved ) counterfactual input data . We will then derive implications of counterfactual invariance that can be measured in the observed data . Regularizing predictors to satisfy these observable implications provides a means for achieving ( partial ) counterfactual invariance . Then , we will connect counterfactual invariance to robust prediction under certain domain shifts , with the aim of clarifying what counterfactual invariance buys and when it is desirable . An important insight that emerges from the formalization is that the true underlying causal structure of the data has fundamental implications for both model training and guarantees . Methods for handing ‘ spurious correlations ’ in data with a given causal structure need not perform well when blindly translated to another causal structure . Counterfactual Invariance Consider the problem of learning a predictor f that predicts a label Y from covariates X . In this paper , we ’ re interested in constructing predictors whose predictions are invariant to certain perturbations on X . Our first task is to formalize the invariance requirement . To that end , assume that there is an additional variable Z that captures information that should not influence predictions . However , Z may causally influence the covariates X . Using the potential outcomes notation , letX ( z ) to denote the counterfactualX we would have seen had Z been set to z , leaving all else fixed . Informally , we can understand perturbative stress tests as a way of producing particular realizations of counterfactual pairs X ( z ) , X ( z′ ) that differ by an intervention on z . Then , we formalize the requirement that an arbitrary change to z does not change predictions : Definition 1.1 . A predictor f is counterfactually invariant to Z if f ( X ( z ) ) = f ( X ( z′ ) ) almost everywhere , for all z , z′ in the sample space of Z . When Z is clear from context , we ’ ll just say the predictor is counterfactually invariant . 2 Causal Structure . Counterfactual invariance is a condition on how the predicted label behaves under interventions on parts of the input data . However , intuitions about stress testing are based on how the true label behaves under interventions on parts of the input data . We will see that the true causal structure fundamentally affects both the implications of counterfactual invariance , and the techniques we use to achieve it . To study this phenomenon , we ’ ll use two causal structures that are commonly encountered in applications ; see Figure 1 . 2.1 Prediction in the Causal Direction . We begin with the case where X is a cause of Y . Example 2.1 . We want to automatically classify the quality of product reviews . Each review has a number of “ helpful ” votes Y ( from site users ) . We predict Y using the text of the product review X . However , we find interventions on the sentiment Z of the text change our prediction ; changing “ Great shoes ! ” to “ Bad shoes ! ” changes the prediction . In the examples in this paper , the covariate X is text data . Usually , the causal relationship between the text and Y and Z will be complex— e.g. , the relationships may depend on abstract , unlabeled , parts of the text such as topic , writing quality , or tone . In principle , we could enumerate all such latent variables , construct a causal graph capturing the relationships between these variables and Y , Z , and use this causal structure to study counterfactual invariance . For instance , if we think that topic causally influences the helpfulness Y , but is not influenced by sen- timent Z , then we could build a counterfactually invariant predictor by extracting the topic and predicting Y using topic alone . However , exhaustively articulating all possible such variables is a herculean task . Instead , notice that the only thing that ’ s relevant about these latent variables is their causal relationship with Y and Z . Accordingly , we ’ ll decompose the observed variable X into parts defined by their causal relationships with Y and Z . We remain agnostic to the semantic interpretation of these parts . Namely , we defineX⊥Z as the part ofX that is not causally influenced by Z ( but may influence Y ) , X⊥Y as the part that does not causally influence Y ( but may be influenced by Z ) , and XY ∧Z is the remaining part that is both influenced by Z and that influences Y . The causal structure is shown in Figure 1a . We see there are two paths that lead to Y and Z being associated . The first is when Z affects XY ∧Z which , in turn , affects Y . For example , a very enthusiastic reviewer might write a longer , more detailed review , which will in turn be more helpful . The second is when a common cause or selection effect in the data generating process induces an association between Z and Y , which we denote with a dashed arrow . For example , if books tend to get more positive reviews , and also people who buy books are more likely to flag reviews as helpful , then the product type would be a common cause of sentiment and helpfulness . 2.2 Prediction in the Anti-Causal Direction . We also consider the case where Y causes X . Example 2.2 . We want to predict the star rating Y of movie reviews from the text X . However , we find that predictions are influenced by the movie genre Z ; e.g. , changing “ Adam Sandler ” ( a comedy actor ) to “ Hugh Grant ” ( a romance actor ) changes the predictions . Figure 1b shows the causal structure . Here , the observed X is influenced by both Y and Z . Again , we decompose X into parts defined by their causal relationship with Z and Y . Here , Z ( and thus X⊥Y ) can be associated with Y through two paths . First , if XY ∧Z is non-trivial , then conditioning on it causes a dependence between Z and Y ( because XY ∧Z is a collider ) . For example , if Adam Sandler tends to appear in good comedy movies but bad movies of other genres then seeing “ Sandler ” in the text induces a dependency between sentiment and genre . Second , Z and Y may be associated due to a common cause , or due to selection effects in the data collection protocol—this is represented by the dashed line between Z and Y . For example , fans of romantic comedies may tend to give higher reviews ( to all films ) than fans of horror movies . 2.3 Non-Causal Associations . Frequently , a predictor trained to predict Y from X will rely on X⊥Y , even though there is no causal connection between Y and X⊥Y , and therefore will fail counterfactual invariance . The reason is that X⊥Y serves as a proxy for Z , and Z is predictive of Y due to the non-causal ( dashed line ) association . There are two mechanisms that can induce such associations . First , Y and Z may be confounded : they are both influenced by an unobserved common causeU . For example , people who review books may be more upbeat than people who review clothing . This leads to positive sentiments and high helpfulness votes for books , creating an association between sentiment and helpfulness . Second , Y and Z may be subject to selection : there is some condition ( event ) S that depends on Y and Z , such that a data point from the population is included in the sample only if S = 1 holds . For example , our training data might only include movies with at least 100 reviews . If only excellent horror movies have so many reviews ( but most rom-coms get that many ) , then this selection would induce an association between genre and score . Formally , the dashed-line causal graphs mean our sample is distributed according to P ( X , Y , Z ) = ∫ P ( X , Y , Z , u | S = 1 ) dP ( u ) where Y , Z are caused by U and are causes of S , and ( X , Y , Z ) are causally related according to the graph . In addition to the non-causal dashed-line relationship , there is also dependency induced by between Y and Z by XY ∧Z . Whether or not each of these dependencies is “ spurious ” is a problem-specific judgement that must be made by each analyst based on their particular use case . E.g. , using genre to predict sentiment may or may not be reasonable , depending on the actual application in mind . However , there is a special case that captures a common intuition for purely spurious association . Definition 2.3 . We say that the association between Y and Z is purely spurious if Y ⊥ X |X⊥Z , Z . That is , if the dashed-line association did not exist ( removed by conditioning on Z ) then the part of X that is not influenced by Z would suffice to estimate Y . 3 Observable Signatures of Counterfactually Invariant Predictors . We now consider the question of how to achieve counterfactual invariance in practice . The challenge is that counterfactual invariance is defined by the behavior of the predictor on counterfactual data that is never actually observed . This makes checking counterfactual invariance impossible . Instead , we ’ ll derive a signature of counterfactual invariance that actually can be measured—and enforced— using ordinary datasets where Z ( or a proxy ) is measured . For example , the star rating of a review as a proxy for sentiment , or genre labels in the movie review case . Intuitively , a predictor f is counterfactually invariant if it depends only on X⊥Z , the part of X that is not affected by Z . To formalize this , we need to show that such a X⊥Z is well defined : Lemma 3.1 . LetX⊥Z be aX-measurable random variable such that , for all measurable functions f , we have that f is counterfactually invariant if and only if f ( X ) is X⊥Z -measurable . If Z is discrete 1 then such a X⊥Z exists . Accordingly , we ’ d like to construct a predictor that is a function ofX⊥Z only ( i.e. , isX ⊥ Z measurable ) . The key insight is that we can use the causal graphs to read off a set of conditional independence relationships that are satisfied by Z , X⊥Z , Y . Critically , these conditional independence relationships are testable from the observed data . Thus , they provide a signature of counterfactual invariance : Theorem 3.2 . If f is a counterfactually invariant predictor : 1 . Under the anti-causal graph , f ( X ) ⊥ Z | Y . 2 . Under the causal-direction graph , if Y and Z are not subject to selection ( but possibly confounded ) , f ( X ) ⊥ Z . 3 . Under the causal-direction graph , if the association is purely spurious , Y ⊥ X | X⊥Z , Z , and Y and Z are not confounded ( but possibly selected ) , f ( X ) ⊥ Z | Y . Remark 3.3 ( Connection to Fairness ) . This result has an interesting interpretation when Z is a protected attribute ( e.g. , sex or race ) that we ’ d like to be fair with respect to . There are many ways of formalizing fairness , which are usually mutually incompatible . In the fairness setting , counterfactual invariance is equivalent to counterfactual fairness [ Kus+17 ; Gar+19 ] , the condition f ( X ) ⊥ Z is equivalent to demographic parity , and the condition f ( X ) ⊥ Z | Y is equivalent to equalized odds [ Meh+19 ] . Theorem 3.2 then says that counterfactual fairness implies either demographic parity or equalized odds , depending on the true underlying causal structure of the problem . Hence , the relationship between disparate fairness notions is clarified by taking the underlying causal structure into account . This also suggests we can take counterfactual fairness as the fundamental notion , then use demographic parity in the causal-confounding case and equalized odds otherwise . However , we leave the ( ethical ) question of whether this is a sound strategy to future work . Causal Regularization Without access to counterfactual examples , we can not directly enforce counterfactual invariance . However , we can require a trained model to satisfy the counterfactual invariance signature of Theorem 3.2 . The hope is that enforcing the signature will lead the model to be counterfactually invariant . To do this , we regularize the model to satisfy the appropriate conditional independence condition . For simplicity of exposition , we restrict to binary Y and Z . The ( infinite data ) regularization terms are marginal regularization = MMD ( P ( f ( X ) | Z = 0 ) , P ( f ( X ) | Z = 1 ) ) ( 3.1 ) conditional regularization = MMD ( P ( f ( X ) | Z = 0 , Y = 0 ) , P ( f ( X ) | Z = 1 , Y = 0 ) ) ( 3.2 ) +MMD ( P ( f ( X ) | Z = 0 , Y = 1 ) , P ( f ( X ) | Z = 1 , Y = 1 ) ) . Maximum mean discrepancy ( MMD ) is a metric on probability measures.2 The marginal independence condition is equivalent to ( 3.1 ) equal 0 , and the conditional independence is equivalent to ( 3.2 ) 1In fact , it suffices that all potential outcomes { Y ( z ) } z are jointly measurable with respect to a single well-behaved sigma algebra ; discrete Z is sufficient but not necessary . 2The choice of regularization by MMD is for concreteness . Any technique for enforcing the independence signatures would do in principle—e.g. , adversarial methods borrowed from the fairness literature . The key point here is the observation that different causal structures imply different independence signatures . equal 0 . In practice , we can estimate the MMD with finite data samples [ Gre+12 ] . When training with stochastic gradient descent , we compute the penalty on each minibatch . The procedure is then : if the data has causal-direction structure and the Y ↔ Z association is due to confounding , add the marginal regularization term to the the training objective . If the data has anti-causal structure , or the association is due to selection , add the conditional regularization term instead . In this way , we regularize towards models that satisfy the counterfactual invariance signature . A key point is that the regularizer we must use depends on the true causal structure . The conditional and marginal independence conditions are generally incompatible . Enforcing the condition that is mismatched to the true underlying causal structure may fail to encourage counterfactual invariance , or may throw away more information than is required . Gap to Counterfactual Invariance The conditional independence signature of Theorem 3.2 is necessary but not sufficient for counterfactual invariance . This is for two reasons . First , counterfactual invariance applies to individual datapoint realizations , but the signature is distributional . In particular , the invariance P ( f ( X ) | do ( Z = z ) ) = P ( f ( X ) | do ( Z = z′ ) ) for all z , z′ would also imply the conditional independence signature . But , this invariance is weaker than counterfactual invariance , since it doesn ’ t require access to counterfactual realizations . Second , f ( X ) ⊥ Z does not imply , in general , that Z is not a cause of f ( X ) . This ( unusual ) behavior can happen if , e.g. , there are levels of Z that we do not observe in the training data , or there are variables omitted from the causal graph that are a common cause of Z and X . Unfortunately , the gap between the signature and counterfactual invariance is a fundamental restriction of using observational data . The conditional independence signature is in some sense the closest proxy for counterfactual invariance we can hope for . In section 5 , we ’ ll see that enforcing the signature does a good job of enforcing counterfactual invariance in practice .

Summary: In this paper, the authors study counterfactual invariance in machine learning models---intervening on “non causal” parts of the input should not change model predictions---and its relation to how models generalize out-of-domain. The authors show that counterfactually invariant predictors rely more on “causal” features as opposed to spurious correlations, thus generalizing to unseen distributions where these spurious correlations may not hold. The authors further propose two regularization schemes (one if the underlying data generation mechanism is represented by a causal model and another if it is an anticausal model) using which we can train counterfactually invariant predictors without access to counterfactual examples---as opposed to most prior work in which researchers often seek to either automatically construct or crowdsource counterfactual examples. They show that these strategies are heavily dependent on the underlying causal mechanisms, and that applying the regularization scheme designed for a causal model could lead to worse performance if the underlying data generation mechanism was anticausal and vice versa. The authors offer empirical evidence on text classification tasks (review helpfulness classification and natural language inference) to support their theoretical findings. I have some concerns about the paper (as discussed below) but overall I believe this paper builds on an exciting line of work and offers good theoretical and empirical contributions.