Counterfactual Invariance to Spurious Correlations in Text Classification

1 Introduction . Our focus in this paper is the sort of spurious correlations revealed by “ poke it and see what happens ” testing procedures for machine-learning models . For example , we might test a sentiment analysis tool by changing one proper noun for another ( “ tasty Mexican food ” to “ tasty Indian food ” ) , with the expectation that the predicted sentiment should not change . This kind of perturbative stress testing is increasingly popular : it is straightforward to understand and offers a natural way to test the behavior of models against the expectations of practitioners [ Rib+20 ; Wu+19 ; Nai+18 ] . Intuitively , models that pass such stress tests are preferable to those that do not . However , fundamental questions about the use and meaning of perturbative stress tests remain open . For instance , what is the connection between passing stress tests and model performance on prediction ? Eliminating predictor dependence on a spurious correlation should help with domain shifts that affect the spurious correlation—but how do we make this precise ? And , how should we develop models that pass stress tests when our ability to generate perturbed examples is limited ? For example , automatically perturbing the sentiment of a document in a general fashion is difficult . The ad hoc nature of stress testing makes it difficult to give general answers to such questions . In this paper , we will use the tools of causal inference to formalize what it means for models to pass stress tests , and use this formalization to answer the questions above . We will formalize passing stress tests as counterfactual invariance , a condition on how a predictor should behave when given 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . certain ( unobserved ) counterfactual input data . We will then derive implications of counterfactual invariance that can be measured in the observed data . Regularizing predictors to satisfy these observable implications provides a means for achieving ( partial ) counterfactual invariance . Then , we will connect counterfactual invariance to robust prediction under certain domain shifts , with the aim of clarifying what counterfactual invariance buys and when it is desirable . An important insight that emerges from the formalization is that the true underlying causal structure of the data has fundamental implications for both model training and guarantees . Methods for handing ‘ spurious correlations ’ in data with a given causal structure need not perform well when blindly translated to another causal structure . Counterfactual Invariance Consider the problem of learning a predictor f that predicts a label Y from covariates X . In this paper , we ’ re interested in constructing predictors whose predictions are invariant to certain perturbations on X . Our first task is to formalize the invariance requirement . To that end , assume that there is an additional variable Z that captures information that should not influence predictions . However , Z may causally influence the covariates X . Using the potential outcomes notation , letX ( z ) to denote the counterfactualX we would have seen had Z been set to z , leaving all else fixed . Informally , we can understand perturbative stress tests as a way of producing particular realizations of counterfactual pairs X ( z ) , X ( z′ ) that differ by an intervention on z . Then , we formalize the requirement that an arbitrary change to z does not change predictions : Definition 1.1 . A predictor f is counterfactually invariant to Z if f ( X ( z ) ) = f ( X ( z′ ) ) almost everywhere , for all z , z′ in the sample space of Z . When Z is clear from context , we ’ ll just say the predictor is counterfactually invariant . 2 Causal Structure . Counterfactual invariance is a condition on how the predicted label behaves under interventions on parts of the input data . However , intuitions about stress testing are based on how the true label behaves under interventions on parts of the input data . We will see that the true causal structure fundamentally affects both the implications of counterfactual invariance , and the techniques we use to achieve it . To study this phenomenon , we ’ ll use two causal structures that are commonly encountered in applications ; see Figure 1 . 2.1 Prediction in the Causal Direction . We begin with the case where X is a cause of Y . Example 2.1 . We want to automatically classify the quality of product reviews . Each review has a number of “ helpful ” votes Y ( from site users ) . We predict Y using the text of the product review X . However , we find interventions on the sentiment Z of the text change our prediction ; changing “ Great shoes ! ” to “ Bad shoes ! ” changes the prediction . In the examples in this paper , the covariate X is text data . Usually , the causal relationship between the text and Y and Z will be complex— e.g. , the relationships may depend on abstract , unlabeled , parts of the text such as topic , writing quality , or tone . In principle , we could enumerate all such latent variables , construct a causal graph capturing the relationships between these variables and Y , Z , and use this causal structure to study counterfactual invariance . For instance , if we think that topic causally influences the helpfulness Y , but is not influenced by sen- timent Z , then we could build a counterfactually invariant predictor by extracting the topic and predicting Y using topic alone . However , exhaustively articulating all possible such variables is a herculean task . Instead , notice that the only thing that ’ s relevant about these latent variables is their causal relationship with Y and Z . Accordingly , we ’ ll decompose the observed variable X into parts defined by their causal relationships with Y and Z . We remain agnostic to the semantic interpretation of these parts . Namely , we defineX⊥Z as the part ofX that is not causally influenced by Z ( but may influence Y ) , X⊥Y as the part that does not causally influence Y ( but may be influenced by Z ) , and XY ∧Z is the remaining part that is both influenced by Z and that influences Y . The causal structure is shown in Figure 1a . We see there are two paths that lead to Y and Z being associated . The first is when Z affects XY ∧Z which , in turn , affects Y . For example , a very enthusiastic reviewer might write a longer , more detailed review , which will in turn be more helpful . The second is when a common cause or selection effect in the data generating process induces an association between Z and Y , which we denote with a dashed arrow . For example , if books tend to get more positive reviews , and also people who buy books are more likely to flag reviews as helpful , then the product type would be a common cause of sentiment and helpfulness . 2.2 Prediction in the Anti-Causal Direction . We also consider the case where Y causes X . Example 2.2 . We want to predict the star rating Y of movie reviews from the text X . However , we find that predictions are influenced by the movie genre Z ; e.g. , changing “ Adam Sandler ” ( a comedy actor ) to “ Hugh Grant ” ( a romance actor ) changes the predictions . Figure 1b shows the causal structure . Here , the observed X is influenced by both Y and Z . Again , we decompose X into parts defined by their causal relationship with Z and Y . Here , Z ( and thus X⊥Y ) can be associated with Y through two paths . First , if XY ∧Z is non-trivial , then conditioning on it causes a dependence between Z and Y ( because XY ∧Z is a collider ) . For example , if Adam Sandler tends to appear in good comedy movies but bad movies of other genres then seeing “ Sandler ” in the text induces a dependency between sentiment and genre . Second , Z and Y may be associated due to a common cause , or due to selection effects in the data collection protocol—this is represented by the dashed line between Z and Y . For example , fans of romantic comedies may tend to give higher reviews ( to all films ) than fans of horror movies . 2.3 Non-Causal Associations . Frequently , a predictor trained to predict Y from X will rely on X⊥Y , even though there is no causal connection between Y and X⊥Y , and therefore will fail counterfactual invariance . The reason is that X⊥Y serves as a proxy for Z , and Z is predictive of Y due to the non-causal ( dashed line ) association . There are two mechanisms that can induce such associations . First , Y and Z may be confounded : they are both influenced by an unobserved common causeU . For example , people who review books may be more upbeat than people who review clothing . This leads to positive sentiments and high helpfulness votes for books , creating an association between sentiment and helpfulness . Second , Y and Z may be subject to selection : there is some condition ( event ) S that depends on Y and Z , such that a data point from the population is included in the sample only if S = 1 holds . For example , our training data might only include movies with at least 100 reviews . If only excellent horror movies have so many reviews ( but most rom-coms get that many ) , then this selection would induce an association between genre and score . Formally , the dashed-line causal graphs mean our sample is distributed according to P ( X , Y , Z ) = ∫ P ( X , Y , Z , u | S = 1 ) dP ( u ) where Y , Z are caused by U and are causes of S , and ( X , Y , Z ) are causally related according to the graph . In addition to the non-causal dashed-line relationship , there is also dependency induced by between Y and Z by XY ∧Z . Whether or not each of these dependencies is “ spurious ” is a problem-specific judgement that must be made by each analyst based on their particular use case . E.g. , using genre to predict sentiment may or may not be reasonable , depending on the actual application in mind . However , there is a special case that captures a common intuition for purely spurious association . Definition 2.3 . We say that the association between Y and Z is purely spurious if Y ⊥ X |X⊥Z , Z . That is , if the dashed-line association did not exist ( removed by conditioning on Z ) then the part of X that is not influenced by Z would suffice to estimate Y . 3 Observable Signatures of Counterfactually Invariant Predictors . We now consider the question of how to achieve counterfactual invariance in practice . The challenge is that counterfactual invariance is defined by the behavior of the predictor on counterfactual data that is never actually observed . This makes checking counterfactual invariance impossible . Instead , we ’ ll derive a signature of counterfactual invariance that actually can be measured—and enforced— using ordinary datasets where Z ( or a proxy ) is measured . For example , the star rating of a review as a proxy for sentiment , or genre labels in the movie review case . Intuitively , a predictor f is counterfactually invariant if it depends only on X⊥Z , the part of X that is not affected by Z . To formalize this , we need to show that such a X⊥Z is well defined : Lemma 3.1 . LetX⊥Z be aX-measurable random variable such that , for all measurable functions f , we have that f is counterfactually invariant if and only if f ( X ) is X⊥Z -measurable . If Z is discrete 1 then such a X⊥Z exists . Accordingly , we ’ d like to construct a predictor that is a function ofX⊥Z only ( i.e. , isX ⊥ Z measurable ) . The key insight is that we can use the causal graphs to read off a set of conditional independence relationships that are satisfied by Z , X⊥Z , Y . Critically , these conditional independence relationships are testable from the observed data . Thus , they provide a signature of counterfactual invariance : Theorem 3.2 . If f is a counterfactually invariant predictor : 1 . Under the anti-causal graph , f ( X ) ⊥ Z | Y . 2 . Under the causal-direction graph , if Y and Z are not subject to selection ( but possibly confounded ) , f ( X ) ⊥ Z . 3 . Under the causal-direction graph , if the association is purely spurious , Y ⊥ X | X⊥Z , Z , and Y and Z are not confounded ( but possibly selected ) , f ( X ) ⊥ Z | Y . Remark 3.3 ( Connection to Fairness ) . This result has an interesting interpretation when Z is a protected attribute ( e.g. , sex or race ) that we ’ d like to be fair with respect to . There are many ways of formalizing fairness , which are usually mutually incompatible . In the fairness setting , counterfactual invariance is equivalent to counterfactual fairness [ Kus+17 ; Gar+19 ] , the condition f ( X ) ⊥ Z is equivalent to demographic parity , and the condition f ( X ) ⊥ Z | Y is equivalent to equalized odds [ Meh+19 ] . Theorem 3.2 then says that counterfactual fairness implies either demographic parity or equalized odds , depending on the true underlying causal structure of the problem . Hence , the relationship between disparate fairness notions is clarified by taking the underlying causal structure into account . This also suggests we can take counterfactual fairness as the fundamental notion , then use demographic parity in the causal-confounding case and equalized odds otherwise . However , we leave the ( ethical ) question of whether this is a sound strategy to future work . Causal Regularization Without access to counterfactual examples , we can not directly enforce counterfactual invariance . However , we can require a trained model to satisfy the counterfactual invariance signature of Theorem 3.2 . The hope is that enforcing the signature will lead the model to be counterfactually invariant . To do this , we regularize the model to satisfy the appropriate conditional independence condition . For simplicity of exposition , we restrict to binary Y and Z . The ( infinite data ) regularization terms are marginal regularization = MMD ( P ( f ( X ) | Z = 0 ) , P ( f ( X ) | Z = 1 ) ) ( 3.1 ) conditional regularization = MMD ( P ( f ( X ) | Z = 0 , Y = 0 ) , P ( f ( X ) | Z = 1 , Y = 0 ) ) ( 3.2 ) +MMD ( P ( f ( X ) | Z = 0 , Y = 1 ) , P ( f ( X ) | Z = 1 , Y = 1 ) ) . Maximum mean discrepancy ( MMD ) is a metric on probability measures.2 The marginal independence condition is equivalent to ( 3.1 ) equal 0 , and the conditional independence is equivalent to ( 3.2 ) 1In fact , it suffices that all potential outcomes { Y ( z ) } z are jointly measurable with respect to a single well-behaved sigma algebra ; discrete Z is sufficient but not necessary . 2The choice of regularization by MMD is for concreteness . Any technique for enforcing the independence signatures would do in principle—e.g. , adversarial methods borrowed from the fairness literature . The key point here is the observation that different causal structures imply different independence signatures . equal 0 . In practice , we can estimate the MMD with finite data samples [ Gre+12 ] . When training with stochastic gradient descent , we compute the penalty on each minibatch . The procedure is then : if the data has causal-direction structure and the Y ↔ Z association is due to confounding , add the marginal regularization term to the the training objective . If the data has anti-causal structure , or the association is due to selection , add the conditional regularization term instead . In this way , we regularize towards models that satisfy the counterfactual invariance signature . A key point is that the regularizer we must use depends on the true causal structure . The conditional and marginal independence conditions are generally incompatible . Enforcing the condition that is mismatched to the true underlying causal structure may fail to encourage counterfactual invariance , or may throw away more information than is required . Gap to Counterfactual Invariance The conditional independence signature of Theorem 3.2 is necessary but not sufficient for counterfactual invariance . This is for two reasons . First , counterfactual invariance applies to individual datapoint realizations , but the signature is distributional . In particular , the invariance P ( f ( X ) | do ( Z = z ) ) = P ( f ( X ) | do ( Z = z′ ) ) for all z , z′ would also imply the conditional independence signature . But , this invariance is weaker than counterfactual invariance , since it doesn ’ t require access to counterfactual realizations . Second , f ( X ) ⊥ Z does not imply , in general , that Z is not a cause of f ( X ) . This ( unusual ) behavior can happen if , e.g. , there are levels of Z that we do not observe in the training data , or there are variables omitted from the causal graph that are a common cause of Z and X . Unfortunately , the gap between the signature and counterfactual invariance is a fundamental restriction of using observational data . The conditional independence signature is in some sense the closest proxy for counterfactual invariance we can hope for . In section 5 , we ’ ll see that enforcing the signature does a good job of enforcing counterfactual invariance in practice .

Summary: The paper casts robustness to spurious correlations as a model's dependence on input data that is not stable under stress testing. Stress testing here corresponds to changing part of the input data that a practitioner believes should not change the prediction. The authors term this counterfactual invariance and consider two different causal structures that could've produced the data. Identifying distribution-level implications (as opposed to unit-level) of these causal structures, the authors propose regularization schemes to enforce when learning predictors to ensure that counterfactual-invariance holds (under certain assumptions). The effectiveness of these schemes is shown both theoretically and empirically.