SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Abhinav Kumar
abhinavkumar.wk@gmail.com
Amit Deshpande
amitdesh@microsoft.com
Amit Sharma
amshar@microsoft.com
In many classification datasets, the task labels are spuriously correlated with some input attributes. Classifiers trained on such datasets often rely on these attributes for prediction, especially when the spurious correlation is high, and thus fail to generalize whenever there is a shift in the attributes' correlation at deployment. If we assume that the spurious attributes are known a priori, several methods have been proposed to learn a classifier that is invariant to the specified attributes. However, in real-world data, information about spurious attributes is typically unavailable. Therefore, we propose a method that automatically identifies spurious attributes by estimating their causal effect on the label and then uses a regularization objective to mitigate the classifier's reliance on them. Although causal effect of an attribute on the label is not always identified, we present two commonly occurring data-generating processes where the effect can be identified. Compared to recent work for identifying spurious attributes, we find that our method, AutoACER, is more accurate in removing the attribute from the learned model, especially when spurious correlation is high. Specifically, across synthetic, semi-synthetic, and real-world datasets, AutoACER shows significant improvement in a metric used to quantify the dependence of a classifier on spurious attributes ( $\Delta$ Prob), while obtaining better or similar accuracy. Empirically we find that AutoACER mitigates the reliance on spurious attributes even under noisy estimation of causal effects or when the causal effect is not identified. To explain the empirical robustness of our method, we create a simple linear classification task with two sets of attributes: causal and spurious. Under this setting, we prove that AutoACER only requires the ranking of estimated causal effects to be correct across attributes to select the correct classifier.
When trained on datasets where the task label is spuriously correlated with some input attributes, machine learning classifiers have been shown to rely on these attributes (henceforth known as spurious attributes) for prediction [10, 23, 11]. For example in a sentiment classification dataset that we evaluate (Twitter-AAE [4]), a demographic attribute like race could be spuriously correlated with the sentiment of a sentence [9]. Classifiers trained on such datasets are at risk of failure during deployment when the correlation between the task label and the spurious attribute changes [2, 22, 30, 10].
Assuming that a set of auxiliary attributes is available at training time (but not at test time), several methods have been proposed to mitigate the classifier's reliance on the spurious attributes. The first category of methods assumes that the spurious attributes are known a priori. They develop regularization [35, 15, 22], optimization [30] or data-augmentation [17, 16, 14] strategies to train a classifier invariant to the specified spurious attributes. The second category of methods relaxes the assumption of known spurious attributes by automatically identifying the spurious attributes and
STAGE 1: Estimate Causal Effect STAGE 2: Causal Effect Regularization
Figure 1: To reduce dependence on spurious attributes in a classifier, our method AutoACER has 20 stages. Stage 1 (middle panel) estimates the causal effect of attributes on the task label. Stage 2 (right panel) ensures that the causal effect of each attribute on the classifier's prediction matches an estimated causal effect on the task label. In the bottom panel, due to high spurious correlation, the estimation of average causal effect is error-prone. AutoACER works well even in such highly correlation scenarios as shown by the Spuriousness score,  $\Delta\mathrm{Prob}$  (see definition in §4.2).
regularizing the classifier to be invariant to them. To identify spurious attributes, these methods impose assumptions on the type of spurious correlation they deal with. For example, they may assume that attributes are modified in input via symmetry transformation [25] or a group transformation [24], or the data-generating process follows a specific graph structure such as anti-causal [38]. However, all these methods assume a strict dichotomy—every attribute is either spurious or not—which makes them susceptible to imposing incorrect invariance whenever there is a mistake in identifying the spurious attribute.
In this paper, we propose a method that regularizes the effect of attributes on a classifier proportional to their average causal effect on the task label, instead of strictly binning them as spurious (or not) using an arbitrary threshold. We propose a two-step method that (1) uses an effect estimation algorithm to find the causal effect of a given attribute; (2) regularizes the classifier proportional to the estimated causal effect for each attribute estimated in the first step. We call the method AutoACER, Causal Effect Regularization for automated removal of spurious correlations. If the estimated causal effects are correct, AutoACER is perfect in removing a classifier's reliance on spurious attributes (i.e., attributes with causal effect close to zero), while retaining the classifier's accuracy. But in practice it is difficult to have good estimates of the causal effect due to issues like non-identifiability, noise in the labels [39], or finite sample estimation error [3]. Under such conditions, we find that AutoACER is more robust than past work [25] since it does not group the attributes as spurious or causal and regularizes the classifier proportional to the estimated effect.
We analyze our method both theoretically and empirically. First, we provide the conditions under which causal effect is identified. An implication of our analysis is that causal effect is identified whenever the relationship between the attribute and label is fully mediated through the observed input (e.g., text or image). This is often the case in real-world datasets where human labellers use only the input to generate the label. Even if the causal effect is not identified, we show that AutoACER is robust to errors in effect estimation. Theoretically, on a simple classification setup with two sets of attributes—causal and spurious—we prove that only the correct ranking of the causal effect estimates is needed to learn a classifier invariant to spurious attributes. For the general case with multiple disentangled high-dimensional causal and spurious attributes, under the same condition on correct causal effect rankings, we prove that the desired classifier (that does not use spurious attributes) is preferred by AutoACER over the baseline ERM classifier.
To confirm our result empirically, we use three datasets that introduce different levels of difficulty in estimating the causal effect of attributes: (1) Syn-Text: a synthetic dataset that includes an unobserved confounding variable and violates the identifiability conditions of causal effect estimators; (2) MNIST34: a semi-synthetic dataset from [24], where we introduce noise in the image labels; (3) Twitter-AAE:, which is a real-world text dataset. To evaluate the robustness of methods to spurious correlation, we create multiple versions of each dataset with varying levels of correlation with the spurious attribute, thereby increasing the difficulty to estimate the causal effect. Even with a noisy
estimate of the causal effect of attributes, our method shows significant improvement over previous algorithms in reducing the dependence of a classifier on spurious attributes, especially in the high correlation regime. Our contributions include,
- Causal effect identifiability guarantee for two realistic causal data-generating processes, which is the first step to automatically distinguish between causal and spurious attributes.
- A method, AutoACER, to train a classifier that obeys the causal effect of attributes on the label. Under a simplified setting, we show that it requires only the correct ranking of attributes.
- Evaluation on three datasets showing that AutoACER is effective at reducing spurious correlation even under noisy, high correlation, or confounded settings.
2 OOD Generalization under Spurious Correlation: Problem Statement
For a classification task, let  $(\pmb{x}^i,y^i,\pmb{a}^i)_{i = 1}^n\sim \mathcal{P}_m$  be the set of example samples from the data distribution  $\mathcal{P}$ , where  $\pmb{x}^i\in \mathbf{X}$  are the input features,  $y^{i}\in Y$  are the task labels and  $\pmb{a}^{i} = (a_{1}^{i},\dots ,a_{k}^{i})$ ,  $a_j^i\in A_j$  are the auxiliary attributes, henceforth known as attributes for the example  $\pmb{x}^i$ . We use  $"x_{a_j}"$  to denote an example "x" to have attribute  $a_{j}\in A_{j}$ . These attributes are only observed during the training time. The goal of the classification task referred to as task henceforth, is to predict the label  $y^{i}$  from the given input  $\pmb{x}^i$ . The task classifier can be written as  $c(h(\pmb {x}))$  where  $h:X\to Z$  is an encoder mapping the input  $\pmb{x}$  to a latent representation  $z\coloneqq h(x)$  and  $c:Z\rightarrow Y$  is the classifier on top of the representation  $Z$ .
Generalization to shifts in spurious correlation. We are interested in the setting where the data-generating process is causal [31] i.e. there is a certain set of causal attributes that affect the task label  $y$ . Upon changing these causal attributes, the task label changes. Apart from these attributes, there could be other attributes defining the overall data-generating process (see Fig. 2 for examples). A attribute  $c \in \mathcal{C}$  is called spurious attribute if it is correlated with the task label in the training dataset and thus could be used by a classifier trained to predict the task label, as a shortcut [38, 22]. But their correlation could change at the time of deployment, affecting the classifier's accuracy.
Using the attributes available at training time, our goal is to train a classifier  $c(h(x))$  that is robust to shift in correlation between the task label and the spurious attributes. We use the fact that changing spurious attributes will not lead to a change in the task label i.e. they have zero causal effect on the task label  $y$ . Hence, we use the estimated causal effect of an attribute to automatically identify its degree of spuriousness. For a spurious attribute, its true causal effect on the label is zero and hence the goal is to ensure its causal effect on the classifier's prediction is also zero. More generally, we would like to regularize the effect of each attribute on the classifier's prediction to its causal effect on the label. In other words, unlike existing methods that aim to discover a subset of attributes that are spurious [24, 25, 38], we aim to estimate the causal effect of each attribute and match it. Since our method avoids hard categorization of attributes that downstream regularizers need to follow, we show that is a more robust way of handling estimation errors when spurious correlation may be high.
3 Causal Effect Regularization: Minimizing reliance on spurious correlation
We now describe our method to train a classifier that generalizes to shift in attribute-label correlation by automatically identifying and imposing invariance w.r.t to spurious attributes. In §3.1, we provide sufficient conditions for identifying the causal effect, a crucial first step to detect the spurious attributes. Next, in §3.2 and §3.3, we present the AutoACER method and its theoretical analysis.
3.1 Causal Effect Identification
Since our method relies on estimating the causal effect of attributes on the task label, we first show that the attributes' causal effect is identified for many commonly occurring datasets. The identifiability of the causal effect of an attribute depends on the data-generating process (DGP). Below we present two illustrative DGPs (DGP-1 and DGP-2 from Fig. 2) under which the causal effect is identified.  $DGP - 1$  refers to the case where the task label is generated based on the observed input  $X$  which in turn may be caused by observed and unobserved attributes.  $DGP - 1$  is common in many real-world datasets where the task labels are annotated based on the observed input  $X$  either automatically using some deterministic function or using human annotators [17, 29]. Thus  $DGP - 1$  is applicable for all
the settings where the input  $X$  has all the sufficient information for creating the label. Mouli and Ribeiro [25] consider another DGP where a set of transformations (like rotation or vertical-flips in image) over a base (unobserved) image  $x$  generates the input  $X$ . We adapt their graph to our setting where we include each observed attribute as a node in the graph, along with hidden confounders. Depending on whether they are spurious or not, these attributes may cause the task label  $Y$ . DGP-2 and DGP-3 represent two such adaptations from their work, with the difference that DGP-3 also allows unobserved attributes to cause the task label.
Generally for identifying the causal effect one assumes that we only have access to observational data  $(\mathcal{P})$ . But here we also assume access to the interventional distribution for the input,  $P(X|do(A))$  where the attribute  $A$  is set to a particular value, as in [24]. This is commonly available in vision datasets via data augmentation strategies over attributes like rotation or brightness, and also in text datasets using generative language models [5, 37]. Having access to interventional distribution  $P(X|do(A))$  could help us identify the causal effect in certain cases where observational data  $(\mathcal{P})$  alone cannot as we see below.
Proposition 3.1. Let DGP-1 and DGP-2 in Fig. 2 be the causal graphs of two data-generating processes. Let  $A, C, S$  be different attributes,  $X$  be the observed input,  $Y$  be the observed task label and  $U$  be the unobserved confounding variable. In DGP-2,  $x$  is the (unobserved) core input feature that affects the label  $Y$ . Then:
1. DGP-1 Causal Effect Identifiability: Given the interventional distribution  $P(X|do(A))$ , the causal effect of the attribute  $A$  on task label  $Y$  is identifiable using observed data distribution.
2. DGP-2 Causal Effect Identifiability: Let  $C$  be a set of observed attributes that causally affect the task label  $Y$  (unknown to us),  $S$  be the set of observed attributes spuriously correlated with task label  $Y$  (again unknown to us), and let  $\mathcal{V} = C \cup S$  be the given set of all the attributes. Then if all the causal attributes are observed then the causal effect of all the attributes in  $V$  can be identified using observational data distribution alone.
Proof Sketch. (1) We show that we can identify the interventional distribution  $P(Y|do(A))$  which is needed to estimate the causal effect of  $A$  on  $Y$  using the given observational distribution  $P(Y|X)$  and interventional distribution  $P(X|do(A))$ . (2) For both causal attribute  $C$  and spurious attribute  $S$ , we show that we can identify the interventional distribution  $P(Y|do(S))$  or  $P(Y|do(C))$  using purely observational data using the same identity without the need to know whether the variable is causal or spurious a priori. See §B for proof.
Note that DGP-1 and DGP-2 are not exhaustive. We provided them simply as illustrative examples to show that there exist DGPs, corresponding to commonly occurring datasets, where the attributes' effect is identified. However, the effect is not identified in DGP-3 and possibly other DGPs. Moreover, in practice, the underlying DGP for a given task may not be known or mis-specified. Hence, in §3.2, we design our method AutoACER such that it does not depend on knowledge of the true DGP. When the effect is identified, our method is theoretically guaranteed to work well. Empirically, as we show in Section 4, our method also works to remove the spurious correlation for datasets corresponding to DGP-3 where the causal effect of A on Y is not identified. In comparison, prior methods like Mouli and Ribeiro [25] provably fail under DGP-3 (see §C for the proof that their method would fail to detect the spurious attribute).
3.2 AutoACER: Causal Effect Regularization for predictive models
Our proposed method proceeds in two stages. In the first stage, it uses a causal effect estimator to identify the causal effect of an attribute on the task label. Then in the second stage, it regularizes the classifier proportional to the causal effect of every attribute.
Stage 1: Causal Effect Estimation. Given a set of attributes  $\mathcal{A} = \{a_1, \dots, a_k\}$ , the goal of this step is to estimate the causal effect  $(TE_{a_i})$  of every attribute  $a \in \mathcal{A}$ . The causal effect of a attribute on the label  $Y$  is defined as the expected change in the label  $Y$  when we change the attribute (see §A for formal definition). If the data-generating process of the task is one of DGP-1 or DGP-2, our Prop 3.1 gives us sufficient conditions needed to identify the causal effect. Then one can use appropriate causal effect estimators that work under those conditions or build their own estimators using the closed form causal effect estimand given in the proof of Prop 3.1. However, to build a
DGP-1
DGP-2
DGP-3
Figure 2: Causal graphs showing different data-generating processes. Shaded nodes denote observed variables; the red arrow denotes an unknown target relationship. Node X denotes the observed input features and node Y denotes the task label in all the DGPs. Node A refers to attributes; could denote either causal or spurious attribute. If A is a causal attribute then the red arrow will be present in the underlying true graph otherwise not. Node U denotes the unobserved confounding variable which could be the potential reason for spurious correlation between an attribute (A or S) and Y. The node C in DGP-2 denotes causal attributes and node S denotes spuriously correlated attributes. The node x in DGP-2 and DGP-3 denotes the unobserved causal attribute that creates the label Y along with other observed causal attributes (C or A). In the first two graphs, the causal effect of attributes (A and C, S respectively) on Y is identified (see Prop 3.1).
general method, we propose a technique that does not assume knowledge of the DGP. Specifically, we use a conditioning-based estimator (based on the backdoor criterion [27]) and assume the observed variables (X) as the backdoor variables. There is a rich literature on estimating causal effects for high dimensional data [12, 33, 6]. We use a deep learning-based estimator from Chernozhukov et al. [6] to estimate the causal effect (henceforth called Riesz). Given the treatment along with the rest of the covariates, it learns a common representation that approximates backdoor adjustment to estimate the causal effect (see §E for details). Even if the causal effect in the relevant dataset is identifiable, we might get a noisy estimate of the effect due to finite sample error or noise in the labels. Later in §3.3 and §4 we will show that our method is robust to error in the causal effect estimate of attributes both theoretically and empirically. Finally, as a baseline effect estimator, we use the direct effect estimator (henceforth called Direct) defined as  $\mathbb{E}_X(\mathbb{E}(Y|X,a = 1) - \mathbb{E}(Y|X,a = 0))$  for attribute  $a\in \mathcal{A}$  that has limited identifiability guarantees (see §E for details).
Stage2: Regularization. Here our method regularizes the model prediction with the estimate of causal effect  $= \{TE_{a_1},\dots,TE_{a_k}\}$  of each attribute  $\mathcal{A} = \{a_1,\dots,a_k\}$ . The loss objective is,
$$
\mathcal {L} _ {\text {A u t o A C E R}} := \mathbb {E} _ {(\boldsymbol {x}, y) \sim \mathcal {P}} \left[ \mathcal {L} _ {\text {t a s k}} (c (h (\boldsymbol {x})), y) + R \cdot \mathcal {L} _ {\text {R e g}} (\boldsymbol {x}, y) \right] \tag {1}
$$
where  $R$  is the regularization strength hyperparameter,  $\mathcal{P}$  is the training data distribution (§2). The first term  $\mathcal{L}_{task}(c(h(\pmb{x})), y)$  can be any training objective e.g. cross-entropy or max-margin loss for training the encoder  $h$  and task classifier  $c$  jointly to predict the task label  $y$  given input  $\pmb{x}$ . Our regularization loss term  $\mathcal{L}_{Reg}$  aims to regularize the model such that the causal effect of an attribute on the classifier's output matches the estimated causal effect of the attribute  $a_i$  on the label. Formally,
$$
\mathcal {L} _ {R e g} := \sum_ {i = \{1, 2, \dots , | A | \}} \mathbb {E} _ {\left(\boldsymbol {x} _ {a _ {i} ^ {\prime}}\right) \sim \mathcal {Q} \left(\boldsymbol {x} _ {a _ {i}}\right)} \left[ \left(c (h (\boldsymbol {x}) _ {a _ {i}}) - c (h \left(\boldsymbol {x} _ {a _ {i}} ^ {\prime}\right))\right) - T E _ {a _ {i}} \right] ^ {2} \tag {2}
$$
where  $\pmb{x}_{a_i'} \sim \mathcal{Q}(\pmb{x}_{c_i})$  be a sample from counterfactual distribution  $\mathcal{Q} \coloneqq \mathbb{P}(\pmb{x}_{c_i'} | \pmb{x}_{c_i})$  and  $\pmb{x}_{c_i'}$  is the input had the attribute in input  $\pmb{x}_{c_i}$  been  $c_i'$ .
3.3 Robustness of AutoACER with noise in the causal effect estimates
Our proposed regularization method relies primarily on the estimates of the causal effect of any attribute  $c_{i}$  to regularize the model. Thus, it becomes important to study the efficacy of our method under error or noise in causal effect estimation, which is expected in real-world tasks due to finite sample issues, incorrect choice of causal effect estimators, or due to properties of the unknown underlying DGP. We consider a simple setup to theoretically analyze the condition under which our method will train a better classifier than the standard ERM max-margin objective (following previous work [19, 13, 26]) in terms of generalization to the spurious correlations. Let  $\mathcal{A} = \{ca_1,\dots ,ca_K,sp_1,\dots ,sp_J\}$  be the set of available attributes where  $ca_{k}$  are causal attributes and  $sp_j$  are the spurious attributes. For simplicity, we assume that the representation encoder mapping  $X$  to  $Z$  i.e  $h: X \to Z$  is frozen and the final task classifier  $(c)$  is a linear operation over the representation. Following Kumar et al. [19], we also assume that  $z$  is a disentangled representation
w.r.t. the causal and spurious attributes, i.e., the representation vector  $\mathbf{z}$  can be divided into two subsets, features corresponding to the causal and spurious attribute respectively. Thus the task classifier takes the form  $c(\mathbf{z}) = \sum_{k=1}^{K} \mathbf{w}_{ca_k} \cdot \mathbf{z}_{ca_k} + \sum_{j=1}^{J} \mathbf{w}_{sp_j} \cdot \mathbf{z}_{sp_j}$ . Note that we make the disentanglement assumption only for creating a simple setup so that we can theoretically study the effectiveness of our method in the presence of noisy estimates of the causal effects. Our method does not require this assumption.
Let  $\mathcal{L}_{task}(\theta; (\pmb{x}, y_m))$  be the max-margin objective (see §A for details) used to train the task classifier "c" to predict the task label  $y_m$  given the frozen latent space representation  $\pmb{z}$ . Let the task label  $y$  and the attribute "a" where  $a \in \mathcal{A}$  be binary taking value from  $\{0, 1\}$ . The causal effect of an attribute  $a \in \mathcal{A}$  on the task label  $Y$  is given by  $TE_a = \mathbb{E}(Y|do(a) = 1) - \mathbb{E}(Y|do(a) = 0) = P(y = 1|do(a) = 1) - P(y = 1|do(a) = 0)$  (see §A for definition). Thus the value of the causal effect is bounded s.t.  $\hat{T}E_a \in [-1, 1]$  where the ground truth causal effect of spuriously correlated attribute "sp" is  $TE_{sp} = 0$  and for causal attribute "ca" is  $|TE_{ca}| > 0$ . Given that we assume a linear model, we instantiate a simpler form of the regularization term  $\mathcal{L}_{Reg}$  for the training objective given in Eq. 1:
$$
\mathcal {L} _ {R e g} := \sum_ {k = \{1, 2, \dots , K \}} \lambda_ {c a _ {k}} \| \boldsymbol {w} _ {c a _ {k}} \| _ {p} + \sum_ {j = \{1, 2, \dots , J \}} \lambda_ {s p _ {j}} \| \boldsymbol {w} _ {s p _ {j}} \| _ {p} \tag {3}
$$