Controlling Learned Effects to Reduce Spurious Correlations in Text Classifiers
Year: 2023

Abstract:
To address the problem of NLP classifiers learning spurious correlations between training features and target labels, a common approach is to make the model’s predictions invariant to these features. However, this can be counterproductive when the features have a non-zero causal effect on the target label and thus are important for prediction. Therefore, using methods from the causal inference literature, we propose an algorithm to regularize the learnt effect of the features on the model’s prediction to the estimated effect of feature on label. This results in an automated augmentation method that leverages the estimated effect of a feature to appropriately change the labels for new augmented inputs. On toxicity and IMDB review datasets, the proposed algorithm minimises spurious correlations and improves the minority group (i.e., samples breaking spurious correlations) accuracy, while also improving the total accuracy compared to standard training. 1

heading: 1 Introduction
text: While classifiers trained on pre-trained NLP models achieve state-of-the-art accuracy on various tasks, they have been shown to learn spurious correlations between input features and the label (Du et al., 2022). Such learned correlations impact accuracy on out-of-distribution samples and in the case of sensitive spurious features, lead to unfair predictions (Sun et al., 2019; Ribeiro et al., 2020). Learned spurious correlations can be over features that are either irrelevant (e.g., tense, gender for profession classification) or relevant (e.g., emoticons for sentiment classification, negation words for contradiction). In both cases, the classifier overweighs their importance compared to other features.
For removing spurious correlations, a common principle underlying past work is to make a model’s prediction invariant to the features that exhibit the
1Code: https://github.com/pbansal5/ feature-effect-augmentation
correlation. This can be done by data augmentation (Kaushik et al., 2019), latent space removal (Ravfogel et al., 2020), subsampling (Sagawa et al., 2019, 2020), or sample reweighing (Mahabadi et al., 2019; Orgad and Belinkov, 2022). In many cases, however, the correlated features may be important for the task and their complete removal can cause a degradation in task performance. For instance, for spurious correlation over negation tokens (e.g., “not”) or lexical overlap in MNLI natural language inference tasks, Williams et al. (2017); Joshi et al. (2022) show that correlated features are necessary for prediction and their removal can hurt accuracy.
As another example, consider the IMDB review dataset (Maas et al., 2011) where the task is classify the sentiment of a given review as positive or negative. Reviewers often include a numeric rating in their text reviews, e.g., “9/10” or “1/10”. The numeric rating is highly correlated with the sentiment label, often regarded as a spurious correlation (Pezeshkpour et al., 2021) that a model should not rely on. In the first review of Fig. 1, for instance, the positive rating can mislead a classifier since the review is overall negative. However, in the second example, the text is ambiguous and the rating “8/10” can provide a helpful signal about the reviewer’s sentiment (and removing it may decrease classifier’s accuracy). Thus, there exist inputs where the rating is a helpful feature for prediction and other inputs where it can be counter-
2271
productive. This shows the trade-off between accuracy on majority groups, (i.e., samples where these correlations hold and constitute a majority of samples) and minority groups (i.e., comparatively fewer samples where these correlations break).
In this paper, we propose a general method to resolve the above trade-off: rather than always removing the effect of a feature on the model’s prediction, we argue that the learned effect should be equal to the true effect of the feature on the output label. We define feature effect using the notion of conditional effect from the causal inference literature (Pearl, 2009): the change in the ground-truth label upon changing the feature, keeping all other input features constant. To enforce the true feature effect, we make two contributions:
1. Novel estimator of the effect of text features on the label that is accurate even at high levels of spurious correlation compared to past work.
2. Automated augmentation method that predicts the labels of new samples using the estimated feature effect and adds them to train data to achieve the desired learned effect in a classifier.
When combined with the standard accuracy loss over training data, the proposed method, Feature Effect Augmentation (FEAG), obtains the highest overall accuracy compared to baselines while reducing the learnt spurious correlation. For our evaluation, we consider the practical goal of increasing the accuracy on the minority groups while not substantially reducing the accuracy over the majority group. On comment toxicity and IMDB review datasets, we find that existing methods tend to increase minority group accuracy but reduce overall accuracy, whereas FEAG obtains a good tradeoff. In some cases, it can obtain both higher overall accuracy and higher average group accuracy. Moreover, by making it easy to change the target feature effect to be enforced, FEAG provides an interpretable control mechanism to obtain any desired tradeoff between minority and majority group accuracy (setting the feature effect to zero, e.g., prioritizes minority group accuracy).
More generally, our work provides a viable direction for automated data augmentation. While existing work requires manual labeling of counterfactual examples for removing spurious correlation (Kaushik et al., 2019; Wu et al., 2021), our method can label new examples using estimated
feature effects. We also show how estimated feature effects can be useful for other tasks, such as detecting annotator bias in a train set.

heading: 2 Related Work
text: Our work combines the debiasing NLP literature with causal effect estimation over text.

heading: 2.1 Estimating causal effect from text
text: Prior work on estimating causal effect on text is based on propensity scores, such as DragonNet (Shi et al., 2019) and follow-up work (Veitch et al., 2020; Gui and Veitch, 2022). However, propensitybased estimators are known to suffer from high variance, especially in text scenarios where overlap may be low (Gui and Veitch, 2022). We utilize a Riesz-based causal estimator (Chernozhukov et al., 2022) that has recently been shown to offer a better bias-variance tradeoff. In particular, it does not need to estimate the full propensity but rather estimates the weight for each sample directly, thus avoiding the variance issues of prior methods.

heading: 2.2 Removing spurious correlations
text: Latent Space Removal. These methods aim to remove the spurious feature from model’s learnt representation. INLP (Ravfogel et al., 2020) removes spurious features by iteratively projecting learnt representations of the classifiers onto the null-space of the target class predictor. RLACE (Ravfogel et al., 2022) models the objective instead as a constrained minimax game. However, recent work shows that spurious correlations are closely entangled with rest of the sentence representation (Kumar et al., 2022; He et al., 2022), hence latent space removal methods often unintentionally remove task critical information too, leading to a degradation in model’s performance. Weighting Methods. Debiased Focal Loss (DFL) & Product of Experts (PoE) (Mahabadi et al., 2019) are two methods which leverage a biased model (which relies heavily on spurious features for prediction) to aid training. Specifically DFL reweighs the samples such that samples belonging to the majority group are weighed less. PoE models the task as product of two models, where one model is limited in capacity and hence captures the spurious features, where as the other learns non-spurious features. More recent versions can work without annotations for the spurious features (Orgad and
Belinkov, 2022), but all methods rely on reweighing the training data. Counterfactual Augmentation. These methods require collection of counterfactual labeled data that can be used to regularize a classifier (Kaushik et al., 2019; Lu et al., 2020; Gupta et al., 2022). Obtaining labels for the augmented data is often prohibitively expensive.
Comparison to our work. All above techniques are specific ways to remove the impact of a spurious feature on the classifier. In comparison, we provide a general method that allows us to control the learned effect of a spurious feature: one can estimate the effect of a feature on the ground-truth label (which may or may not be zero) and enforce that effect on the classifier. (He et al., 2022) make a similar argument against complete removal of spurious features in the context of gender bias and rationale-based methods, while we focus on general spurious correlations and general NLP classifiers. (Joshi et al., 2022) characterise spurious correlations by necessity and sufficiency and argue for a more finegrained treatment of spurious features. In terms of implementation, our method can be seen as an extension to the counterfactual augmentation method where we automatically infer the labels for new inputs based on the modified feature’s causal effect.

heading: 3 Estimating feature effects on labels
text: Our task is to estimate the effect of text features on the label Y in training dataset. This is important for many use cases : 1) regularising a text classifier to obey the feature’s effect on the label in its prediction; 2) identifying annotator artifacts (Sap et al., 2021) for the label Y in the dataset, e.g., when the
estimated effect does not match the ground-truth known effect of a feature. For 1), we present an automated augmentation algorithm in Sec 4 based on the estimated feature effect. For 2), we use the feature effect estimation technique and present results on a comment toxicity dataset in Sec 5.4.
For feature effect estimation, we assume that the data is generated from a distribution D following the causal graph in Fig. 2 (Joshi et al., 2022; Gui and Veitch, 2022). The writer has some intent C, which generates the input sentence (Z). The sentence Z can conceptually be disentangled into 2 parts, 1) the feature of interest (T ∈ {0, 1}) and 2) rest of the text X . Annotators perceive the outcome label (Y ) from the complete text Z. The samples {(Zi, Yi)} are drawn independently from D. Note that the same dataset may contain multiple features T j (j = 1...m) whose effect needs to be estimated, leading to a different decompositions (Xj , T j).
We term the feature T as treatment, and X as covariates, following the causality literature. Since the variables X and T are sampled from the same latent variable C, they are not independent of each other. For example, in context of IMDB data, if the intent of the writer is to write a positive review then it is highly likely that X will contain positive adjectives while treatment T might be the inclusion of rating as the string 9/10. This unobserved latent variable (intent of writer) is called the confounder C. The correlations between treatment feature T and rest of text X due to the presence of confounder C can lead to the classifier model learning incorrect effect for the treatment feature. For computing feature effect, we leverage the causal inference literature (Pearl, 2009; Imbens and Rubin, 2015) and estimate Average Treatment Effect (ATE).

heading: 3.1 Background
text: Definitions. Propensities (Pearl, 2009) model the probability of a covariate being treated i.e. T = 1. They can hence be written as P(X) = P (T = 1|X). Overlap is defined as the condition when any covariate X has a non-zero probability of T = 1 and T = 0 i.e. 0 < P (T |X) < 1 for all X . Overlap is a necessary condition for causal effect estimation. Counterfactual : Given an input Z = (X,T ), a counterfactual input is defined as ZC = (X, 1 − T ), i.e. an input with treatment flipped and rest of the inputs kept constant. The original sample is called the factual input. Average Treatment Effect (ATE). It is defined
as the change in label Y on changing treatment T from 0 → 1 keeping everything else constant.
EX [Y |X, do(T = 1)]− EX [Y |X, do(T = 0)]
where do() is the do-operator (Pearl, 2009), implying an interventional change in treatment T while the covariates X are kept constant. Assume an oracle model g0 for the task, defined as g0(X,T = t) = E[Y |X, do(T = t)]. Removing the do notation, ATE estimate can succinctly be written as,
ATE = 1
n
∑
i
(g0(Xi, 1)− g0(Xi, 0)) (1)
The above equation requires access to the oracle model g0 which correctly outputs the label for counterfactual inputs ZC .
An alternate formulation for computing ATE utilises propensities (of treatment T ) i.e. P0(Xi) instead of the oracle model. The ATE using this formulation is EX [α0(Z)Y ] (α0 defined below in Eq 3). Hence the ATE estimate is
ATE = 1
n
∑
i
α0(Zi)Yi. (2)
where
α0(Zi) = ( Ti P0(Xi) − 1− Ti 1− P0(Xi) ) (3)
are the multipliers computed from propensities.
Direct Estimate. The simplest method for estimating the average treatment effect is by training a model g(.) as an approximation of the oracle g0(.) using the loss g = argming ED[L(Y, g(Z))]. The direct estimate of the ATE can then be computed by substituting g0(.) by g(.) in Eqn. 1. This gives the direct estimate (Shalit et al., 2017),
ˆATEDirect = 1
n
∑
i
(g(Xi, 1)− g(Xi, 0)) (4)
The problem with using the direct estimate is that, in cases where T is correlated with X under D, a loss optimizing method might exploit spurious correlations between X and T to learn a biased model g(.). That is, the model might over(or under)estimate the effect of T on the output Y . This leads to a biased ˆATE.
Propensity-based Doubly Robust (DR) Estimate. To resolve the issue of a biased model g, DR estimator (Kang and Schafer, 2007; Veitch et al., 2020) utilises propensities. Since the true propensities P0 are unknown we learn these propensities using the loss PPr = argmin
P ED[L(T,P(X))] giving
estimated multipliers αPr(Zi).
ˆATEDR,Pr = ˆATEDirect + 1
n
∑
i
αPr(Zi)(Yi − g(Zi)) (5)
The DR estimator corrects the bias in g using the correction term (second term in Eqn 5). If g is systematically wrong on a minority group of examples, their residual error will add up in the correction term. Also, weighing by αPr(Zi) breaks correlation between X and T , giving an unbiased correction.

heading: 3.2 Riesz Representer (RR) Estimator
text: While propensity-based methods are the most popular for estimating treatment effect, they suffer from high variance when P (T = 1|X) is close to either 1 or 0 (Swaminathan and Joachims, 2015), due to the propensity terms in the denominator of the multipliers αPr(.). This is especially a problem in high-dimensional text data, where given a treatment T (e.g., a token) the probability of it occurring with most covariate texts X may be close to 0 (e.g., if the covariate X is about a happy incident, probability of a token like "kill" occurring in the sentence is near 0). Therefore, we propose a doubly robust estimator for text data based on recent work (Chernozhukov et al., 2022) that avoids estimating the propensities as an intermediate step. Instead it models the coefficient αPr(Z) directly.
The proposed method depends on the Reisz representation theorem (Chernozhukov et al., 2018).
Theorem (Riesz Representer Theorem). For a square integrable function f(Z) (i.e. E[f2(Z)] < ∞), there exists a square integrable function αR(Z) such that
E[m((Y,Z); f)] = E[αR(Z)f(Z)]
if and only if E[m((Y, Z); f)] is a continuous linear functional of f .
Since the moment functional in ATE formulation (i.e. m((Y,Z); f) = f(X, 1)− f(X, 0)) is indeed a continuous linear functional of f , Riesz theorem for our purposes can be written as :
E[f(X, 1)− f(X, 0)] = E[αR(Z)f(Z)]
for a square integrable function f . Taking f as g0 (assuming g0 is square integrable), LHS of the equality (E[g0(X, 1) − g0(X, 0)]) is exactly the ATE and the RHS (E[αR(Z)g0(Z)]) can be interpreted as a weighted average, as in the propensity formulation of ATE (Eqn. 2). This means that αR serves as an alternative formulation for α0. Thus, rather than using the inverse of learnt propensities PPr (i.e. αPr), we can use the Riesz Representer function αR as an approximation for α0.
The challenge now remains on how we can estimate the αR function. To derive an estimation method for αR, we use its definition from the Riesz Representation theorem, i.e., αR(Z) weighed by any bounded function f(Z) gives E[f(X, 1) − f(X, 0)], as done by Chernozhukov et al. (2022).
αR = argmin α
E[(αR(Z)− α(Z))2]
= argmin α
E[αR(Z)2 − 2αR(Z)α(Z) + α(Z)2]
= argmin α
E[−2αR(Z)α(Z) + α(Z)2]
= argmin α
E[−2(α(X, 1)− α(X, 0)) + α(Z)2]
The first step is a trivial equality, which says that αR is the solution for the equation argmin
α E[(αR(Z) − α(Z))2]. In the third step, αR(Z) 2 can be ignored as the minimization is over α and then we use the Riesz Representation theorem to expand the term E[αR(Z)α(Z)] as E[α(X, 1) − α(X, 0)], thus getting rid of αR and providing an optimization objective.
The new learnt riesz function αR can then be used for computing our Doubly Robust estimate. We can simply substitute αPr in the DR estimate Eqn 5 by αR, giving us RR-based ˆATE,
ˆATEDR,R = ˆATEDirect + 1
n
∑
i
αR(Zi)(Yi − g(Zi)) (6)

heading: 4 Controlling learnt effects in a classifier
text: Armed with an estimator of feature effect on the label, we now describe methods to enforce the feature effect on a predictive model’s output. Given data {(Z, Y )} where Z are input sentences and Y is output label, the goal is to learn a predictive model f for Y such that the causal effect of a feature on f(Z) is the same as the true feature effect, τ j for the jth feature. That is, τ j should be equal to ED[f(Xj , T j = 1) − f(Xj , T j = 0)] where
Xj refers to all input features except T j and the expectation is over the training distribution. As discussed in Section 3, the ideal predictive function is g0 since it will ensure the correct feature effect,τ j = ED[g0(Xj , T j = 1) − g0(Xj , T j = 0)], and will also provide high accuracy since it is the true data generating function.

heading: 4.1 Counterfactual-based Regularisation
text: To approximate the oracle function g0(Z), for a given loss L, Standard ERM loss minimisation optimizes, argminf ED[L(Y, f(Z))]. But machine learning data is often underspecified (D’Amour et al., 2020; Lee et al., 2022), leading to the ERM returning multiple solutions f with similar accuracy on validation set. These different solution f weigh different features in input text differently. As a result, the obtained solution can be far from g0.
Therefore, we use the provided feature effect to constraint the solution space. A first idea is to add a regularization term that aligns the model’s learnt feature effect with the provided effect. Suppose that we are given a list of m binary features {T j}1...m which are suspected to have a spurious correlation (e.g., such features can be discovered using explanation methods on an ERM model (Wang et al., 2021)). We can conceptually decompose an input sentence Z into m different pairs {(Xj , T j)}1...m, where Xj is the part of the sentence Z apart from T j . Then using the given feature effect {τ j}1...m for each feature, we can write the regularized loss,
L+ λ 1 m
∑
j
(f(Xj , 1)− f(Xj , 0)− τ j)2 (7)
where λ is the regularisation constant. While we proposed regularizing to τ j , sometimes one may want to completely remove a feature’s effect based on domain knowledge. For example, a biased dataset may exhibit a non-zero feature’s effect on the label, but due to fairness reasons, one would like to completely remove its effect. In that case, we can simply set τ j = 0 and apply Equation 7. When τ j is set to zero, FEAG can be seen as optimizing the same objective as methods that aim to fully remove the feature’s effect (Ravfogel et al., 2020; Mahabadi et al., 2019).

heading: 4.2 Augmentations for Estimated Effect
text: We also consider a data augmentation alternative to regularization. Given distribution (Z, Y ) ∼ D, m binary features {T j}1...m, and their feature effects {τ j}1...m, we can augment along any of the
m features to generate a counterfactual distribution. When we augment along the j feature, the new input becomes Zj,C = (Xj , 1− T j). Using the feature’s effect τ j , we can estimate the corresponding label Y j,C for the input Zj,C . Intuitively, a higher feature effect makes it more likely that the label will change (see Supp H for details). We get a new counterfactual distribution, (Zj,C , Y j,C) ∼ Dj,C .
Similarly other counterfactual distributions can be found, giving us {Dj,C}1...m. A union can be taken over these distributions to give us the counterfactual distribution over these m features as DC = ∪mj=1Dj,C This new generated distribution can then be included in training as counterfactual augmentations while minimising the loss,
argmin f
ED[L(Y, f(Z))] + λEDC [L(Y, f(Z))] (8)
where we now draw samples from the combined distribution D + DC . λ signifies the weighting of samples drawn from augmented counterfactual distribution DC in the loss function.
While both regularisation and data augmentation can help us control the learned effect of features, owing to the scalability and ease of optimization, we use the augmentation version of our algorithm to present our results.

heading: 4.3 FEAG: Two-phase algorithm
text: To summarize, the proposed algorithm, Feature Effect Augmentation (FEAG), proceeds in two phases. It takes as input a set of features T j : j = 1...m, that may be suspected to be spurious, which can be derived using an automated saliency method (e.g., top-k important tokens) (Pezeshkpour et al., 2022; Wang et al., 2021) or based on domain knowledge. Feature effect estimation. For each of the features T j , we estimate the feature effect using the Reisz estimator from Section 3.2. We follow the 2- headed model architecture with shared parameters
(Shi et al., 2019) to learn the Riesz representer αR and the model g for Y (details are in Supp J, Fig 4). Note that αR and g should share sentence representation extraction module to ease learning (Chernozhukov et al., 2022) (i.e., they have the same BERT model, but different final layer linear heads). These learnt models can be used in Eqn 6 to get feature effect estimates ({τ j}1...m) on held-out data. Counterfactual Augmentation. Our modular pipeline allows practitioners to change the feature estimate τ j according to their needs before using them for counterfactual augmentations. Using the features and their effect estimates, we create counterfactually augmented data DC as described in Sec 4.2 and include them while training (Eqn 8) to learn the final classifier.

heading: 5 Experiments
text: We have three goals for evaluation: 1) RR-based estimators of feature effect are more accurate than propensity-based estimators; 2) FEAG using RRbased estimators provides better overall accuracy while minimizing spurious correlation compared to existing baselines for removing spurious correlations; 3) Our feature effect estimator is a general method and can be used to detect annotator bias.

heading: 5.1 Datasets
text: Since the true feature effect is unknown for realworld data, we construct a semi-synthetic dataset based on the CiviComments dataset (Borkan et al.,
2019). In addition, we evaluate on subsampled versions of the CivilComments and IMDB dataset. CivilComments Semi-Synthetic (SS). CivilComments is a toxicity detection dataset {(X,Y )}, where X are input sentences and Y is the toxicity label (1 means toxic). To evaluate our methods, we need to construct a dataset generated from the causal graph in Fig. 2. Since the writer’s intent (confounder) is unknown, we construct it as a property of the input text, W = h(X) ∈ {0, 1}, leading to the modified causal graph in Fig. 3 (Supp G). To obtain h(X), we train a binary classifier using a DistilBERT model on (X,Y ) pairs. Finally we sample a new label as Y ′ ∼ Bernoulli((1− τ)Y + τT ), giving the true feature effect as τ . The complete text Z = (X,T ) is constructed by prepending each covariate sentence X with the word Treated if T = 1 and Untreated if T = 0. CivilComments Subsampled. Rather than introducing a new treatment, here we subsample CivilComments to introduce a spurious correlation between an existing token kill and label Y . Here all sentences with token kill are considered as treated, while others untreated. To exacerbate the spurious correlation between T and Y , we subsample our data based on the learnt property W (from above), following the causal graph in Fig 3a. IMDB. From the IMDB reviews dataset (Maas et al., 2011), we consider reviews that contain a numerical rating—text string from either the set {7/,8/,9/} or {2/,3/,4/}. To construct a binary treatment variable, occurrences of these strings are replaced by Treated if the rating is 7, 8, or 9 and an empty string otherwise. The Treated token is predictive of the sentiment with 90% accuracy.
For dataset and training details, see Supp B, Supp A respectively. All results are run for 3 seeds.

heading: 5.2 Evaluating Feature Effect Estimation
text: We evaluate the performance of different estimators in Sec 3 on the CivilComments SS dataset (with different overlap ϵ and feature effects τ ). We
compare the Riesz-based DR estimator (Eqn 6) with the Direct (Eqn 4) and Propensity-based DR (Eqn 5) baselines. All estimators are finetuned using either BERT or DistilBERT as base model. See Supp ?? Quantitative Results. Table 1 shows the mean error in estimating feature effect across τ ∈ {0.10, 0.30, 0.50} and ϵ∈ {0.01, 0.05, 0.10}. For hyperparameter selection, see Supp. D. Across all settings (barring 1% overlap with high τ ), Riesz is able to estimate the effect with low error. Direct fails to do well in high τ and low ϵ ranges, failing for both τ=0.50 and ϵ=0.01. Due to its high variance, Propensity is unable to work well, often producing an estimate worse than Direct.
For the two real-world datasets, true feature effect is unknown. But comparing the effect estimates of Direct and Riesz, Direct tends to overestimate the feature effect (due to spurious correlation), which is corrected to a lower value by Riesz. Qualitative Results. To understand how the Reisz estimator works, we show qualitative results for Civil Comments Subsampled dataset in Table 3. To counter the spurious correlation of token kill (T) with other parts of text (X) that cause toxicity (Y), the Riesz estimator provides a low weight to sentences having features X that commonly occur with T, and higher weight to sentences having X that rarely occur with T. Treated samples (T=1) have a positive Riesz value and vice versa. We can see that sentences with violent language (in addition to kill) are assigned a low score while other sentences with kill are assigned a high score, thus serving to extract the isolated feature effect of kill (without confounding due to other tokens).
5.3 Accuracy of FEAG classifiers
We now compare FEAG classifiers based on Riesz, FEAG(ate), and based on zero effect, FEAG(0), with prior debiasing algorithms. Groups. Classifiers that reduce spurious correlation are expected to decrease total accuracy but
increase the accuracy of minority inputs that do not exhibit those correlations. To study such effects on accuracy, we divide our evaluation data into four groups: Group1 (Y =0, T =0), Group2 (Y = 0, T = 1), Group3 (Y = 1, T = 0), Group4 (Y =1, T =1). In addition, we report the average group accuracy across the four groups as a measure of debiasing/reduced spurious correlation. An ideal model should achieve both high overall accuracy and high average group accuracy, demonstrating its reduced reliance on spurious features.
Baselines. We consider popular baselines from prior work (Joshi et al., 2022; He et al., 2022; Orgad and Belinkov, 2022): weighting methods like DFL, DFL-nodemog, Product of Experts (Mahabadi et al., 2019; Orgad and Belinkov, 2022) and latent space removal methods like INLP (Ravfogel et al., 2020). We also include worst-group accuracy methods like GroupDRO, Subsampling (Sagawa et al., 2019, 2020) from the machine learning literature, and a baseline RemoveToken that removes the treatment feature from input (see Supp C).
Results. For the semi-synthetic dataset (CivilComments SS) in Table 4, FEAG(ate) increases the average group accuracy while retaining similar overall accuracy as Direct. FEAG(ate) also has bet-
ter minority group accuracy (i.e. Group2,Group3) than Direct. In comparison, FEAG(0) leads to a decrease in overall accuracy and also average group accuracy compared to FEAG(ate). Other baselines like Subsample, GroupDRO or DFL achieve a higher average group accuracy as they improve accuracy on the minority groups, but they suffer a substantial reduction in overall accuracy, from 87 to 66-73, which hinders usability of the model. Methods like DFL-nodemog or POE have no impact or obtain worse results compared to Direct. These results show the fundamental tradeoff between total and average group accuracy and how FEAG(ate) provides a good tradeoff between the two.
For the subsampled dataset (CivilComments Subsampled) in Table 5, we see a similar trend, where FEAG(ate) gives the best tradeoff between overall and average accuracy. FEAG(0) is substantially worse than FEAG(ate), showing the importance of not fully removing the effect of a spurious token. Except POE, Subsample and GroupDRO, all other methods obtain both lower total and average group accuracies compared to FEAG(ate). As before, POE is near identical to Direct while the weighting methods Subsample and GroupDRO lead to significant decreases in total accuracy.
Finally, we show results for IMDB where the causal graph is unknown and our assumptions from Fig. 3a may not be valid. Nonetheless Table 6 shows that both FEAG(ate) and FEAG(0) achieve better average group accuracy with slightly better total accuracy than the Direct model. Other baselines follow their usual trend: ML weighting baselines (Subsample, GroupDRO) suffer reductions in total accuracy, DFL and POE methods are unable to improve average group accuracy substantially, and INLP is worse for both total and average group accuracy. Besides BERT, results using DistilBERT as a base model show a similar trend (Supp F). We also report FEAG(propen) numbers in Supp E.

heading: 5.4 Detecting Annotator bias
text: While we focused on the debiasing task for classifiers, our feature effect estimator is general: we apply it to detect annotator bias in the CivilComments dataset. If the true feature effect of a token is known, we can compare it to the estimated effect to detect any annotator bias in the dataset. For tokens like “racist” and “guys” where the true effect is likely to be high and zero respectively, the estimated effect confirms the prior (see Table 7). But for tokens like “gay” or “black”, our method shows a significant non-zero feature effect on the label which may indicate annotator bias, as it may be known that these tokens should have a zero effect on the toxicity label. Compared to the naive conditional probability (Y |T ), our effect estimator can be used to provide a better sense of how impor-
tant certain keywords are for generating the output label. (e.g., “guys” obtains a zero causal effect but P (Y |T ) shows a substantial deviation from 0.5).

heading: 6 Conclusion
text: Rather than fully removing a feature’s effect on the classifier, we presented a method for fine-grained control of the feature’s effect based on causal inference. We showed how our method allows a better tradeoff between overall accuracy and accuracy over subgroups in the data. Our preliminary study on annotator bias demonstrated that our method may be useful for detecting biases in the classification label too. As future work, a natural direction is to combine these two threads and explore how we can develop methods to regularize features’ effect on the debiased label, rather than the (possibly confounded) labels provided in the dataset.
Limitations One major shortcoming of FEAG method is the dependency on creation of counterfactual inputs. If there is an error in counterfactual generation, we might get a wrong feature effect estimate. Thus, for simplicity, our evaluation considered tokens as features. The parallel development of counterfactual input generation methods (Wu et al., 2021; Howard et al., 2022) would hopefully ease this issue and allow FEAG to be used reliably for spurious correlations on more complex features too. Ethics Statement This project aims to check when methods are using spurious correlation. Identification of these spurious correlation is important for debiasing i.e. removal of dependence of the model on these correlations. Our work shows how instead of complete removal of these spurious features, regularising them might be better. At the same time, this is early research work and shouldn’t be used in real-world systems without further evaluation.

heading: A Training Details
text: Architecture All classification methods were trained using a single linear layer on top of
BERT(/DistilBERT) [CLS] token. Riesz uses a common BERT model for sentence reprensentation and then uses 2 seperate linear layers for learning αR and g seperately.
Seeds We use three seeds for our experiments. 0,11,44. All numbers are reported with mean and std errors over these three seeds.
Optimization We use 1e-5 learning rate for BERT parameters and 1e-4 for the final linear layer parameters. We train with 32 batch size for all our experiments. The learning rate linearly decays over training iterations. We use Adam optimizer with 1e-2 weight decay for all methods.
Best Model Selection All models are trained to completion (i.e. number of epochs specified for particular dataset). The evaluation is done after every epoch and the best model is chosen over all the epochs using the validation set.
Loss Binary cross entropy loss is used for all methods.
Tokenization We use the standard uncased tokenizers with max length of 256 tokens.

heading: B Dataset Specific Details
text: For all datasets we set the number of epochs such that for all methods the validation loss has bottomed and starts increasing.
CivilComments Semi-Synthetic Since CivilComments is heavily skewed towards the 0 label, we resample the dataset to create a balanced data which we use in all our experiments. Since the writer’s intent (confounder) is unknown, we construct it as a property of the input text, W = h(X) ∈ {0, 1}, leading to the modified causal graph in Fig. 3. This property could be something simple like presence of a certain word like police in text or something more complex like inferred ethnicity of the writer. Rather than choosing a property manually, we train distilbert for modeling h(.) for a few hundred iterations. We hence use W = h(X) as the property. h(.) achieves ∼ 78% accuracy on the task. To ensure overlap, the treatment variable is sampled from W such that 0 < P (T |X) < 1 or equivalently 0 < P (T |W ) < 1. We do this by using T equal to W with ϵ > 0 fraction of samples flipped. Finally we sample a new label as Y ′ ∼ Bernoulli((1−τ)Y+τT ), giving the true feature effect as τ . The complete text Z = (X,T ) is
constructed by prepending each covariate sentence X with the word Treated if T = 1 and Untreated if T = 0. This is true for all the experiments and datasets in our setup. This also eases counterfactual generation by just changing the prepended text from Treated to Untreated (and vice-versa). The dataset has 7K train samples and 2K test samples. We train the model for 10 epochs. For controlling learnt effect, we use 0.50 ATE and 5% overlap SS.
CivilComments Subsampled Since kill doesn’t occur often in dataset (3%) we retain only 10% of the untreated sentences.
We subsample so as to retain only 5% of the samples having T = 1& W = 0. Samples having T = 1,W = 1 are untouched. Samples having T = 0 are subsampled by 10% (as mentioned above). Our dataset has 5K train samples and 2K test samples. We train the model for 10 epochs.
IMDB The dataset is subsampled to have equal number of positive and negative sentiment reviews. The Treated token is predictive of the sentiment with 90% accuracy. The test set is constructed similarly. The dataset has 1354 train samples and 1328 test samples. We train the model for 30 epochs.

heading: C Method Specific Details
text: FEAG We use λ = 0.1 for our feature effect augmentation, i.e. loss on augmented samples is weighed 1e-1 times the loss on original samples.
Subsample,GroupDRO These method considers an alternate objective of maximising worst group accuracy as a condition for learning models robust to spurious correlations. For Subsample we break the correlation between T and Y but maintain P (T = 1) and P (Y = 1) invariant (following (Joshi et al., 2022)). i.e. for an input sample P (T = 1, Y = 1) = P (T = 1)P (Y = 1). For GroupDRO we sample from all the four groups (as defined in Sec 5.3) equally, i.e. P (T = 1, Y = 1) = 0.25. Additionally we have corresponding groups weights (following the original paper) with step size of 0.01. We use heavy regularisation of 1e-2 with Adam optimizer (regularisation of 1e-1 led to degradation in numbers).
DFL,POE,DFL-nodemog For training the biased/weak learner model we use TinyBERT model 2. The optimization parameters for TinyBERT model were same as that of the main model
2https://huggingface.co/prajjwal1/bert-tiny
(described above). We observed that while DFL and POE’s weak learner was able to capture the bias, DFL-nodemog struggled to learn main model’s success and collapsed to constant value. For POE we use λ = 1.0, i.e. the loss minimised is CE(fm(X), Y ) + CE(Softmax(Log(fb(X)) + Log(fm(X))), Y )
INLP We train INLP in post-hoc fashion i.e we first train a Direct model, select the best model and then apply INLP on its representation. We take the code from the official repository 3 and run it for 100 iterations with minimum accuracy stopping criterion of 0.50. We tried RLACE algorithm too, but it yeilded similar/worse results than INLP

heading: D Best Propensity and Riesz Eval
text: Propensity Eval We choose λ = 1.0 as the best value from the table below.
Riesz Eval We choose λ = 0.01 as the best value from the table below.

heading: E BERT Propensity-DR based FEAG numbers
text: Propensity-DR based FEAG numbers on the three datasets are given in Table 10, Table 11 and Table 12.

heading: F DistilBERT FEAG numbers
text: We also show FEAG numbers on the three datasets using DistilBERT as the model in Table 13, Table 15 and Table 14
3https://github.com/shauli-ravfogel/nullspace_ projection

heading: G Alternative Causal Graphs
text: We present alternate version of the primary causal graph (Fig 2) in Fig 3

heading: H Label Flipping Algorithm
text: Consider treatment T , label Y . The desired effect as τ . WLOG we can assume τ > 0 (if τ < 0, then make T ′ = 1− T and proceed with T ′). The new counterfacutal labels are Y C and new treatment is TC = 1 − T (we will only use T and TC will implicitly be 1− T )
Consider probabilities as :
P (Y = 1|T = 1) = p1 P (Y = 0|T = 1) = 1− p1 P (Y = 0|T = 0) = p2 P (Y = 1|T = 0) = 1− p2
(9)
Going from untreated to treated Since τ > 0, changing treatment from 0 to 1, should increase the probability of outcome label being 1 (and decrease probability of it being 0) i.e. P (Y C = 1|T = 0) > (Y = 1|T = 0)&P (Y C = 0|T = 0) < (Y = 0|T = 0). This can be achieved by keeping Y C = Y whenever Y = 1 and randomly flipping certain fraction (say η) of samples having Y = 0 to Y C = 1 ( the other 1−η would have Y C = Y = 0) With the goal of P (Y C = 1|T = 0) − P (Y = 1|T = 0) = τ , η can be easily computed as τp2 . To verify we can compute
P (Y C = 1|T = 0) = P (Y = 1|T = 0)+ ηP (Y = 0|T = 0) P (Y C = 1|T = 0) = P (Y = 1|T = 0) + ( τ p2 )p2 P (Y C = 1|T = 0)− P (Y = 1|T = 0) = τ (10)
Going from treated to untreated Similarly we can argue that Y C = Y whenever Y = 0 and
randomly flipping τp2 fraction of samples having Y = 1 to Y C = 0.

heading: I Computational Budget
text: GPUs used We run our experiments on NVIDIA RTX A6000 gpus. On an average each experiment takes 1 hour to complete.
We use the BERT-base (110 Million parameters) and DistilBERT model (55 Million parameters) for computation.

heading: J Two-Head Riesz Model
text: Sharing parameters between classifier and Riesz estimator using a two-headed model forces the shared model (e.g. BERT) to learn representations which are important for both classifier and Riesz model. While this may cause a decrease in either model’s performance, this leads to a better estimate due to reduced noise in estimation (Shi et al., 2019). We present our architecture in Fig 4
(a) Alternate Causal Graph. The Red arrow is different from Fig. 2. The confounding now instead of being the intent of writer, is a property of text P
(b) Causal graph from Semi-Synthetic setting. The graph is derived from Fig. 3a, with addition of new node Y ′. The function h(.) is used to get property W from X . Noise ϵ (in form of label flipping) is added to W to ensure non-zero P (T = t|X), i.e. every co-variate X has non-zero probability of being treated and being untreated. τ is T ’s effect on the new outcome label Y ′, while 1− τ is Y ’s affect on Y ′
Figure 3: Alternate Causal Graphs
Figure 4: Two-head model for jointly training αR and g for Riesz estimator. The top figure has a treated sentence as input while the bottom figure has an untreated sentence as input.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Sec 6
3 A2. Did you discuss any potential risks of your work? Sec 7
3 A3. Do the abstract and introduction summarize the paper’s main claims? Sec 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Supplementary A,B,C
3 B1. Did you cite the creators of artifacts you used? Section 2,5
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? They are all open source
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 5
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Data used doesn’t contain any identifying information
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Supplementary B,C
C 3 Did you run computational experiments? Sec. 5
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Supplementary I
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Supplementary C,D,E
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Supplementary A
7 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? We haven’t used any packages
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: As future work, a natural direction is to combine these two threads and explore how we can develop methods to regularize features’ effect on the debiased label, rather than the (possibly confounded) labels provided in the dataset. Limitations One major shortcoming of FEAG method is the dependency on creation of counterfactual inputs. If there is an error in counterfactual generation, we might get a wrong feature effect estimate. Thus, for simplicity, our evaluation considered tokens as features. The parallel development of counterfactual input generation methods (Wu et al. , 2021; Howard et al. , 2022) would hopefully ease this issue and allow FEAG to be used reliably for spurious correlations on more complex features too