Fighting Spurious Correlations in Text Classification via a Causal Learning Perspective (2025)

Abstract: In text classification tasks, models often rely on spurious correlations for predictions, incorrectly associating irrelevant features with the target labels. This issue limits the robustness and generalization of models, especially when faced with out-of-distribution data where such spurious correlations no longer hold. To address this challenge, we propose the Causally Calibrated Robust Classifier (CCR), which aims to reduce models’ reliance on spurious correlations and improve model robustness. Our approach integrates a causal feature selection method based on counterfactual reasoning, along with an unbiased inverse propensity weighting (IPW) loss function. By focusing on selecting causal features, we ensure that the model relies less on spurious features during prediction. We theoretically justify our approach and empirically show that CCR achieves state-of-the-art performance among methods without group labels, and in some cases, it can compete with the models that utilize group labels. Our code can be found at: https://github.com/yuqing-zhou/Causal-Learning-For-Robust-Classifier.

## introduction despite their success on standard benchmarks, neural networks often struggle to generalize to outof-distribution (ood) data. a primary cause of this problem is their tendency to rely on features not causally related to tasks but holding spurious correlations to labels, which can reduce model robustness when the data distribution shifts @xcite @xcite @xcite . for example, in natural language inference (nli) tasks, if contradictory sentences in a dataset frequently contain negation words, a model trained on this dataset might predict contradiction simply based on the presence of negation words rather than relying on the true underlying features. when encountering data where such spurious correlations do not hold, the model is likely to make incorrect predictions. prior works divide the data into different groups based on combinations of class labels and spurious features. the distribution of these groups is often unbalanced, and the correlation between labels and spurious features in the majority groups leads to spurious correlations. as a result, a model trained on such a dataset inevitably performs worse on the minority groups compared to the majority groups. existing methods have focused on improving the worst-group accuracy. some methods tackle this issue by utilizing group labels, which provide information about both the true labels and the spurious features @xcite @xcite . however, obtaining annotations for group labels is costly and not always feasible, limiting the utility and applicability of these methods. other approaches employ a two-stage process, where group labels are first inferred, followed by an adjustment of the loss function based on this inferred information. examples include reweighting samples @xcite or incorporating supervised contrastive loss @xcite . however, these methods attempt to calibrate the model learning process by only adjusting the loss function and hope the model's over-reliance on spurious features can be alleviated. the effects are indirect and limited. a more direct and effective approach would involve modifying the model architecture itself to explicitly identify causal features and filter out spurious ones. existing works show that models trained using standard empirical risk minimization (erm) can learn high-quality representations of both spurious and causal features @xcite . therefore, our goal is to encourage models to rely more on causal features than spurious ones. we propose a method, causally cal-ibrated robust classifier (ccr), to identify causal feature representations through counterfactual reasoning. specifically, we first disentangle the representation vector from the model's last layer and then construct counterfactual representation vectors. we adapt causal inference theory @xcite to calculate the probabilities of necessity and sufficiency for each feature within the representation vector and the entire representation. by retraining the last layer to maximize the probability of necessity and sufficiency of the overall feature representation, the model is guided to assign higher weights to causal features, ultimately relying more on these features. additionally, we derive a theoretically unbiased loss function based on inverse propensity weighting (ipw), further enhancing the model's robustness to spurious correlations. our contributions are summarized as follows: ## method in this paper, we focus on the impact of spurious correlations on transformer-based text classifiers and aim to enhance their robustness. while implemented on bert @xcite , our method can be generalized to other models as well. our method, ccr, consists of two stages. in stage 1, we apply standard erm with a covariance matrix regularization to disentangle the representations in the model's last layer, ensuring each feature representation is independent. in stage 2, we retrain the last layer to encourage the model to rely on causal features by introducing a sample-wise causal constraint based on counterfactual representation vectors. additionally, we reweight each sample's loss in stage 2 to achieve an unbiased loss, ensuring accurate causal feature selection. ## experiments in this section, we investigate the effect of ccr through three research questions. rq1: what is the performance of ccr compared with other stateof-the-art methods on the text classification benchmarks? rq2: how is the contribution of each component of ccr to the model's robustness? rq3: what is the effect of causal feature selection under different hyperparameters? ## related work spurious correlations challenge the robustness of machine learning models trained via erm, especially when there are shifts between the training and test data distribution @xcite @xcite @xcite . @xcite addresses this problem by training a classifier via a regularized group distributionally robust optimization (group-dro) method, which minimizes the loss of the worstcase group. this approach is to train the entire model. in contrast, some methods propose that simply retraining the last layer of an erm-trained model can effectively improve robustness. for example, dfr @xcite retrain the last layer of the model trained via erm on a small dataset where the spurious correlations break. however, both @xcite and @xcite require prior knowledge of spurious correlations to define groups in the training data, making it costly and limiting its ability to generalize to unseen spurious attributes. some methods upweight samples during retraining where the initially trained model performs poorly @xcite . another approach uses contrastive learning to improve robustness by aligning same-class representations and separating misclassified negatives, allowing the model to learn similar representations of same-class samples and ignore spurious attributes @xcite . these methods do not require prior knowledge of group labels. all these approaches focus on reducing spurious correlations by modifying the loss function, while our method tackles the underlying issue by directly identifying causal features using a counterfactual feature generation module. our method for identifying causal features is based on the theorem of probability of necessity and sufficiency from causality theory @xcite . a related work also applies this theory @xcite , but their goal is to identify minimal tokens in input text that lead to correct predictions. in contrast, our approach operates on feature representations within the model to identify causal features, aiming to build a robust classifier that performs well across varying data distributions. ## conclusion in this paper, we address the problem of models relying on spurious correlations in their predictions, particularly in text classification tasks. to mitigate this issue, we propose the causally calibrated robust classifier (ccr). specifically, we introduce a causal feature selection model based on counterfactual reasoning, combined with an unbiased inverse propensity weighting (ipw) loss function. we provide theoretical justification for our method and empirically demonstrate that it achieves stateof-the-art performance among approaches that do not require group labels, and in some cases, it can even compete with models that utilize group labels.