Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis (2024)

Abstract: Recent research has revealed that machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances. For instance, a sentiment classifier may erroneously learn that the token “performances” is commonly associated with positive movie reviews.Relying on these spurious correlations degrades the classifier’s performance when it deploys on out-of-distribution data.In this paper, we examine the implications of spurious correlations through a novel perspective called neighborhood analysis. The analysis uncovers how spurious correlations lead unrelated words to erroneously cluster together in the embedding space. Driven by the analysis, we design a metric to detect spurious tokens and also propose a family of regularization methods, NFL (doN’t Forget your Language) to mitigate spurious correlations in text classification.Experiments show that NFL can effectively prevent erroneous clusters and significantly improve the robustness of classifiers without auxiliary data. The code is publicly available at https://github.com/oscarchew/doNt-Forget-your-Language.

## introduction disclaimer: this paper contains examples that may be considered profane or offensive. these examples by no means reflect the authors' view toward any groups or entities. pre-trained language models (plms) such as bert @xcite and its derivative models have shown impressive performance across natural language understanding tasks @xcite @xcite . however, previous studies @xcite @xcite manifest the vulnerability of models to spurious correlations which neither causally affect a task label nor hold in future unseen data. for example, in ## analyzing spurious correlations with neighborhood analysis as mentioned in section 2.1, the literature does not reveal how spurious tokens acquire excessive importance weight. therefore we present a novel perspective by which to understand spurious correlations using neighborhood analysis and also demystify the representations learned by models in the presence of spurious tokens. ## don't forget your language as we have determined using neighborhood analysis that the heart of the problem is the misalignment of spurious tokens and genuine tokens in the lm, we propose nfl, a family of regularization techniques by which to restrict changes in either the parameters or outputs of an lm. our core idea is to use off-the-shelf plms which are not exposed to spurious correlations to protect the model from spurious correlations. below we list nfl variations: the main takeaway is that any sensible restriction on the lm to preserve each token's semantics is helpful in learning a robust model. figure 2 summarizes nfl techniques and compares them with ordinary fine-tuning side-by-side. the weights of the regularization terms in nfl-co and nfl-cp are discussed in appendix d. ## naturally occurring spurious correlations to further demonstrate the practical benefits of the proposed methods, we apply our neighborhood analysis on naturally occurring spurious correlations. spurious correlations naturally occur in datasets for reasons such as annotation artifacts, flaws in data collection, and distribution shifts @xcite @xcite . previous works @xcite indicate that in the sst2 dataset, the token spielberg has a high cooccurrence with positive but the token itself does not cause the label to be positive. therefore it is likely spurious. @xcite reveal that models tend to capture spurious correlations in toxicity detection datasets by relating the names of frequently targeted identity groups such as gay and black with toxic content. ## conclusion we conduct a neighborhood analysis to explain how models interact with spurious correlation. through this analysis, we learn that corrupted language models capture spurious correlations in text classification tasks by mis-aligning the representation of spurious tokens and genuine tokens. the analysis not only yields a deeper understanding of the spurious