Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models (2022)

Abstract: Recently, NLP models have achieved remarkable progress across a variety of tasks; however, they have also been criticized for being not robust. Many robustness problems can be attributed to models exploiting “spurious correlations”, or “shortcuts” between the training data and the task labels. Most existing work identifies a limited set of task-specific shortcuts via human priors or error analyses, which requires extensive expertise and efforts. In this paper, we aim to automatically identify such spurious correlations in NLP models at scale. We first leverage existing interpretability methods to extract tokens that significantly affect model’s decision process from the input text. We then distinguish “genuine” tokens and “spurious” tokens by analyzing model predictions across multiple corpora and further verify them through knowledge-aware perturbations. We show that our proposed method can effectively and efficiently identify a scalable set of “shortcuts”, and mitigating these leads to more robust models in multiple applications.

## introduction despite great progress has been made over improved accuracy, deep learning models are known to be brittle to out-of-domain data @xcite , adversarial attacks @xcite @xcite , partly due to sometimes the models have exploited spurious correlations in the existing training data @xcite . in figure 1 , we show an example of a sentiment classification model making spurious correlations over the phrases "spielberg" and "new york subway" due to their high co-occurrences with positive and negative labels respectively in the training data. most existing work quantifies spurious correlations in nlp models via a set of pre-defined patterns based on human priors and error analyses over the models, e.g., syntactic heuristics for natural language inference @xcite , synonym substitutions @xcite , or adding adversarial sentences for qa @xcite . more recent work on testing models' behaviour using @xcite ) also used a pre-defined series of test types, e.g., adding negation, temporal change, and switching locations/person names. however, for safe deployment of nlp models in the real world, in addition to predefining a small or limited set of patterns which the model could be vulnerable to, it is also important to proactively discover and identify models' unrobust regions automatically and comprehensively. in this work, we introduce a framework to automatically identify spurious correlations exploited by the model, sometimes also denoted as "shortcuts" in prior work @xcite foot_0 , at a large scale. our proposed framework differs from existing literature with a focus more on automatic shortcut identification, instead of pre-defining a limited set of shortcuts or learning from human annotations ( the intuition is that if we have data from multiple domains for the same task, then "genuine" tokens are more likely to remain useful to labels across domains, while "spurious" tokens would be less useful. our last step further applies a knowledgeaware perturbation to check how stable the model's prediction is by perturbing the extracted tokens to their semantically similar neighbors. the intuition is that a model's prediction is more likely to change when a "spurious" token is replaced by its semantically similar variations. to mitigate these identified "shortcuts", we propose a simple yet effective targeted mitigation approach to prevent the model from using those "shortcuts" and show that the resulting model can be more robust. our code and data have been made publicly. 2 our contributions are as follows: interpretability there has been a lot of work on better interpreting models' decision process, e.g., understanding bert @xcite and attention in transformers @xcite , or through text generation models @xcite . in this paper we utilize the attention scores as a generic way to understand what features a model relies on for making its predictions. other common model interpretation techniques @xcite , or more recent work on hierarchical attentions @xcite and contrastive explanations @xcite , can be used as well. in @xcite , the authors found that attention scores can be manipulated to deceive human decision makers. the reliability of existing interpretation methods is a research topic by itself, and extra care needs to be taken when using attention for auditing models on fairness and accountability @xcite . ## framework for identifying shortcuts in this section, we introduce our framework to identify spurious correlations in nlp models. our overall framework consists of first identifying tokens important for models' decision process, followed by a cross-dataset analysis and a knowledge-aware perturbation step to identify spurious correlations. ## conclusion in this paper, we aim to improve nlp models' robustness via identifying spurious correlations automatically at scale, and encouraging the model to rely less on those identified shortcuts. we perform experiments and human studies over several benchmark datasets and nlp tasks to show a scalable set of shortcuts can be efficiently identified through our framework. note that we use existing interpretability approaches as a proxy to better understand how a model reaches its prediction, but as pointed out by prior work, the interpretability methods might not be accurate enough to reflect how a model works (or sometimes they could even deceive human decision makers). we acknowledge this as a limitation, and urge future research to dig deeper and develop better automated methods with less human intervention or expert knowledge in improving models' robustness.