H ua SLIM : Human Attention Motivated Shortcut Learning Identification and Mitigation for Large Language models (2023)

Abstract: Large language models have made remarkable progress on a variety of NLP tasks. However, it has been found that they tend to rely on shortcut features that spuriously correlate with labels for prediction, which weakens their generalization on out-of-distribution samples. In this paper, we propose a human attention guided approach to identifying and mitigating shortcut learning, which encourages the LLM-based target model to learn relevant features. We define an attention-based measurement to capture both model and data bias and identify shortcut tokens by exploring both human and neural attention. In a self-distillation framework, we mitigate shortcut learning by dynamically adjusting the distillation temperature according to the detected shortcut tokens and estimated shortcut degree. Additionally, we utilize human attention as a supervisory signal to constrain large language models to pay more attention to relevant tokens. Experimental results on multiple NLP tasks show that our proposed method can effectively identify shortcut tokens, and significantly improve the robustness of large language models on OOD samples, while not undermining the performance on IID data.

## introduction large language models (llms), e.g., bert @xcite , roberta @xcite , @xcite , have achieved state-of-theart performance in a wide variety of nlp tasks. however, recent studies show that these models often exploit spurious correlations between shortcut tokens and labels, rather than capture underlying semantics related to the target task @xcite @xcite . such llms may suffer from the robustness issue when confronted with out-of-distribution (ood) samples as spurious correlations (i.e., shortcut features) learned from the training data are usu- * corresponding author ally absent in ood samples @xcite . the main causes of shortcut learning include data bias usually caused by data crowdsourcing @xcite and model bias towards learning simple features @xcite . previous works measure the degree of shortcut learning often by data statistics and model interpretability methods @xcite . particularly, they estimate the shortcut degree of each sample based on the tokens that are correlated with labels. however, they do not distinguish shortcut tokens (spuriously correlated tokens) from genuine correlated tokens @xcite . with identified shortcut tokens, various approached have been proposed to suppress llmbased task-specific models learning shortcut features by adjusting the loss function to mitigate shortcut learning, such as sample re-weighting @xcite , product of experts @xcite . these methods have achieved significant improvements on ood samples, but at the cost of undermining the performance of llm-based task-specific models on independent and identically distributed (iid) samples @xcite . additionally, recent studies have found that these methods actually encode more biases to the inner representations of llms @xcite . in this paper, we propose to address these issues via human attention that implicates the cognitive processing behaviour of human brains. with the aid of human attention, we want to encourage llm-based task-specific models to learn relevant features, so as to improve the performance on both iid and ood samples. incorporating human attention into neural models can be regarded as a humanin-the-loop learning, where human feedback has proven capable of not only effectively improving both the accuracy and robustness of models, but also building strong interpretability and credibil-ity for models @xcite . additionally, human attention based approaches have been successfully applied in a range of nlp tasks recently, such as paraphrase generation @xcite , entity linking @xcite . encouraged by these, we propose huaslim, a human attention motivated shortcut learning identification and mitigation framework for large language models. specifically, to identify shortcut tokens, we introduce an attention-based local mutual information metric that takes into account both lexical bias and model behavior bias to detect tokens highly correlated with certain labels. we then automatically distinguish spurious correlations from genuine correlations based on the orthogonal information between human attention-based correlation and neural attention-based correlation, instead of directly using the tokens that are highly correlated with labels. intuitively, 'spurious' tokens are paid more attention to during model training, while less attention in human reading. for shortcut learning mitigation, we based huaslim on self-distillation @xcite . we utilize the estimated shortcut learning degree of each sample to dynamically adjust the temperature in distillation, in the goal of softening the output distribution of the teacher model, thereby discouraging the reliance of llm-based task-specific models on shortcut learning. additionally, we force llm-based task-specific models to learn how humans understand language by simulating human reading behavior. specifically, we introduce a new training objective that drives neural attention to fit human attention distribution. in this way, llm-based task-specific models are trained to explicitly pay more attention to relevant tokens identified by human attention. to avoid the effect of attention heads playing different roles in transformer @xcite , we add an additional soft attention layer after the last layer of llms. human attention signals used in this paper are deduced from human gaze duration generated by ez-reader that has been widely used in the study of human reading process @xcite . this avoid expensive collection of eye movement signals. in a nutshell, our contributions are listed as follows: ## methodology our huaslim aims to use human attention to guide model training, introducing human prior knowledge and reasoning ability into llm-based taskspecific models, thereby improving the robustness of llm-based task-specific models. the architecture of huaslim is illustrated in figure 1 . we use human attention produced by @xcite to identify shortcut tokens. to quantitatively detect shortcut learning, we propose a human attention-based sample shortcut degree measurement. with estimated shortcut learning degree scores, we inhibit llm-based task-specific models from making overconfident predictions for samples containing shortcut features by dynamically adjusting temperature. to further force llm-based task-specific models to focus on relevant features, we minimize the distance between human attention and neural attention with an attention loss. ## conclusions in this paper, we have presented a human attention guided framework that can effectively distinguish spurious correlations from genuine correlations, and significantly alleviate the reliance of llmbased task-specific models on shortcut tokens. by constraining neural attention with human attention, llm-based task-specific models are encouraged to focus on more relevant tokens. experimental results on three nlp tasks demonstrate that our method achieves remarkable improvements on the robustness of llm-based task-specific models on ood samples and preserves the iid performance. further analyses show that our approach is highly interpretable and capable of paying more attention to relevant tokens.