Robust Natural Language Understanding with Residual Attention Debiasing (2023)

Abstract: Natural language understanding (NLU) models often suffer from unintended dataset biases. Among bias mitigation methods, ensemble-based debiasing methods, especially product-of-experts (PoE), have stood out for their impressive empirical success. However, previous ensemble-based debiasing methods typically apply debiasing on top-level logits without directly addressing biased attention patterns. Attention serves as the main media of feature interaction and aggregation in PLMs and plays a crucial role in providing robust prediction. In this paper, we propose REsidual Attention Debiasing (READ), an end-to-end debiasing method that mitigates unintended biases from attention. Experiments on three NLU benchmarks show that READ significantly improves the OOD performance of BERT-based models, including +12.9% accuracy on HANS, +11.0% accuracy on FEVER-Symmetric, and +2.7% F1 on PAWS. Detailed analyses demonstrate the crucial role of unbiased attention in robust NLU models and that READ effectively mitigates biases in attention.

## introduction natural language understanding (nlu) models often suffer from unintended dataset biases @xcite @xcite @xcite , causing them to learn spurious shortcuts and make unfaithful or under-generalized prediction @xcite @xcite . while a number of methods have been proposed to tackle this problem based on prior knowledge of specific biasing features @xcite @xcite @xcite , various unintended biases exist in nlu datasets, and not all of them are identifiable @xcite . more recent works start to focus on mitigating unknown biases @xcite @xcite @xcite . among them, ensemble-based debiasing methods, especially product-of-experts (poe), have stood out for their impressive empirical success @xcite @xcite . although the attention mechanism @xcite is essential to the success of transformer-based pretrained language models (plms), attention can also capture potentially spurious shortcut features leading to prediction biases. for example, too much or too little attention across sentences in natural language inference may lead to the lexical overlap bias @xcite or the hypothesis-only bias @xcite . since attention serves as the main media for feature interactions in plms, many of the aforementioned biases can be associated with biased attention patterns. in fact, a number of recent studies have shown that appropriate attention plays a critical role in ensuring robust foot_0 prediction @xcite @xcite . however, existing ensemble-based debiasing methods typically apply debiasing on top-level logits @xcite @xcite @xcite . these methods do not proactively mitigate attention biases, but instead, rely on debiasing signals being propagated from final predictions to the attention modules in a top-down manner. top-level logits are highly compressed and the propagation may suffer from information loss, thus providing limited debiasing signal to low-level attention. instead, we seek for an effective attention debiasing method that prevents models from learning spurious shortcuts, especially those captured by the attention mechanism. in this paper, we propose residual attention debiasing (read), an end-to-end debiasing method that mitigates unintended biases from attention. our method is inspired by the recent success of onestage poe @xcite . as an ensemblebased debiasing method, it trains a biased model to capture spurious in-distribution shortcuts and trains the ensemble of the biased model and a main model to prevent the main model from relying on spurious shortcuts. to do this end-to-end, one-stage poe trains the biased model and its ensemble with the main model simultaneously in a weight-sharing manner. in read, we let the two models share all weights except attention modules and classification heads, allowing the main model to fit the unbiased attention residual with respect to the attention of the biased model. intuitively, since they are trained on the same batch of data at each iteration, biased model attention and main model attention are likely to capture similar spurious features, making their residual free of such biases. fig. 1 presents an example of the attention change. given a nonduplicate sentence pair, bert, which suffers from lexical overlap bias, does not aggregate much information from non-overlapping tokens. in contrast, read learns to pay more attention to informative non-overlapping tokens. experiments on three nlu tasks show that read significantly improves the performance of bert-based models on ood data where common types of shortcuts are removed, including +12.9% accuracy on hans, +11.0% accuracy on fever-symmetric, and +2.7% f1 on paws. we further examine the attention scores of the debiased main model and find that its distribution is more balanced ( §4.1). these results indicate the crucial role of unbiased attention in robust nlu models. we also demonstrate that our method is still effective when using a biased model with the same parameter size as the main model ( §4.2), which differs from the previous assumption that the biased model of unknown biases should be weaker foot_1 @xcite . our contributions are three-fold. first, we propose read, an ensemble-based debiasing method for nlu models, mitigating attention biases through learning attention residual. second, experiments on three nlu tasks consistently demonstrate that read can significantly improve the ood performance of different nlu tasks with various dataset biases. third, detailed analyses provide useful insights for developing robust nlp models, including the importance and properties of unbiased attention, and the design of biased models in ensemble-based debiasing methods. ## method our method, read, combines one-stage productof-experts (poe) with learning attention residual to mitigate unknown dataset biases for nlu tasks. based on the problem definition, we introduce the two key components of our method, followed by the details of training and inference. ## analysis to provide a comprehensive understanding of key techniques in read, we further analyze the debiased attention distribution ( §4.1) and the effect of number of ensemble layers ( §4.2). ## related work we present two lines of relevant research topics, each of which has a large body of work, so we can only provide a highly selected summary. debiasing nlu models. unintended dataset biases hinder the generalizability and reliability of nlu models @xcite @xcite . while a wide range of methods have been proposed to tackle this problem, such as knowledge distillation @xcite , neural network pruning @xcite , and counterfactual inference @xcite , ensemble-based methods @xcite @xcite stand out for their impressive empirical success. recent works extend ensemble-based methods, such as poe, to mitigate unknown biases by training a weak model to proactively capture the underlying data bias, then learn the residue between the captured biases and original task observations for debiasing @xcite @xcite . @xcite further improves the performance of these methods using a biased model with uncertainty calibration. nevertheless, most prior works only mitigate unintended biases from toplevel logits, ignoring biases in low-level attention. attention intervention. in current language modeling technologies, the attention mechanism is widely used to characterize the focus, interactions and aggregations on features @xcite . although the interpretation of attention is under discussion @xcite @xcite , it still provides useful clues about the internal behavior of deep, especially transformer-based, language models @xcite . through attention intervention, which seeks to re-parameterize the original attention to represent a conditioned or restricted structure, a number of works have successfully improved various model capabilities, such as long sequences understanding @xcite @xcite , contextualizing entity representation @xcite , information retrieval @xcite , and salient content selection @xcite . some recent works also add attention constraints to improve model robustness towards specific distribution shifts, including identity biases @xcite @xcite and structural perturbations @xcite . ## conclusion in this paper, we propose read, an end-to-end debiasing method that mitigates unintended feature biases through learning the attention residual of two models. evaluation on ood test sets of three nlu tasks demonstrates its effectiveness of unknown bias mitigation and reveals the crucial role of attention in robust nlu models. future work can apply attention debiasing to mitigate dataset biases in generative tasks and multi-modality tasks, such as societal biases in language generation @xcite and language bias in visual question answering @xcite .