Guide the Learner: Controlling Product of Experts Debiasing Method Based on Token Attribution Similarities (2023)

Abstract: Several proposals have been put forward in recent years for improving out-of-distribution (OOD) performance through mitigating dataset biases. A popular workaround is to train a robust model by re-weighting training examples based on a secondary biased model. Here, the underlying assumption is that the biased model resorts to shortcut features. Hence, those training examples that are correctly predicted by the biased model are flagged as being biased and are down-weighted during the training of the main model. However, assessing the importance of an instance merely based on the predictions of the biased model may be too naive. It is possible that the prediction of the main model can be derived from another decision-making process that is distinct from the behavior of the biased model. To circumvent this, we introduce a fine-tuning strategy that incorporates the similarity between the main and biased model attribution scores in a Product of Experts (PoE) loss function to further improve OOD performance. With experiments conducted on natural language inference and fact verification benchmarks, we show that our method improves OOD results while maintaining in-distribution (ID) performance.

## introduction overfitting to the training data is a big obstacle in learning patterns that generalize to unseen data. traditionally, this is diagnosed by monitoring the performance of a trained model on an indistribution (id) test set. however, a bigger challenge is when both the training and test data have the same non-generalizable patterns, emerged as spurious correlations between input features and output labels @xcite . for instance, in the natural language inference (nli) task, it is shown that the occurrence of some task-neutral words, like a negation in hypothesis, is highly correlated with a specific class @xcite . while high-capacity models can learn a generalized distribution of labels from the inputs, they are prone to spurious patterns, also known as dataset biases @xcite . a model could exploit these biases during fine-tuning, leading to a model that achieve high id performance, while it is highly fragile in out-of-distribution (ood) settings @xcite . besides trying to prevent these nongeneralizable artifacts from entering the dataset @xcite , it is reasonable to seek for more robust learning methods. this has been the basis for a multitude of research works that encourage models to rely on truly generalizable patterns. most of these methods are based on the assumption that the learning method will inevitably exploit biases if they are present in a training example @xcite @xcite @xcite . therefore, they discourage the main model from paying much attention to the examples which are correctly classified by a biased model. recently, it is shown that this assumption is questionable in the way that for a significant number of cases, the main model does not follow the biased model in treating biased examples @xcite . therefore, depriving the training algorithm from the examples which are detected to be biased is a waste of training data. in this paper, we propose an alternative way to discard biased examples. instead of considering the mere prediction of the biased model, we monitor the way the model processes each example by computing its attribution scores over the input tokens. with the resulting scores, we adjust the proportion of the loss function that is a cross-entropy loss (ce) versus a product of experts loss (poe). if the attribution scores are similar between the main and biased models, the loss becomes a poe loss where a correct prediction from the biased model down-weights the contribution of the corresponding example. in contrast, dissimilarity between the scores suggests a different behaviour from the biased model and leads to a ce loss that only considers the main model's prediction. experiments on natural language inference and fact verification demonstrate that our method significantly outperforms previous approaches in terms of ood performance while preserving its id performance. ## methodology in this section we explain our debiasing solution for a given classification task with dataset d{x i , y i } n @xmath0. for the i th training example, we denote the input sequence of tokens as x i and the gold label as y @xmath1, 2, ..., y } where y is the number of classes. specifically, the goal is to train a modelwe denote it as the main model-on the training dataset (d) so that it can also perform well on ood datasets that do not necessarily share the same biases. following other well-known paradigms for identifying biases in the dataset, we first need to design or train a biased model that employs shortcut methods to complete the task @xcite @xcite . in what follows, we will first review the product of experts (poe) method for bias reduction, which is based solely on the outputs of the main and biased models. then we present our contribution to developing a novel debiasing method that takes into account not only the models' outputs but also their attribution scores. ## experiments in this section, we will introduce the datasets, explain the experimental setup, and then demonstrate the results of our method. ## conclusions in this paper, we introduced a strategy for improving ood performance by incorporating the similarity values between the token attribution scores of main and biased models into a product of experts (poe) loss function. the gist of this approach is that it takes into account the decision-making process in addition to the output predictions of the main and biased models (which are often taken as the primary signal). by comparing our method to multiple recent debiasing methods on two widelyused nlu tasks, we demonstrated that our method improves performance on out-of-distribution data while preserving performance on in-distribution data. future work could include exploring other loss formulations with attribution similarities that may produce better results. additionally, it may be beneficial to investigate methods to reduce the variability of results, which is present in the majority of debiasing solutions.