Learning from Others: Similarity-based Regularization for Mitigating Dataset Bias. (2024)

Abstract: Common methods for mitigating spurious correlations in natural language understanding (NLU) usually operate in the output space, encouraging a main model to behave differently from a bias model by down-weighing examples where the bias model is confident.While improving out of distribution (OOD) performance, it was recently observed that the internal representations of the presumably debiased models are actually more, rather than less biased. We propose SimgReg, a new method for debiasing internal model components via similarity-based regularization, in representation space: We encourage the model to learn representations that are either similar to an unbiased model or different from a biased model. We experiment with three NLU tasks and different kinds of biases.We find that SimReg improves OOD performance, with little in-distribution degradation. Moreover, the representations learned by SimReg are less biased than in other methods.

## introduction recent studies @xcite show that in many cases neural models tend to exploit spurious correlations (a.k.a dataset biases, artifacts foot_1 ) in datasets and learn shortcut solutions rather than the intended function. for example, in mnli-a popular natural language understanding dataset-there is a high correlation between negation words such as "not, don't" and the contradiction label @xcite . thus models trained on mnli confidently predict contradiction whenever there is a negation word in the input without considering the whole meaning of the sentence. as a result of relying on such shortcuts, models fail to generalize and perform poorly when tested on out-of-distribution data (ood) in which such associative patterns are not present @xcite ; these models are commonly known as 'biased' models. moreover, this behavior limits their practical applicability in cases where the real-world data distribution differs from the training distribution. recent efforts to mitigate learning spurious correlations (a.k.a debiasing methods) perform the debiasing extrinsically, i.e., operating on the output space of the model and dictating how its output should look like. typically, by downweigh the importance of training samples that contain such correlations, effectively performing data reweighting @xcite @xcite . one might expect that such an extrinsic debiasing would lead to "suppressing the model from capturing non-robust features" @xcite . however, @xcite showed a counter-intuitive trend: a higher accuracy of such models on ood challenge sets is correlated with a higher representation bias, foot_2 i.e.,the more extrinsically de-biased a model is, the stronger its intrinsic bias. such superficial debiasing is problematic as the bias may reappear when the model is used in another setting @xcite , such as fine-tuned on more data or transferred to other similar tasks. inspired by this finding, we investigate whether debiasing the model intrinsically (i.e., operating in the representation space) leads to better models both extrinsically and intrinsically. to this end, we develop simreg, a new debiasing method based on similarity-regularization. simreg encourages a model to learn unbiased internal representations by either pushing the learned representations towards a model with good (unbiased) representations, or pushing it away from a model with biased representations. our approach is different from previous methods, in that we push the model to learn the "good" behavior from other models, while other approaches usually focus on learning to be different from biased models @xcite @xcite . we evaluate our approach on three tasksnatural language inference, fact checking, and paraphrase identification-and multiple spurious correlations attested in the literature: lexical overlap, partial inputs, and unknown biases from weak models (see section 2.1). we demonstrate that our approach improves performance on out-ofdistribution (ood) challenge sets, while incurring little degradation in in-distribution (id) performance. finally, we design an experiment to test the bias remaining in the representations, and find that simreg models tend to have better performance compared to other debiasing methods. ## related work a growing body of work has revealed that models tend to exploit spurious correlations found in their training data @xcite . spurious correlations are correlations between certain features of the input and certain labels, which are not causal. models tend to fail when tested on out-ofdistribution data, where said correlations do not hold. we briefly mention several relevant cases and refer to @xcite for a recent overview of shortcut learning and its mitigation in natural language understanding. ## methodology the key idea of our approach is to guide the representation learning of the model in a coarse-grained manner. we achieve this by encouraging the model to learn representations that are either similar to those of an unbiased model or dissimilar from those of a biased model. we design a three-stage procedure (figure 1 ): ## experimental setup we run our experiments in two settings: (a) knownbias settings, where we assume the type of bias existing in the dataset, and can construct targeted biased models; and (b) unknown-bias settings, where the specific type of bias is not presumed, requiring a more general approach of mitigating unknownbiases. ## analysis 6.1 similarity heat-map analysis to investigate whether our similarity-based regularization achieves its goal, we compute the similarity between every layer in the main model and every layer in the (unbiased) guidance model, and likewise the similarity between layers of the baseline model and layers of the guidance model. we expect our similarity regularization to increase the similarity of the main model to the guidance model, compared to that of the baseline model. figure 2 (upper) shows that, without similaritybased regularization, the bottom layers of the baseline and guidance model are already similar, but the top layers are rather different. this is consistent with findings on how fine-tuning affects mostly the top layers @xcite , as both models started from a pretrained bert. figure 2 (lower) shows that after our similarity-based regularization, the top layers of the main and guidance models become very similar, as desired. moreover, the regularization also indirectly affects lower layers (bottom row of the heatmap). we conclude that the similarity regularization is successful and affects large parts of the model even when applied only on a few layers. ## conclusion in this work, we have introduced simreg, a new debiasing approach that employs similarity-based regularization at the representation level. we have demonstrated the effectiveness of simreg across several nlu tasks, where it notably enhances performance on ood challenge sets with minimal impact on id sets. additionally, we evaluated the representations of simreg by testing the amount of bias recovered from the debiased models and found that models debiased using simreg were least biased after retraining their classifier on a dataset that contains bias. future work may investigate the effect of simultaneously learning from unbiased and biased models. another interesting direction is to extend our approach to generation tasks, which would require different similarity measures. moreover, it is worth testing the efficacy of simreg on other types of biases such as social biases.