Learning from Others: Similarity-based Regularization for Mitigating Dataset Bias (2024)

Abstract: Common methods for mitigating spurious correlations in natural language understanding (NLU) usually operate in the output space, encouraging a main model to behave differently from a bias model by down-weighing examples where the bias model is confident. While improving out-of-distribution (OOD) performance, it was recently observed that the internal representations of the presumably debiased models are actually more, rather than less biased. We propose SimReg, a new method for debiasing internal model components via similarity-based regularization, in representation space: We encourage the model to learn representations that are either similar to an unbiased model or different from a biased model. We experiment with three NLU tasks and different kinds of biases. We find that SimReg improves OOD performance, with little indistribution degradation. Moreover, the representations learned by SimReg are less biased than in other methods.1

heading: 1 Introduction
text: Recent studies (McCoy et al., 2019; Geirhos et al., 2020, inter alia) show that in many cases neural models tend to exploit spurious correlations (a.k.a dataset biases, artifacts2) in datasets and learn shortcut solutions rather than the intended function. For example, in MNLI—a popular Natural Language Understanding dataset—there is a high correlation between negation words such as “not, don’t” and the contradiction label (Gururangan et al., 2018). Thus models trained on MNLI confidently predict contradiction whenever there is a negation word in the input without considering the whole meaning of the sentence. As a result of relying on such shortcuts, models fail to generalize and perform poorly when tested on out-of-distribution data (OOD) in
1Our code is available at: github.com/simreg/SimReg 2We use these words interchangeably.
which such associative patterns are not present (McCoy et al., 2019); these models are commonly known as ‘biased’ models. Moreover, this behavior limits their practical applicability in cases where the real-world data distribution differs from the training distribution.
Recent efforts to mitigate learning spurious correlations (a.k.a debiasing methods) perform the debiasing extrinsically, i.e., operating on the output space of the model and dictating how its output should look like. Typically, by downweigh the importance of training samples that contain such correlations, effectively performing data reweighting (Schuster et al., 2019; Utama et al., 2020a; Sanh et al., 2021; Cadene et al., 2019). One might expect that such an extrinsic debiasing would lead to “suppressing the model from capturing non-robust features” (Du et al., 2023). However, Mendelson and Belinkov (2021) showed a counter-intuitive trend: a higher accuracy of such models on OOD challenge sets is correlated with a higher representation bias,3 i.e.,the more extrinsically de-biased a model is, the stronger its intrinsic bias. Such superficial debiasing is problematic as the bias may reappear when the model is used in another setting (Orgad et al., 2022), such as fine-tuned on more data or transferred to other similar tasks.
Inspired by this finding, we investigate whether debiasing the model intrinsically (i.e., operating in the representation space) leads to better models both extrinsically and intrinsically. To this end, we develop SimReg, a new debiasing method based on similarity-regularization. SimReg encourages a model to learn unbiased internal representations by either pushing the learned representations towards a model with good (unbiased) representations, or pushing it away from a model with biased representations. Our approach is different from previous
3Representation bias is measured by how easy it is to classify whether a given representation stems from a biased sample or not.
37
methods, in that we push the model to learn the “good” behavior from other models, while other approaches usually focus on learning to be different from biased models (Utama et al., 2020a; Sanh et al., 2021; Clark et al., 2019; Nam et al., 2020).
We evaluate our approach on three tasks— natural language inference, fact checking, and paraphrase identification—and multiple spurious correlations attested in the literature: lexical overlap, partial inputs, and unknown biases from weak models (see Section 2.1). We demonstrate that our approach improves performance on out-ofdistribution (OOD) challenge sets, while incurring little degradation in in-distribution (ID) performance. Finally, we design an experiment to test the bias remaining in the representations, and find that SimReg models tend to have better performance compared to other debiasing methods.

heading: 2 Related Work
text: A growing body of work has revealed that models tend to exploit spurious correlations found in their training data (Geirhos et al., 2020). Spurious correlations are correlations between certain features of the input and certain labels, which are not causal. Models tend to fail when tested on out-ofdistribution data, where said correlations do not hold. We briefly mention several relevant cases and refer to Du et al. (2023) for a recent overview of shortcut learning and its mitigation in natural language understanding.

heading: 2.1 Dataset bias
text: Partial-input bias. A common spurious correlation in sentence-pair classification tasks, like natural language inference (NLI), is partial-input bias – the association between words in one of the sentences and certain labels. For example, negation words are correlated with a ‘contradiction’ label when present in the hypothesis in NLI datasets (Gururangan et al., 2018; Poliak et al., 2018) and with a ‘refutes’ label when present in the claim in fact verification datasets (Schuster et al., 2019). A common approach for revealing the presence of such spurious correlations is to train a partial-input baseline (Feng et al., 2019). When such a model performs well despite having access only to a part of the input, it indicates that that part has spurious correlations.
Lexical overlap bias. Another common bias is when certain labels are associated with lexical over-
lap between the two input sentences. McCoy et al. (2019) found that high lexical-overlap between the premise and hypothesis correlates with ‘entailment’ in NLI datasets. As a result, NLI models fail when evaluated on HANS, a challenge set where that correlation does not hold. Similarly, Zhang et al. (2019) found that models trained on a paraphrase identification dataset fail to predict ‘non-duplicate’ questions that have high lexical-overlap.
Unknown biases. Identifying the preceding biases assumes prior knowledge of the type of bias existing in the dataset. A few studies have used weak learners to identify biases in the dataset without an prior assumption(Sanh et al., 2021; Utama et al., 2020b). Utama et al. (2020b) proposes to train a model on limited number of samples, the hypothesis is that pre-trained models “operate as a rapid surface learners”, and will learn the bias in the beginning of the training (i.e., with small number of samples). On the other hand, Sanh et al. (2021) proposed to train a limited capacity models such as Tiny-BERT, where the limited capacity tends the learn and recover previously known biases in the literature.

heading: 2.2 Debiasing methods
text: Spurious correlation mitigation can be performed on different levels: Data-based mitigation, where the data is augmented with samples that do not align with the bias found in the dataset (Wang and Culotta, 2021; Kaushik et al., 2020, inter alia). Model/training-based mitigation, where the either the model or the training procedure is modified. A common strategy in this approach is to train a bias model, which latches on the bias in the dataset, and use its outputs to train the final, debiased, main model. (He et al., 2019) and (Clark et al., 2019) used variants of product-of-experts (PoE) to combine the outputs of the biased and main model during training to encourage the main model to “ignore” biased samples. (Utama et al., 2020a) proposed confidence regularization (ConfReg), where they perform self-distillation with re-weighted teacher outputs using bias-weighted scaling, i.e., they induce the model to be less confident on biased samples. These methods can be viewed as data reweighting methods, similar to (Liu et al., 2021), who proposed to up-weigh examples that are missclassified by the biased model, i.e., hard examples. Similarly, Yaghoobzadeh et al. (2021) proposed to perform additional fine-tuning on forgettable
samples after training to increase the robustness (FBOW/FHANS). All these methods work in the output space (extrinsically), while we work in representation space.
Most relevant to our work, Bahng et al. (2020) debias vision models by learning representations that are statistically independent from those of a biased model, by minimizing a statistical independence measure (HSIC) in a min-max optimization objective. We propose a simpler objective function, based on similarity regularization, which can easily be trained by SGD. Additionally, while they focus only on learning representations independent of a biased model, we propose learning representations that are either dissimilar from biased models or similar to unbiased ones.

heading: 2.3 Knowledge Distillation
text: Our approach shares some similarity with Knowledge-Distillation (KD) methods, which transfer knowledge from a teacher model to a (typically smaller) student model. In our framework, we utilize such transfer to improve the robustness of a model. Aguilar et al. (2020) perform KD using internal representations, by minimizing the cosine similarity between the representations of the two models. They compare the similarity of the classification token (CLS) whereas we compare all the tokens. Additionally we use second-order isomorphism methods, whereas they use first-order methods.
To our best knowledge, second-order isomorphism methods were previously mainly used for comparing representations and behaviors of models. Our work is one of the first to utilize them to regularize models during training.

heading: 3 Methodology
text: The key idea of our approach is to guide the representation learning of the model in a coarse-grained manner. We achieve this by encouraging the model to learn representations that are either similar to those of an unbiased model or dissimilar from those of a biased model. We design a three-stage procedure (Figure 1):
1. We train a bias model, fb, on the original training set, D. This model is meant to capture dataset biases, as explained in Section 3.1. In the case of decreasing similarity, we use fb as our target model, fg, and continue directly to Stage 3.
2. In order to obtain an unbiased guidance model, we filter the training set based on the predictions of fb and train a target model fg on the unbiased part of the training set, DU (Section 3.2).
3. We train the main model on D while encouraging its representations to be (dis)similar to those of fg (Section 3.3).

heading: 3.1 Training a biased model
text: To mitigate a specific bias (known-bias), we use a bias-specific model, fb, which is designed to capture that intended bias. For example, to mitigate lexical-overlap bias we use the model proposed in Clark et al. (2019): an MLP whose input features are the ratio of overlap between the two parts of the input, and the average of the minimum cosine similarity between the embeddings of each word from the two sentences. To mitigate unknown-biases, we follow Sanh et al. (2021) and use limited capacity models, such as TinyBert (Turc et al., 2020) and Bag-of-Words (BOW).
In the case of decreasing dissimilarity from a biased model, we use this fb as the target model, i.e., fg = fb, and proceed to Stage 3 (Section 3.3). In the case of increasing similarity to an unbiased model, we cannot use fb as we need an unbiased model; the next section describes how to obtain it.

heading: 3.2 Obtaining an unbiased model
text: To obtain an unbiased guidance model fg, we run fb on the training set, D, and exclude samples on which fb is correct and confident. The remaining samples comprise our unbiased dataset, DU :
DB = {xi|xi ∈ D ∧ fb(x) = yi ∧ c(fb(xi)) > ct}
(1)
DU = D \ DB (2)
where ct is a confidence threshold and c(·) is the models’ confidence. Our unbiased model, fg, is obtained by training a new model on DU .
Choosing the threshold ct is performed manually by plotting the confidence of the bias model over the training set. When there is a significant bias signal in the dataset, we see a spike in the number of biased samples. Figure 5 (Appendix A.4) shows an example for claim-only bias in FEVER.
A natural question is the following: What is the advantage of our framework if we already have
an unbiased model? We emphasize that the unbiased model was trained on DU , a subset of D, and argue that other samples in D could also be useful. Indeed, we show experimentally that training a model on the full training set while regularizing it to be similar to the unbiased model leads to a better ID–OOD tradeoff.

heading: 3.3 Training the main model
text: The final step is to train the main model, fm. We propose two approaches. The first is to encourage the model during training to learn different representations than a biased model, by penalizing its similarity to said biased model, fb (in this case, fg = fb). Thus the model would learn different decision boundaries than the biased model. The second approach is to increase the similarity of the learned representations to an unbiased model, fg. Thus, our model will encode the data in an unbiased manner and its predictions will be less dependant on bias features.
In both cases, we need to compute the similarity between the representations of the main model and those of the target model, fg. Directly comparing the representations of the models on a single example is not possible, since each model might learn a different latent space for representing the data. Furthermore, the two models might have different architectures and dimensionalities. For instance, in some of our experiments we compare BERT-base (768 dimensions) with TinyBERT (128
dimensions) or with an MLP of 70 dimensions. To overcome these challenges, we use second-order similarity measures, which operate at the batch level (Section 3.3.1).
Formally, we add a similarity regularization term to the batch training loss to promote the similarity/dis-similarity. Given a batch B, we minimize the following objective:
L = ∑
i∈B LCE(fm(xi), yi) + λ · sim(Z,H) (3)
where LCE is the cross-entropy loss, λ is a tradeoff hyper-parameter, Z and H are respectively the main and target model representations of the batch, f(x) is the prediction of the model on input x, and sim is a similarity measure. To increase the similarity, we use λ < 0.
Since we wish the main model, fm, to resemble or differ from fg only on biased samples, we apply regularization only on the biased subset, DB: We stochastically sample a batch either from DU and optimize regular cross-entropy, or from DB minimizing the objective in Eq. 3. Section 6.3 shows that regularizing only DB results in better OOD performance, supporting our intuition.

heading: 3.3.1 Models similarity
text: Different models may represent the same input differently in their learned latent spaces. Directly comparing vectors from different models can be problematic. To address this, we employ second-order
isomorphism methods. We assess the similarity of the inputs relative to each other within each modality in a given training batch, then compare the similarity matrices of the two modalities to gauge the resemblance of the model encodings. Specifically, we utilize a well-known similarity measure called Centered Kernel Alignment (CKA; Kornblith et al. 2019) with a linear kernel.

heading: 4 Experimental Setup
text: We run our experiments in two settings: (a) Knownbias settings, where we assume the type of bias existing in the dataset, and can construct targeted biased models; and (b) Unknown-bias settings, where the specific type of bias is not presumed, requiring a more general approach of mitigating unknownbiases.

heading: 4.1 Datasets

heading: 4.1.1 Natural Language Inference
text: We train models on MNLI, a popular NLI dataset consisting of ∼ 400k English examples in multiple genres (Williams et al., 2018). Each example is a pair of premise and hypothesis sentences, and the task is to predict whether the hypothesis is entailed, contradicted, or neutral w.r.t the premise. MNLI contains several spurious correlations as discussed in Section 2, such as lexical overlap and hypothesisonly biases. We train on the MNLI training set and report ID results on dev-matched.
As OOD test set, we use HANS (McCoy et al., 2019) for evaluation against lexical overlap bias. HANS is constructed using structured templates that obey bias heuristics, e.g., the hypothesis overlaps with premise, but with half of the examples having non-entailment labels, as opposed to the bias in MNLI. For hypothesis-only bias we use MNLI-hard, a subset of MNLI’s dev-mismatched set where a hypothesis-only model failed to classify correctly (Gururangan et al., 2018).

heading: 4.1.2 Synthetic MNLI
text: As a sanity test, we introduce synthetic spurious correlations to MNLI (Synthetic-MNLI), following prior work (He et al., 2019; Sanh et al., 2021; Dranker et al., 2021). We prepend the input with a ‘label-token’ that correlates highly with the label. We used tokens <0>, <1>, and <2>, corresponding to entailment, neutral, and contradiction. Following (Dranker et al., 2021), we denote the probability of injecting a token to the input as the prevalence of the bias, and the probability of the
prepended token being correct as the strength of the bias. Through all our experiments, we used prevalence of 1.0 and strength of 0.95. The subsets of examples containing bias token with wrong and right correlations are denoted anti_bias and bias subsets, respectively.
The goal of this setting is to demonstrate the viability of the proposed approaches. Thus we use an oracle unbiased model as fg for the case of increasing similarity, i.e., a model trained on regular MNLI (without synthetic bias). For the bias model, fb, we train a model for a small enough number of steps to capture the bias, judging by the rapid drop of training loss; we found 1k steps sufficient.

heading: 4.1.3 Fact Verification
text: Fact Extraction and VERification (FEVER) (Thorne et al., 2018) is a dataset for fact verification against textual sources. Given evidence and claim sentences, the task is to predict the relation between them: SUPPORTED, REFUTED, or NOT ENOUGH INFO. We followed (Schuster et al., 2019) and trained, evaluated on their processed version of FEVER.
Similar to MNLI, the claim part of the input is spuriously correlated with REFUTES label. We use FEVER-Symmetric (Schuster et al., 2019) for OOD evaluation against claim-only bias. The construction of FEVER-Symmetric ensures that there is no correlation between partial input and labels, thus it enables us to evaluate the extent of debiasing on this type of bias.

heading: 4.1.4 QQP
text: Quora Question Pairs (QQP) is a collection of >400K question pairs from the Quora platform. Given a pair of questions, the task is to predict whether they are duplicate (paraphrase) or nonduplicate. QQP is biased in that question pairs with low lexical-overlap between them are correlated with the non-duplicate label. We train on the QQP training set and evaluate ID on the development set.
Paraphrase Adversaries from Word Scrambling (PAWS) (Zhang et al., 2019) is a dataset for paraphrase identification that is built in a adversarial manner to lexical-overlap bias. The authors scramble the words of a sentence to generate samples with high lexical-overlap that are not a paraphrase. We use the QQP subset of PAWS as our OOD evaluation set for lexical-overlap bias.

heading: 4.2 Models
text: We evaluate our approach using BERT (Devlin et al., 2018) as both fg and the main model. We repeat some of the experiments using DeBERTa-V3 (He et al., 2023) to verify that our method is not specific to BERT. For bias modeling, we used an MLP with lexical features as input following Clark et al. (2019) for lexical-bias modeling. For partialinput bias modeling, we simply train BERT with limited input (only on hypothesis / claim for MNLI / FEVER respectively). In unknown-bias modeling, we use TinyBERT (Turc et al., 2020) for MNLI and QQP, and BOW for FEVER, as our limitedcapacity model, following Sanh et al. (2021). For full training details, see Appendix A.1.

heading: 5 Results

heading: 5.1 Synthetic bias
text: The results on Synthetic-MNLI are in Table 2. All of the SimReg approaches resulted in an increase compared to the baseline on the anti-biased subset, where the synthetic token is mis-aligned with the label. Increasing similarity (↑) performed better than decreasing it (↓). The improvement comes at a cost of a small decrease on the biased subset, which is expected. Compared to an oracle model, which was trained without the synthetic bias, the regularized models perform worse, indicating that they were not able to completely discard the bias.

heading: 5.2 Known bias
text: Tables 1 show the results on known bias cases. All our SimReg models outperform the baseline on the OOD test sets. Increasing similarity (↑) seems to work better than decreasing similarity (↓), consistent with synthetic-bias results. In partialinput bias (MNLI-HARD and FEVER), SimReg
performs almost as well as PoE on the challenge sets, while PoE has a greater degradation on ID dev sets. Turning to lexical-overlap bias (QQP and MNLI-HANS), we see a similar pattern: SimReg performs much better than the baseline on HANS and PAWS (the OOD sets), with little or no degradation on the corresponding ID dev sets. In contrast, PoE and ConfReg struggle. Generally, increasing similarity works better than decreasing it.
A telling comparison is between SimReg and the guidance model fg, which is a model that was trained only on unbiased examples (Section 3.2). In most of the cases, when we increase similarity to this model (rows with ↑), we get models that perform better or similar, on both ID and OOD sets. These results support our hypothesis that increasing similarity to an unbiased model can lead to better representations than those of the unbiased model itself by utilizing more data points.

heading: 5.3 Unknown bias
text: The results of unknown bias mitigation are in Table 3. In this settings, we see similar patterns to known-bias results: SimReg outperforms the baseline and the competitive approaches on challenge sets. Interestingly, in these scenarios, the improvement of SimReg over fg is more prominent, both in challenge datasets and in ID sets.

heading: 5.4 Results with Stronger Models
text: In this section we investigate whether our approach improves the performance of stronger models than BERT. While most work tends to compare with BERT as the baseline, it is important to demonstrate that a new debiasing method is effective also when applied to stronger models.4 We experiment with DeBERTa-V3 (He et al., 2023). As Table 4 shows, SimReg still leads to improvements above the strong DeBERTa-V3. where we see similar patterns to the main results, with SimReg↑ outperforming other approaches. Note that we used here the non-entailment subset of HANS to as our OOD evaluation set in MNLI (Lexical-bias and unknown-bias) to emphasize the improvement on the bias-misaligned subset.

heading: 6 Analysis

heading: 6.1 Similarity Heat-map Analysis
text: To investigate whether our similarity-based regularization achieves its goal, we compute the similarity between every layer in the main model and every layer in the (unbiased) guidance model, and likewise the similarity between layers of the baseline model and layers of the guidance model. We expect our similarity regularization to increase the similarity of the main model to the guidance model, compared to that of the baseline model.
Figure 2 (Upper) shows that, without similaritybased regularization, the bottom layers of the baseline and guidance model are already similar, but the top layers are rather different. This is consistent with findings on how fine-tuning affects mostly the top layers (Mosbach et al., 2020; Merchant
4Bowman (2022) made such a claim about analyzing stronger models; we believe it is similarly important to work on robustifying stronger models.
et al., 2020), as both models started from a pretrained BERT. Figure 2 (Lower) shows that after our similarity-based regularization, the top layers of the main and guidance models become very similar, as desired. Moreover, the regularization also indirectly affects lower layers (bottom row of the heatmap). We conclude that the similarity regularization is successful and affects large parts of the model even when applied only on a few layers.

heading: 6.2 Bias recovery
text: To examine whether representations debiasing does indeed lead to better representations, we designed an experiment to test the bias in the representations. Retraining the classification layer allows us to test to what extent a linear classifier recovers the bias
existing in the dataset from the representations. In Table 5, we present the results of retraining the classifier of the debiased models in unknown-bias settings. In all approaches we see a drop in OOD accuracy when retraining the classifier 5, consistent with Mendelson and Belinkov (2021)’s observation that debiased models still encode the bias in their representations. However, in SimReg↑ we generally get the highest performance compared to other methods. This indicates that the representations produced by SimReg↑ have the weakest signal of the spurious correlations. We repeated this experiment on debiased models in known-bias settings in App A.5, and found similar patterns.

heading: 6.3 Ablations
text: In this section we perform ablations on SimReg↑ on MNLI datasets in unknown-bias settings (using TinyBERT as fb). Table 6 shows the results of ablating different parts of our method while keeping the reset unchanged.
SimReg↑BOW refers to When using a bag-ofwords model as our limited capacity model fb, SimReg obtains only slightly worse performance (SimReg↑BOW row). However it also shows that
5Check Table 10 in the appendix for HANS- evaluation.
the results can depend on the biases that the weak model fb discovers.
Using a pre-trained BERT as our guidance model (-fg row) performs poorly. This highlights that the model that is being used to increase similarity to is an important factor in the process, and that indeed the information is being distilled from fg into the main model.
The last row shows that applying similarity regularization on the entire training set D performs poorly. This result supports our intuition in regularizing only the biased samples DU (Section 3.3).

heading: 7 Conclusion
text: In this work, we have introduced SimReg, a new debiasing approach that employs similarity-based regularization at the representation level. We have demonstrated the effectiveness of SimReg across several NLU tasks, where it notably enhances performance on OOD challenge sets with minimal impact on ID sets.
Additionally, we evaluated the representations of SimReg by testing the amount of bias recovered from the debiased models and found that models debiased using SimReg were least biased after retraining their classifier on a dataset that contains bias. Future work may investigate the effect of simultaneously learning from unbiased and biased models. Another interesting direction is to extend our approach to generation tasks, which would require different similarity measures. Moreover, it is
worth testing the efficacy of SimReg on other types of biases such as social biases.
Ethics Statement
Our work develops a new approach to mitigate spurious correlations in NLU tasks. These are also known as dataset biases, but are different from social biases such as gender or racial bias. One could use our approach to debias against social biases. However, a malicious actor could use our basic approach to increase such social bias, rather than decrease it, by reversing the optimization.
Limitations
Similar to most debiasing methods, the success of our method relies on the existing of enough non-biased samples in the training set, which is used to guide our learning process. Additionally, a notable limitation is in the case of debiasing against unknown-bias, where one might speculate that a certain weak model captures the bias, however, it could either miss the bias, or be more powerful and capture additional non-biased samples. In this case, an inspection to the predictions of the weak model might help.

heading: Acknowledgements
text: This research has been supported by an AI Alignment grant from Open Philanthropy, the Israel Science Foundation (grant No. 448/20), and an Azrieli Foundation Early Career Faculty Fellowship.

heading: A Appendix
text: A.1 Training details
We used pre-trained bert-base-uncased from HuggingFace models (Wolf et al., 2020) for both the main model training and the guidance model in SimReg↑. Trained for 5 epochs with batch size of 64, for better similarity estimation. For MNLI, QQP we used learning rate of 5e−5 and 2e−5 for FEVER, that warms up for 3k steps and decays linearly to 0. The reported results in the tables are the mean and standard deviation of 3 different random seeds. This is true also for competitive methods.
For computing the similarity, we use the mean token representation per layer as the representation of each layer, then we aggregate the similarities of the layers by summing them. We applied the similarity regularization on multiple layers. For increasing similarity we used the last 3 layers from fg and the main model, following insights from Section A.2.
For decreasing similarity, fg had a different architecture, we used a combination of layers that ranged across the models. for example, in FEVER claim bias we used first layer, middle layer and last two layers from both fb and the main model.
As for the threshold ct, we used 0.8 for unknownbias experiments, for known bias we used 0.65 except for FEVER claim-bias where we used 0.8. With λ = 100 for SimReg↑ and λ = 10 for SimReg↓.
A.2 Layers
In the main experiments, we regularized multiple layers together, as described in Appendix A.1. Our choice of layers is based on Figure 3, where we performed SimReg↑ debiasing on Synthetic-MNLI across layers. The results indicate that deeper layers have the most effect on the debiasing, thus in our main experiments we choose layers 10-12 for regularization in SimReg↑. In decreasing representation similarity, individual layers are not effective, as opposed to regularizing multiple layers as in the main experiments. Thus we chose to regularize in a wide manner over multiple layers.
A.3 Synthetic Bias
In this section we present more detailed results for synthetic-MNLI. In Table 7 we show wider range of configuration for the case of increasing similarity. Note that higher λ values for resulted in models
with better performance on the anti-biased set. Bertbase is BERT trained on Synthetic-MNLI, while BERT (Oracle) is trained on MNLI. In Table 8 we present the case of decreasing similarity. Where we see that in this case, λ = 10 is a sweet spot, between not changing much (λ = 1) and changing to much to the level of collapse (λ = 100).
A.4 Threshold choosing
A.5 Bias recovery
In Table 9, we present an additional results of our bias-recovery experiments. Where we re-train the
classification layer of the model on the dataset, to test the amount of biased recovered when evaluating the re-trained classifier + model on the OOD challenge sets. Note that on MNLI we evaluate on the bias-misaligned subset of HANS (the nonentailment subset). For results of the models on these subsets before retraining, check Table 10.
We observe that SimReg↑ generally retains high performance on challenge sets after re-training their classifier on the whole dataset D (with the spurious correlations).
A.6 HANS subsets Table 10 contains the evaluation of debiasing methods on HANS subsets (non-entailment and entailment).

heading: Future Work
text: Future work may investigate the effect of simultaneously learning from unbiased and biased models. Another interesting direction is to extend our approach to generation tasks, which would require different similarity measures. Moreover, it is
worth testing the efficacy of SimReg on other types of biases such as social biases