Learning from Others: Similarity-based Regularization for Mitigating Artifacts
Authors: Reda Igbaria
Yonatan Belinkov
Venue: Submitted to ICLR 2023 (2023)
Keywords: NLP
robustness
spurious correlations
Dataset bias
natural language understanding
shortcut learning

Abstract: Common methods for mitigating spurious correlations in natural language understanding (NLU) usually operate in the output space, encouraging a main model to behave differently from a bias model by down-weighing examples where the bias model is confident.
While improving out of distribution (OOD) performance, it was recently observed that the internal representations of the presumably debiased models are actually more, rather than less biased. 
We propose SimgReg, a new method for debiasing internal model components via similarity-based regularization, in representation space: We encourage the model to learn representations that are either similar to an unbiased model or different from a biased model. We experiment with three  NLU tasks and different kinds of biases.
We find that SimReg improves OOD performance, with little in-distribution degradation. Moreover, the representations learned by SimReg are less biased than in other methods.


Under review as a conference paper at ICLR 2023
LEARNING FROM OTHERS: SIMILARITY-BASED REG- ULARIZATION FOR MITIGATING ARTIFACTS
Anonymous authors Paper under double-blind review
ABSTRACT
Common methods for mitigating spurious correlations in natural language under- standing (NLU) usually operate in the output space, encouraging a main model to behave differently from a bias model by down-weighing examples where the bias model is confident. While improving out of distribution (OOD) performance, it recently observed that the internal representations of the presumably debiased models are actually more, rather than less biased. We propose SimgReg, a new method for debiasing internal model components via similarity-based regulariza- tion, in representation space: We encourage the model to learn representations that are either similar to an unbiased model or different from a biased model. We ex- periment with three NLU tasks and different kinds of biases. We find that SimReg improves OOD performance, with little in-distribution degradation. Moreover, the representations learned by SimReg are less biased than in other methods.
INTRODUCTION
Recent studies (McCoy et al. 2019 Geirhos al. 2020, inter alia) show that in many cases neu- ral models tend to exploit spurious correlations (a.k.a dataset biases, artifacts) in datasets and learn shortcut solutions rather than the intended function. For example, in MNLI-a popular Natural Language Understanding dataset-there is high correlation between negation words such as "not, don't" and the contradiction label (Gururangan et al., 2018). Thus models trained on MNLI confi- dently predict contradiction whenever there negation word in the input without considering the whole meaning of the sentence. As a result of relying on such shortcuts, models fail to generalize and perform poorly when tested on -uttof-distribution data (OOD) in which such associative pat- terns are not present (McCoy et al. 2019) these models are commonly known as 'biased' models Moreover, this behavior limits their practical applicability in cases where the real-world data distribution differs from the training distribution.
Recent efforts to mitigate learning spurious correlations (a.k.a debiasing methods) downweigh the importance of training samples that contain such correlations, effectively performing data reweight- ing SScuuster et al. (2019) Utama al. (020aa): Sanh et al. (2021): Cadene al. (2019). Typically, bias-only model is trained and its confidence is used to reweigh training samples. One might expect that such an extrinsic debiasing would lead to "suppressing the model from capturing non-robust fea- tures" Du et al.) (2022). However, Mendelson & Belinkov (202I) showed a counter-intuitive trend: the more extrinsically de-biased model is, the more biased are its representations. i.e., higher ac- curacy of such models on OOD challenge sets is correlated with increase of intrinsic bias. | Such superficial debiasing is problematic as the bias may reappear when the model is used in another setting (Orgad et al. 20222).
Inspired by this finding, we propose to perform intrinsic debiasing on the internal model com- ponents. We develop SimReg, new debiasing method based similarity-regulaizzttii we encourage the internal representations to be either (i) similar to representations of an unbi- ased model or (ii) dissimilar from representations of biased model. We further apply the (dis)similarity regularization on either the model representations or its gradients.
where
'we measure representation-Bu (intrinsic bias) by the easiness of identifying the spurious correlations in the representations (more details in Scc
In case, unbiased model is model that does not rely on the spurious correlations in its decision making.

---

Under review as a conference paper at ICLR 2023
Our regularization framework allows to encourage certain constraints on how the data should be represented (representation regularization), or how the model should be sensitive to the representa- tions of the data (gradient regularization). This is different from previous methods, where the main model usually learns to avoid the errors of bias models. Using our approach allows us to transfer knowledge from other models regarding "good" representation of the data and encourage avoiding "biased" representations.
We evaluate our approach on three tasks-natural language inference, fact checking, and paraphrase identification-and multiple spurious correlations attested in the literature: lexical overlap. partial inputs, and unknown biases from weak models (see Section 2.J). We demonstrate that our approach improves performance on out-of-distributio (OOD) challenge sets, while incurring little degra- dation in in-distribution (ID) performance. Finally, by measuring bias extractability, we find that SimReg representations are less biased than those obtained with competing debiasing methods.
RELATED WORK
A growing body of work has revealed that models tend to exploit spurious correlations found their training data (Geirhos et al. 2020). Spurious correlations are correlations between certain features of the input and certain labels, which are not causal. Models tend to fail when tested on out of distribution data, where said correlations do not hold. We briefly mention several relevant cases and refer to Du et al. (2022) for recent overview of shortcut learning and its mitigation in natural language understanding.
2.1 DATASET BIAS
Partial-input bias. A common spurious correlation in sentence-pair classification tasks, like nat- ural language inference (NLI), is partial-imput bias - the association between words in one of the sentences and certain labels. For example, negation words are correlated with a 'contradiction' label when present in the hypothesis in NLI datasets (Gururangan et al., 2018; Poliak et al.] (018) and with 'refutes" label when present in the claim in fact verification datasets (Schuster al. 2019). A common approach for revealing the presence of such spurious correlations is to train a partial-input baseline (Feng al. 2019). When such model performs well despite having access only to a part of the input, it indicates that that part has spurious correlations.
Lexical overlap bias. Another common bias is when certain labels are associated with lexical overlap between the two input sentences. MMCCo al.) (2019) found that high lexical-overlap be- tween the premise and hypothesis correlates with "entailment in NLI datasets. As a result, NLI models fail when evaluated on HANS, a challenge set where that correlation does not hold. Sim- ilarly, Zhang al. (2019) found that models trained on paraphrase identification dataset fail to predict 'non-duplicate" questions that have high lexical-overlap.
Unknown biases. Identifying the preceding biases assumes prior knowledge of the type of bias existing in the dataset. A few studies have used weak learners to identify unknown biases (Sanh) et al. 2021 Utama et al., 2020b). When limiting either the model capacity or its training data, tends exploit simple patterns.
2.2 DEBIASING METHODS
Spurious correlation mitigation can be performed on different levels: Data-based mitigation, where the data is augmented with samples that do not align with the bias found in the dataset (Wang & Culotta 1 Kaushik et al. 20200, inter alia). Model/training-basee based mitigation, where the either model or the training procedure is modified. A common strategy in this approach
train bias model, which latches on the bias in the dataset, and use its outputs to train the final, debiased, main model. He et al. 2019) and Clark et al. (2019) used variants of product-of-expert (PoB) to combine the outputs of the biased and main model during training to encourage the main model to "ignore" biased samples. Utama et al. (2020a) proposed confidence regularization, where they perform self- distillation with re- weighted teacher outputs using bias-weighe weighted scaling, i.e., they induce the model to be less confident on biased samples. These methods can be viewed as data re- reightig methods, similar to Liu et al. (202T), who proposed up-weigh examples that miss-classified by the
2

---

Under review as a conference paper at ICLR 2023
Araan
Stgge2
Figure I: Illustration of SimReg: (1) train a bias model fa: (2) use its predictions to filter the training set and train a target model fgi (3) train a main model while regularizing its representations to be similar to fg-
biased model, i.e., hard examples. All these methods work in the output space, while we work in representation space.
Most relevant to our work, Bahnng et al. (2020) debias vision models by learning representations that are statistically independent from those of biased model, by minimizing statistical independence measure (HSIC) in min-max optimization objective. We propose simpler objective function, based on similarity regulariation, which can easily be trained by SGD. Additionally, while they focus only learning representations independent of biased model, we propose learning representations that are either dissimilar from biased models or similar to unbiased ones.
2.3 KNOWLEDGE DISTILLATION
Our approach have some similarity with Knowledge-Distilaatoo (KD) methods, where we transfer knowledge from teacher model to a student model (usually to a smaller model to 'compress' the teacher model). In our framework we utilize this transfer knowledge to improve robustness of model. Aguilar et al.) 22200) perform KD using internal representations, by minimizing the sine similarity between the representations of the two models. They compare the similarity of the second-order
classification token (CIS) whereas compare all of the tokens. Additionally isomorphism methods to compare the models whereas they perform first-order methods.
METHODOLOGY
The key idea behind our approach is to encourage model be either similar to unbiased model or dissimilar from a biased model. To achieve this, we design a three-stage procedure (Figure I]:
1. We train bias model, fo. on the original training set, D. This model is meant to capture dataset biases, as explained in Section 3.1 In the case of decreasing similarity, we use fb target model, fg. and continue directly to Stage 3.
2. In order to obtain an unbiased model, we filter the training set based on the predictions of fo and train target model fy on the unbiased part of the training set, DM (Section 3.2).
3. We train the main model on D while encouraging its representations to be (dis)similar to those of fg (Section 3.3)
3.1 TRAINING A BIASED MODEL
To mitigate a specific bias, we use bias-specific model, ft. which is designed to capture the in- tended bias. To mitigate lexical-overlap bias, we use the model proposed in (Clark et al. 2019): an MLP whose input features are the ratio of overlap between the two parts of the input, and the aver- age of the minimum cosine similarity between the embeddings each word from the two sentences. For partial-input bias, we use partial-input model (Gururangan et al.] 2018 Belinkov et al. 2019), ie., we train a model only on the hypothesis/claim part of the ipput for MNLI/FEVER partial-ing inpput biases, respectively. To mitigate unknown biases, model (fo (Sanh et al.] (2021) showed that limited capacity models recover previously wocuusly-koowww known biases in the dataset without explicitly modeling them.
TinyBert (Ture et al. 2020)
bias
3

---

Under review as a conference paper at ICLR 2023
In the case of decreasing dissimilarity from a biased model, this ft as the target model, i.e., fg = fo. and proceed to Stage (Section (.3). In the case of increasing similarity to an unbiased model. we cannot use fa as we need an unbiased model; the next section describes how to obtain it.
3.2 OBTAINING AN UNBIASED MODEL
To obtain an unbiased model, we run fo on the training set, D. and exclude samples on which fo is correct and confident. The remaining samples compose our unbiased dataset, D
D [xi|2 DA fb(x) - yi A c(f(zz))) > a
DVd = DE
(2)
where is a confidence threshold and c(-) is the models' confidence, i.e., the highest probability it assigns among the predictions. Our unbiased model, fg. is obtained by training a new model on D
Choosing the threshold is performed manually by plotting the confidence of the bias model over the training set. When there is significant bias signal in the dataset, we see a spike the number of biased samples. Figure shows an example for claim-only bias in FEVER.
A natural question is the following: What is the advantage of our framework if we already have an unbiased model? We emphasize that the unbiased model was trained on DVd subset of DD and argue that other samples D could also be useful. Indeed, we show experimentally that training model the full training set while regularizing it to be similar to the unbiased model leads to better IID-OOD tradcoff.
3.3 TRAINING THE MAIN MODEL
The final step is to train the main model, fm- We propose two approaches. The first is encourage the model during training to learn different representations than a biased model, by penalizing its similarity to said biased model, fo (in this case, fa ft). Thus the model would learn different decision boundaries than the biased model. The second approach is to increase the similarity of the learned representations to an unbiased model, fg. Thus, our model will encode the data in an unbiased manner and its predictions will be less biased.
In both cases, we need to compute the the similarity between the representations of the main model and those of the target model, fg- Directly comparing the representations of the models on a single example not possible, since each model might learn a different latent space for representing the data. Furthermore, the two models might have different architectures and dimensionalitiis. For instance, in some of our experiments we compare BERT-base (768 dimensions) with TinyBERT (128 dimensions) with MLP of 7 dimensions. To overcome these challenges, order similarity measures, which operate at the batch level (S.ction
second-
Formally, add similarity regularization term to the batch training loss to promote the similarity/dis--imilrrry Given a batch B, we minimize the following objective:
L Σ CC (Z:() vi) + + sim(Z, H)
IEB
where Lex is the cross-entropy loss, À is a trade-off hyper- paaameeprrameter, 7 and H are respectively the main and target model representations of the batch, f(x) is the prediction of the model on input x, and sim is a similarity measure. To increase the similarity, we use À < 0. Unless otherwise noted, we set x = 10 (-10) for decreasing (increasing) similarity.
Since wish the main model, fm. to resemble or differ from fg only on biased samples, apply regularization only on the biased subset, D We stochastically sample batch either from Dd and optimize regular cross-entropy, or from D minimizing the objective in Eq. 3 Appendix A.4 shows that regularizing only DB results in better OOD performance, supporting our intuition.
Finally, we examine two techniques to increase similarity/dis--Se similarity/dis-siiiarry similarit of the main and target mod- els. The first works directly at the representations level, i.e., we regularize the similarity of the representations (activations). The second approach is indirect: We regularize the similarity of the gradients of the main and target models, inducing similar (different) changes to model weights,
4

---

Under review as a conference paper at ICLR 2023
which should make the model more similar (different) to unbiased (biased) model. In this case, we replace the second term in Eq. with sim (VZ, VH).
3.3.1 CHOICE SIMILARITY
Instead of directly comparing two vectors that originate from different models, we use second-order isomorphism methods. In particular, in representational similarity analysis (RSA) Kriegeskorte et al. (200)) we compute a representation similarity matrix (RSM) for each model, which is matrix where each entry is the similarity between two representations the batch: (SM = 2 where column-wise L2 normalization. Then, we compare two RSMs via their correlation:
CosCor H) (RSM -(SM())RRM((() (R) - RSM (H)
(4)
(
where Z the mean of entries and ر the Frobenius norm.
As an alternative, we consider a popular RSA-based measure called centered kernel alignment (CKA:)Kornllith 2019). CKA is computed in a similar manner CosCor, replacing the cosine distance with dot-produe in the RSM computation and using different centring transformation.
Experimentally, we found that CosCor works better, so we report its results in the main paper, and compare with CKA in Appendix|A.2
4
EXPERIMENTAL SETUP
4.1 DATASETS
4.1.1
NATURAL LANGUAGE INFERENCE
We train models on MNLI, a popular NLI dataset consisting of 400k English examples in multiple genres (Williams et al. 2018). Each example is pair of premise and hypothesis sentences, and the task is to predict whether the hypothesis is entailed, contradicted, or neutral w.r.t the premise. MNLI contains several spurious correlations discussed in Section 2 such lexical overlap and hypothesis-only biases. We train on the MNLI training set and report IID results on dev-matched.
As OOD test HANS is constructed using structured templates that obey bias heuristics, e.g., the hypothesis over- laps with premise, but with half of the examples having non-entailment labels, opposed to the bias in MNLI. For hypothesis-only bias we use MNLI-hard, a subset of MNLI's dev-mismatched set where hypothesis-onf model failed classify correctly (Gururangan al. 2018
HANS (McCoy al. 2019) for evaluation against lexical overlap bias.
4.1.2 SYNTHETIC MNLI
As sanity test, we introduce synthetic spurious correlations to MNLI (Synthetic-MNLD) following prior work (He et al.) 2019 Sanh et al.) 2021 Dranker et al.) 2021). We prepend the input with 'label-token' that correlates highly with the label. We used tokens <0> </>, and <2>, corre- sponding to entailment, neutral, and contradiction. Following Dranker et al. 2021), we denote the probability of injecting token the input as the prevalence of the bias, and the probability of the prepended token being correct as the strength of the bias. Through all our experiments, we used prevalence of 1.0 and strength of 0.95. The subsets of examples containing bias token with wrong and right correlations are denoted anti_bia and bias subsets, respectively.
The goal of this setting is demonstrate the viability of the proposed approaches. Thus use an oracle unbiased model as fa for the case of increasing similarity, i.c., a model trained on regular MNLI (without synthetic bias). For the bias model, fo. we train model for a small enough number of steps to capture the bias, judging by the rapid drop of training loss; we found 1k steps sufficient.
4.1.3 FACT VERIFICATION
Fact Extraction and VERification (FEVER) (Thoree et al. 2018) is a dataset for fact verification against textual sources. Given evidence and claim sentences, the task is predict the relation be-
5

---

Under review as a conference paper at ICLR 2023
tween them: SUPPORTED, REFUTED, NOT ENOUGH INFO. We train the FEVER training set and evaluate IID on the development set.
We FEVER- --yymmerrriiccc Schuster al. (2019) for OOD evaluation against claim-only bias. The construction of FEVER-Syming Symmetric ensures that there is no correlation between partial input and labels, thus it enables to evaluate the extent of debiasing this type of bias.
4.1.4 QQP
Quora Question Pairs (QQP) is collection of >>00K question pairs from the Quora platform. Given a pair of questions, the task is to predict whether they are duplicate (paraphrase) or non- con-duplicate. QQP is biased in that question pairs with low lexical-overlap between them correlated with the non-duplicate label. We train on the QQP training set and evaluate III) on the development set.
Paraphrase Adversaries from Word Scrambling (PAWS) Zhang al. (2019) is dataset for para- phrase identification that is built in adversarial manner to lexical-overlap bias. The authors scram- ble the words of a sentence generate samples with high lexical-overlap that
a paraphrase. We use the QQP subset of PAWS as our OOD evaluation set for lexical-overlap bias. For IID, we report Fc-sc of 'duplicate' label, and for OOD we report F1 score of 'non-duplicate" label.
4.2 MODELS
We evaluate our approach using BERT (Devlin al. some of the experiments using DeBERTa (He et al. 2021 to verify that our method is not specific BERT. For full training details see Appendix A.1
2018) the main and target model. We repeat
RESULTS
Table 1: Results SyntheticMMNLL
5.1 SYNTHETIC BIAS
Model
Biased
Anti-biased
BERT-base
98.5 + 0.1 83.8
41.8 + 1.1 82.1
The results on Synthetic-NNLI are in Table D All of the SimReg approaches resulted in a large in- crease compared to the baseline the anti-biased subset, where the synthetic token mis-aligned with the label. Increasing similarity (f) performed better than decreasing it (L). The improvement comes at cost of a small decrease on the biased subset, with the exception of increasing gradient similarity (V SimReg t), where the drop is large. Compared an oracle model, which was trained without the synthetic bias, the regularized models perform worse, indicating that they were not able to completely discard the bias.
Oracle
SimReg t
96.7 = 0.1
61.00 + 0.9
SimReg 4
97.0 = 0.0
49.0 2.4
V SimReg
82.8 = 0.0
72.1 0.0
Siimeg
94.7 = 0.4
57.1 + 2.4
5.2 KNOWN BIAS
Tables and show the results on known bias cases. Focusing first on partial-inpul bias (Table 2 all our SimReg models outperform the baseline on the OOD sets (MNLI hard and FEVER Symmet- ric), except for V SimReg J. which on par. Increasing similarity (t) seems work better than decreasing similarity (L). The improvements are comparable to or better than those of competitive approaches (POE and ConfReg).
Turning to lexical-overlap bias (Table we see a similar pattern: SimReg performs much better than the baseline on HANS non-entailed and PAWS (the OOD sets), with little or no degradation on the corresponding IID dev sets. In this bias type, increasing similarity works much better than decreasing Compared to competitive methods, ConiReg performs bit better on HANS non- entailed. However, SimReg is much better average HANS.
A telling comparison is between SimReg and the guidance model, which a model that was trained only on unbiased examples (fg. Section 3.2) In almost all cases, when we increase similarity to this model (rows with t), we get models that perform better than it, on both IID and OOD sets. Interestingly, in QQP fg performed poorly on PAWS, but using guidance model still improved

---

Under review as a conference paper at ICLR 2023
Table Known bias: Mitigating partial-inpll bias.
MNLI
FEVER
dev
hard
dev
Symmetric
BERT-base Guidance
83.9 0.1 79.2
76.9 + 0.2 78.0
85.4 + 0.1 70.6
58.2 + 0.6 59.7
POE
84.1 0.1
78.02 + 0.4
81.44 + 0.1
59.2 + 0.1
ConfReg
84.3 + 0.1
78.4 + 0.6
85.2 + 0.3
61.0 + 1.7
SimReg f
84.4 0.2
7.1
8..5
61.2 + 0.4
SimReg +
8.1
0.5
84.1
60.3 + 1.1
V SimReg
$3.1
0.1
8.1
60.1 + 0.6
SimReg
$1.7 0.2
0.2
79.9
58.3 + 0.1
Table 3: Known bias: Mitigating lexical-overlap bias.
MNLI
HANS
QQP
dev
non-entailment
entailment
avg
dev
PAWS
BERT-base
83.9 10.1 83.0
30.1 11.3
98.1 1 10.7
64.1
88.4 10.1 87.9
28.2 12.3
Guidance
24.3
89.2
56.7
29.3
PoE
83.9 +0.4
41.8 15.3
93.3 13.4
67.6
75.4 10.3
73.6* +1.7
ConfReg
84.3 +0.2
60.9 +6.6
72.3 +8.5
66.6
85.4 +0.5
28.7 +366
SimReg
83.6
57.6 16.0
82.3 +2.6
70.0
86.9 #0.0
36.9 #1.
SimReg
83.9 +0.1
40.1 #1.
89.4 13.6
64.8
87.7 #0.3
32.6 1.2
V SimReg t
83.4 +0.0
57.9 +1.4
73.0 +2.6
65.5
+0.1
33.2 +1.7
V SimReg |
84.0 10.1
48.8 164
89.8 45.5
69.3
87.5 40.2
31.6 +3.5
performance. These results support our hypothesis that increasing similarity to an unbiased model can lead to better representations than those of the unbiased model itself.
5.3 UNKNOWN BIAS
The results of unknown bias mitigation in Table 4 more challenging than known bias mitigation, with smaller improvements of all debiasing methods compared to the baseline. This is consistent with prior work unknown biases (Sanh al. [021]] Utama et al. 2020b). On MNLI hard, there is practically no improvement, but in other cases we find improvements of up 6-7 points (on HANS and PAWS).
Debiasing from unknown bias seems to be
Comparing the different SimReg models, increasing similarity (t) works better than decreasing (L), and regularizing representations generally better than regularizing gradients, except for PAWS, where increasing gradient similarity works very well. When compared to competitive ap- proaches, SimReg works better HANS and PAWS, but ConiReg is better on MNLI hard and FEVER Symmetric.
5.4 RESULTS WITH STRONGER MODELS
In this section we investigate whether our approach improves the performance of stronger mod- els than BERT. While most work tends compare with BERT as the baseline, it is important to demonstrate that new debiasing method is effective also when applied to stronger models We experiment with DeBERTa-VI (He et al. 202T) and DeBERTa- V3 (Federer et al. 2019). As Table shows, SimReg still leads improvements above the strong DeBERTa- V3. Notably, ConfReg does
Bowman 2022 made such claim about analyzing stronger models; we helieve it is similarly important work robustifying stronger models.

---

Under review as a conference paper at ICLR 2023
Table 4: Results for unknown bias mitigation.
MNLI
HANS
FEVER
QQP
dev
hard
avg
dev
Symm.
dev
PAWS
BERT
83.9 +0.1
76.94-0.2
64.1
85.4+0.11
58.240.6
88.4 #0.1
28.2 +2.2
Biased
64.6:t0.1
51.4+0.2
50.0
71.3#0.1
45.3+0.4
80.6 #0.1
30.4 #0.2
Guidance
80.9
74.9
61.9
64.9
50.9
73.13
55.4
POE
83.1#0.1
75.8+0.5
65.4
84.2+0.1
57.7+1.7
87.6 #0.4
34.7 #6.1
ConfReg
83.4+0.4
77.0 +0.5
63.2
86.0 +0.2
60.0 =1.6
86.6 +0.3
16.6 =3.0
BERT + Fbow
83.0 +0.4
76.6
70.4
87.1 #0.2
61.0 +1.4
SimRegt
82.9
76.1+0.1
68.2
85.6+0.3
59.1+0.2
84.4 #0.1
40.6 +0.6
SimRegl
82.9+0.3
75.8+0.6
63.5
84.1+00.
57.5+2.2
86.6 +0.7
32.4 +4.6
SimRegt
83.4+0.0
75.9+0.33 75.3+0.4
64.8
81.1+0.00
59.1+0.1
82.2 +0.3
45.5 +1.5
VimRegg
82.9+0.3
62.7
83.8+1.11
57.2+1.5
84.0 +2.0
36.4 +13.8
Table 5: Results with DeBERTa-V3.
MNLI
HANS
QQP
dev
hard
non-ent
dev
PAWS
baseline ConfReg
89.9 #0.1
85.2 #0.1
56.7 +2.2
89.9 #0.1
55.7 15,6
90.1 #0.1
86.0 #0.1
54.5 #2.0
88.8 #0.1
61.1 #2.0
SimReg f
89.1 =0.2
85.1 #0.3
66.8 #0.7
86.3 #0.2
67.0 #2.0
VSimReg
87.9 1.6
83.9 #1.1
74.8 #13
89.3 #0.1
58.2 #1.5
not improve the baseline in this case, while SimReg does. Similar improvements are obtained with DeBERTa-VI (Table 13 App. A.8).
ANALYSIS
6.1 PROBING FOR BIAS EXTRACTABILITY
Recall that Mendelson Belinkov (202T) found that, counter-intuitively. extrinsic debiasing meth- ods, like POE and ContReg, increase the bias in models' internal representations. Their finding motivated us to propose SimReg, which regularizes the representations themselves. Their discov- ery made by measuring the extractability of bias from model representations, for which they used compression, a measure from minimum description length probing (Voita & Titov 2020). We perform similar analysis of models. Figure shows the increase in compression of debi- ased models compared to the baseline, against the increase of performance on OOD challenge set HANS (non-entailed). Consistent with Mendelson & Belinkov (2021), ConfReg leads to large performance increase on HANS, but suffers from a large increase in bias extractability. In con- trast, SimReg able improve performance OOD challenge with little increase in bias extractabilitty supporting our motivation to perform debiasing at the representation level.
6.2 BIAS RECOVERY
We demonstrate the importance and effectiveness of de- biasing the representations of model. We retrained the classification layer of the resulted models after debias- ing. In Table 6 we present the results on lexical-overlap bias; in A.6 we present other configurations. all drop HANS non-entailment
Table 6: lexical-overlap bias recovery
IID
HANS
POE ConfReg
83.4 +0.2
38.2 +4.2
approaches there accuracy. In SimRegt we got the highest performance on the OOD challenge set. This indicates that the repre- sentations produced by SimRegt have the least signal of lexical-overian spurious correlation.
84.8 10.1
20.5 16.2
BERT + From
83.7 10.2
42.2 11.2
SimRegt
84.3
44.2 +4.4
8

---

Under review as a conference paper at ICLR 2023
Overlap
Subsequence
7.
63-
Conf Tog
Coo Regi
60
Cooppeessio
6.-
RimRig
CSanfig
Sunflog
POE
Sumfing
boadlone
POE
Vi
0.
HANS noo-entilment
A HANS
Figure 2: Compression increment relative to HANS non-entailment subset. Left: Overlap bias. Right: Subsequence bias. SimReg leads to lower increase intrinsic bias than ConfReg.
Figure 3: Similarity of unbiased model, fg. either a baseline (Left) or a SimReg model (Right). Similarity regularization makes top layers more similar to the unbiased model, as desired.
6.3 SIMILARITY HEAT-MAP ANALYSIS
To investigate whether similarity- -based regularization achieves its goal, compute the sim- ilarity between every layer in the main model and every layer in the (unbiased) guidance model, and likewise the similarity between layers of the baseline model and layers of the guidance model. We expect our similarity regularization to increase the similarity of the main model to the guidance model, compared to that of the baseline model.
Figure 3 (Left) shows that, without similarity- -based regularization, the bottom layers of the baseline and guidance model are already similar, but the top layers are rather different. This consistent with findings on how fine-tuning affects mostly the top layers (Mosbach et al. 2020 Merchant al., 2020), as both models started from pre-trained BERT. Figure 3 (Right) shows that after similarity-based regularization, the top layers of the main and guidance models become very similar, as desired. Moreover, the regularization also indirectly affects lower layers (bottom row of the heatmap). We conclude that the similarity regularization is successful and affects large parts of the model even when applied only on a few layers.
6.4 ABLATIONS
Our main reported results were with CosCor the similarity method (Section 3.3.I In Ap- pendix A.2 we report experiments with CKA as an alternative measure, showing it also leads to consistent improvements compared the baseline, albeit not good CosCor. Additionally, in the main experiments we reported results when regularizing only on batched from a biased sub- set, DB_ In Appendix A.4 we compare with results when regularizing all samples, finding that regularizing only on the biased samples is better.
7 CONCLUSION
In this work, we have proposed SimReg, a new debiasing approach using similarity- based regular- ization. We have explored several variants of this approach, regularizing by increasing or decreasing similarity of model to biased unbiased models, respectively, the level of either the representa- tions or the gradients. We found SimReg to improve performance on OOD challenge sets on multiple bias types and NLU tasks, with little decrease in IID performance. Future work may investigate the effect of simultaneously learning from unbiased and biased models. Another interesting direction is extend approach to generation tasks, which would require different similarity measures.

---

Under review as a conference paper at ICLR 2023
ETHICS STATEMENT
Our work develops a new approach to mitigate spurious correlations NLU tasks. These are also known dataset biases, but different from social biases such gender racial bias. One could use our approach to debias against social biases. However, a malicious actor could use our basic approach to increase such social bias, rather than decrease it, by reversing the optimization. In such a case, probing for bias extractability as we performed in this work may expose biased representations.
REPRODUCIBILITY STATEMENT
We provide detailed configurations in Appendix A.T Code in PyTorch (Paszke et al. 2019) for reproducing our results available at lhttps://githe regularization experiments require twice the memory of representation regularization. For gradients experiments we used an NVIDIA A40 GPU; a typical training time was ~ 7 hours. For representa- tions experiments, used NVIDIA A4000 RTX GPU, and training time was hours.
com/ sts//iiiii Swamee//Simeee
Gradient
REFERENCES
quora...oo.o.oooooo... com/F han//Firsttt
Gustavo Aguilar, Yuan Ling, Yu Zhang, Benjamin Yao, Xing Fan, and Chenlei Guo. Knowl- edge distillation from internal representations. Proceedings of the AAAI Conference on Artifi- cial Intelligence, 34(05):7350-73577 Apr. 2020. doi: 10.1609/aami v34i05.6229. URL https:
Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Scong Joon Oh. Learning de- net/
biased representations with biased representations, 2020. UtLittss://
Forum??d-=SJeHw11SStt
Yonatan Belinkov, Adam Poliak, Stuart Shieber, Benjamin Van Durme, and Alexander Rush. Don't take the premise for granted: Mitigating artifacts in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 877-891, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.18653//1/P19-1066 URL
Samuel Bowman. The dangers of underclaiming: Reasons for caution when reporting how NLP systems fail. In Proceedings of the 60th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers). pp. 7484-7499, Dublin, Ireland, May 2022, As- sociation for Computational Linguistics. doi: 12211666//1/222222a---ooggg //aclanthology crt. 2occlatthoooyy.oog/2222...o acl-long. 516
URL https:
Remi Cadene, Corentin Dancette, Matthieu Cord, Devi Parikh, al. Rubi: Reducing unimodal biases for visual question answering. Advances in Neural Information Processing Systems, 32: 841-852, 2019.
Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. Don't take the easy way out: Ensem- ble based methods for avoiding known dataset biases. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLLP-JCCLLP) pp. 4069-4082, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18553/v URL https::/aclaathh...e
wtt/ 1418
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina N. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. 2018. URL wtllhttts:///axiv.oom
05s/18110048000
Yana Dranker, He He, and Yonatan Belinkov. IRM- when it works and when doesn't: A test of natural language inference. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https::/ openreview net for.m
10

---

Under review as a conference paper at ICLR 2023
Mengnan Du, Fengxiang He, Na Zou, Dacheng Tao, and Xia Hu. Shortcut learning of large language models in natural language understanding. survey. arXiv preprint rrXiv:2208.11851 2022.
Callie Federer, Haoyan Xu, Alona Fyshe, and Joel Zylberberg. Training neural networks to have brain-like representations improves object recognition performance. CoRR, abs/1905.10679,
2019. PRLbttt://aaet wwtpw/wwwww..aaaa..aaaaaaaaaaaaa org/ 1900. 10679
10679
Shi Feng, Eric Wallace, and Jordan Boyd-Graber. Misleading failures of partial-input baselines. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5533-5538, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/ v1/P19-1555
R. Geirhos, J. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665 673, 2020. doi: 10. 101101043225600----222 022-002---
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation artifacts natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 2 (Short Papers), pp. 107-112, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10. 10.18653/v1/N18- 1/8122017. URL https: ac /all:://cllaaa http/aaaanntto......oo org/N18 2017
He He, Sheng Zha, and Haohan Wang. Unlearn dataset bias in natural language inference by fit- ting the residual. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low- Resource NLP (DeepLo 2019), pp. 132-142 Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653////119-61111 URL /aatps://accaattholl ogy.
105/D19-6115
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Wei Chen. Deberta: Decoding-enhanced bert with disentangled attention. In 202/ International Conference on Learning Representations, May 2021. URL htRl.httss:// wwwww//wwwwwm.. deberrta eeatreeee bert wwttetwwtthhheet htt.://wwwtttt.aaaaaaaaaaaaaaaar
Under review.
Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. Learning the difference that makes a differ- ence with coutterfacualll--agmeeeeee data. In International Conference on Learning Represen- tations, 2020. URRt:tts:://openeree
net -tt.:e/ooooommddd-11lg00000
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neu- ral network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceed- ings of Machine Learning Research, pp. 3519-3529, PMLR, 09-15 Jun 2019. URL https: //proceedings ml www.:mww........mmmmmmmmmmm
html
Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini. Representational similarity analysis connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 2008. ISSN 1662-5137. doi: 10.3389/neuro. 06. 000.00008
URL https::/ h/wtps:://ww........... front
org/articles/10 3389 itt./3333//neno neuro. 06. 004 2008
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In Marina Meila and Tong Zhang (eds.), Proceedings the 38th International Conference on Machine Learning, volume 139 Proceedings of Machine Learning Research, pp. 6781-6792. PMLR, 18-24 Jul 2021. URL https: http:://procceedingg mww. pta....rees//1//////
1iu21f. html
Tom McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pp. 3428-32282 3448, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 18.13//3/v 1/P19- 1334. URL https: //ap///thoaattog.........
org/ P1 9334
11

---

Under review as a conference paper at ICLR 2023
Michael Mendelson and Yonatan Belinkov. Debiasing methods in natural language understanding make bias more accessible. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 1545-1557, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguisties. doi: 10. 10.18653/v1/2021 /ttp://///221..elmmmmoo 116. httLhtttsss///claamii http////....aao.ooooooor///////a
emn p-
116
Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick, and Ian Tenney. What happens to BERT em- beddings during fine-tuning? Proceedings of the Third BlackboxNLP Workshop on Analyz-
ing and Interpreting Neural Networks for NLP, pp. 33-44, Online, November 2020. Associa- tion for Computational Linguistics. doi: 10.18653/v http///000...aaoooooooooooo hww.aatthooooyy.oo/2222022 http////w...aaaoooono1o--o1--//.
URL https:
Marius Mosbach, Anna Khokhlova, Michael A. Hedderich, and Dietrich Klakow. On the interplay between fine-tune one-tuning and sentence-level probing for linguistic knowledge in pre-trained transform- ers. In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pp. 68-82, Online, November 2020. Association for Computational Linguis- tics. doi: 10.18653/v1/2200 blackboxnlp-1.7
http////0/20..bbcooooooppp blackboxnlc- 1.7.
Hadas Orgad, Seraphina Goldfarb-Tarran and Yonatan Belinkov. How gender debiasing affects internal model representations, and why it matters. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.
Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10. 18655//112222..naa22 2022. www.naaallaaaam -aaee-l-mainn.mm 188
188. URL
org/
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. Hypothesis only baselines in natural language inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pp. 180-191, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10. 1/118866///SS1820223 URL https: wwc.lannhooogy..rgg....o org/S18 211//188822333
Victor Sanh, Thomas Wolf, Yonatan Belinkov, and Alexander Rush. Learning from others' mis- takes: Avoiding dataset biases without modeling them. In International Conference on Learning Representations, 2021. WRLLhtpps:t.
Hf3ggoiNNRRRRR
Tal Schuster, Darsh Shah, Yun Jie Serene Yeo, Daniel Roberto Filizzola Ortiz. Enrico Santus, and Regina Barzilay. Towards debiasing fact verification models. In Proceedings of the 2019 Con- ference on Empirical Methods Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNIP), pp. 3419-3425, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 10.18653/1//D19-133 1341.
URL
1341
James Thorne, Andreas Vlachos, Christos Christodoulopoulls, and Arpit Mittal. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume (Long Papers), pp. 809-80 919, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10. 10.18653/11/N18-1000 URL
Tulia Ture, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Well-read students learn better: On the importance of pre-training compact models, 2020. URL. htttps://openrevee
Prasetya Ajie Utama, Nafise Sadat Moosavi, and Iryna Gurevych. Mind the trade-off: Debias- ing NLU models without degrading the in-distribution performance. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8717-8729, Online, July 2020a. Association for Computational Linguistics. doi: 10.18653/v1/2000..m acl- 770..770. URL
https: /opps//acc antho www.:ololl....gg..... accl mmain.
12

---

Under review as a conference paper at ICLR 2023
Prasetya Ajie Utama, Nafise Sadat Moosavi, and Iryna Gurevych. Towards debiasing NLU mod- els from unknown biases. In Proceedings of the 2020 Conference on Empirical Methods in
Natural Language Processing (EMNLP), pp. 7597-7610, Online, November 2020b. Associa- tion for Computational Linguistics. doi: 10. 12211663/1112200.mml 1/265///122200..mmmmmam........
URL https:
//aclanthology .rg 2020. emn poamnp-mmainn 613
Elena Voita and Ivan Titov. information-theoree probing with minimum description length. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process-
ing (EMNLP), pp. 183-196, Online, November 2020. Association for Computational Linguis- tics. doi: 10.18653/v httpp///e..omm 1/2020.emmlp-mmin..44 http:t http//claannhhoooooo org 2020.
emm.pp ma mnain.. 14
Zhao Wang and Aron Culotta. Robustness to spurious correlations in text classification via au- tomatically generated counterfactuals. Proceedings of the AAAI Conference on Artificial In- telligence, 35(16):14024410031 May 2021. doi: 10.1609/aaai 10.1609/maii.v3511...7...
URL https:
[/ojs. orr. /ptp/AAAl//rrttitle//eee///////a
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume / (Long Papers), pp. 1112-1122. Association for Computational Linguistics, 2018. URL http://awwwebbbb.raaaaaaae wwt.o///n/nooooo/o/ooloo 11C1
Thomas Wolf, Lysandre Dehut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, al. Huggingface's 's transformers: State-of-the-att natural language processing. arXiv preprint arXiv:1910.03771 2019.
Yuan Zhang, Jason Baldridge, and Luheng He. PAWS: Paraphrase adversaries from word scram- bling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume I (Long and Short Pa- pers), pp. 1298-1308, Minneapolis, Minnesota, June 2019. Association for Computational Lin- guistics. doi: 18.18653/v 1/t1:///////-11111 11311, Utthttpss: / accl anthonatthll. www.oooooooooog.i.... 1131
13

---

Under review as a conference paper at ICLR 2023
A
APPENDIX
A.1 TRAINING PARAMETERS
from HuggingFace models (Wolf et al. 2019), trained for 10 epochs with batch size of 64 and learning rate 2e-5 that warms up the first 10% of the train- ing steps and decays linearly. The results reported in the tables are the mean and standard deviation of different random seeds (changes the initialization of the model and the training batches).
We used pre-trained
We performed the similarity regularization on multiple layers, aggregating the similarities by sum- ming it across the layers. When increasing similarity fg is the same architecture fm- Experi- mentally we found that upper layers are most effective for this case. We regularize layers 10, 11, and 12, each layer in fg paired with its parallel layer in fm. For gradient regularization, we choose layers from across the model: embedding layer and layers 1, 7, 11, and 12, and use this set of lay- ers for increasing and decreasing gradient similarity whenever ʃg has the same architecture as fm (partial-input biases for ))
For decreasing similarity from models with a different architecture, using a combination of layers yielded the best results, and we used the upper two layers and the embeddings layers. For gradient dis-similaritt, regularizing the gradient of lower layers was the major factor as validated in Appendix A.3 which provides additional results when regularizing different layers.
A.2 RESULTS WITH CKA
In Table we repeat experiments from the main paper, but using CKA as the similarity measure instead of CosCor. This approach also leads to improvements compared to the baseline, but not as large as using CosCor.
Table 7: Results of SimReg with CKA as the similarity measure.
MNLI
HANS
dev
hard
non-entailment
entailment
Unknown bias
SimReg f
84.1 + 0.2
77.1 0.4
35.5
96.4 + 0.2
SimReg 1
82.4 0.3
7
3.5
79.0 2.0
VimReg
76.5 + 0.1
70.7 0.3
87.9 0.5
25.5 + 1.6
VimReg
81.8 0.5
7.72
84.4 + 3.0
Lexical bias
SimReg
83.6 1:6:100 0.1
6.0
82.3 + 2.7
SimReg $
83.9 + 0.1
36.7
96.2 + 1.4
VimReg
82.9 0.2
59.0
68.7 5.5
VimmReg
83.8 + 0.1
41.8 4.8
93.6 + 1.1
Hypothesis-only
SimReg f
82.4 0.2 0.2
SimReg |
82.3 0.1
7.73
VimmReg
77.0
0.2
SimReg
81.6 0.7
A.3 LAYERS
In the main experiments, we regularized multiple layers together, as described in Appendix A.1 In Tables and we present the results when regularizing different layers, where we regularize only one layer at a time. We see several patterns. In the case of increasing representation similarity, deeper layers work better. In the case of regularizing gradients, early layers work better. decreas- ing representation similarity, individual layers not effective, opposed to regularizing multiple layers as in the main experiments. To simplify the choice of layers, we reported results with a set of layers described in Appendix A.1
14

---

Under review as a conference paper at ICLR 2023
Table Layers search.
SimRegT
VimReg
bias
anti-bias
bias
anti-bias
Embeddings
98.7 =0.1
39.3 10.9
96.1 =0.2
63.8 #0.7
Layer-2
98.2 =0.0
45.6
96.9 0.00
59.9 =0.0
Layer-4
98.7 #1.0
112.
97.3 #.1
56.6 #0.3
Layer-6
97.3
50.9
98.4
#20.4
Layer-8
96.3 =0.0
59.8
98.5 =10
45.1 #114
Layer-10
94.9 10.1
68.1
98.6 10.5
41.2 19.9
Layer-12
96.0 10.1
11.2
99.0 10.1
35.0 12.9
Table dis-similarity Layers search.
SimRegl
ViimReg 4
bias
anti-bias
bias
anti-bias
Embeddings
98.9 +0.0
35.7 +0.1
94.7 +0.4
57.1 4244
Layer-2
98.8 +0.1
37.1 +0.3
77.2 =14.2
40.1 +31.9
Layer-7
99.0 +0.0
38.4.:0.
91.1 100.0
43.6 121.3
Layer-12
98.7 +0.0
36.4 +1.0
100.0 #0.0
0.5 #0.7
A.4 REGULARIZZTION SET
In the main body have reported results when performing similarity-based regularization only on the biased samples, D as deseribed in Section 3.3 In Table 10 we compare to the case of regularizing all the training samples. Clearly, it is important to perform regularization only on DB This is especially true for the dissimilarity case, where the model regularized on all samples sometimes fails converge, leading random performance MNLI dev and constant predictions on HANS (see last row).
Table 10: Comparing regularizing only D regularizing all samples.
Bias
MNLI dev
HANS
non-entailment
entailment
SimReg f
83.6 43.6 0.1
57.5 + 6.0
82.3 + 2.7
SimRegt (all)
84.4 +0.0
7.4 41.4
98.4 +0.4
Lexical-overlap
SimReg J
0.1
40.1 + 1.7
89.4 + 3.6
SimReg (all)
8.2
4.3
96.8 + 1.3
VimRRgg
$3.3
56.4 #2.8
75.6 +0.8
Vimneggt (all)
46.1
61.7 454.1
44.4 162.7
VimRegg
84.0 40.1
48.8 16.4
89.8 15.5
SimReg (all)
32.8 #0.2
100.0
0.0
SimReg ?
0.2
1.11
82.4 2.5
SimReg (all)
0.1
7.7
08.3
SimReg
0.3
41.4
85.6 + 4.3
Unknown
SimReg (all)
8.2
3.4
98.9
VSimRegt
82.7
54.4
75.0
SeimRegt (all)
82.8
32.7
82.7
SimReg +
8.2
4.00
06.2+633
SimReg 1 (all)
3.6
1.0
0.0 0.0
A.5 SYNTHETIC BIAS
In this section we present more detailed results for MNLLL In Table II we show wider range of configuration for the of increasing similarity. Note that À values for resulted in models
15

---

Under review as a conference paper at ICLR 2023
with better performance the anti-biased set. Bert-base is BERT trained synthetic-MNLLI while BERT (Oracle) trained on MNLI. Table 12 we present a partial list of decreasing similarity experiments.
'Table 11: Increasing similarity: Synthetic-MNLJ evaluation for prevalence=1 and strength=0.95, denotes regularizing gradients.
Biased
Anti-biased
Unbiased
Bert-base
98.5 + 0.1 83.5 + 0.3
41.8 + 1.1 82.0 0.9
78.0 + 0.4 84.0 0.3
BERT (Oracle)
CosCor
SimReg t [A 1)
9.0
57.0
62.6
SimReg f (A = 10)
9.0
0.8
8.1
SimReg (A 100)
961
0.6
8.1
ViimReg t (A = 1)
9.1
6.6
8.7
VimmReg t (A 10)
9.2
56.7
81.1
VimReg f (A = 100)
82.8
72.1
79.1
Linear-CKA
SimReg t (A = 1)
97.3 0.0
57.7
83.
SimReg f (A = 10)
9..
6.4
83.
SimReg f (A = 100)
96.7
6..0
8.2
SimRegg t (A =
98.1
3.0
7.74
VimmReg t (A = =100
9..7
1..5
5.7
VSimReg t (A 100)
0.0
32.2
22.7=000
Table 12: Decreasing similarity: Synthetic- evaluation
evaluttio for prevalence=1 and strength=0.95.
Model
Biased
Anti-biased
Unbiased
Bert-base
98.5 0.1 99.9
41.8 + 05.7
78.0 + 0.4 64.3
Biased Biased-BERT
CosCor
SimReg (A = 1)
99.6 +0.6
12.2 11.3
53.4 +25.9
SimReg () 10)
97.0 0.1
49.0 + 2.4
76.1 + 2.0
SimRegl (A = 100)
33.5 +16
32.3 +0.4
33.3 +1.6
Linear-CKi
SimReg 4 () = =1)
9.1
2..9
69.0 0.4
SimReg J (A = 10)
0.7
0.0
32.3
SimReg (A = -100)
32.3
0.6
3.7
A.6 BIAS RECOVERY
Here we present the full results of our bias-rey experiments.
A.7
THRESHOLD CHOOSING
A.8 RESULTS WITH DEBERTA-VI
Table 13 provides results with DeBERTa- VI. SimReg consistently improves the baseline, demon- strating its success with stronger models than BERT.
16

---

Under review as a conference paper at ICLR 2023
FEVER development
Frequency
6.0%
3.0%
0.0%
0.4
0.6
0.8
1.0
Cf ))
Figure 4: Confidence distribution of a claim-only model (fb) FEVER; here c 0.8.
Table 13: Results with stronger models DeBERTa VI.
Hypothesis
Lexical
IID
OOD
IID
OOD
baseline
88.2 #0.2
82.3 #0.1
88.2 +0.2
75.2 +0.9
Known bias
SimReg
88.2 +0.0
83.1 +0.0
+0.0
80.5 +0.6
SimReg
87.0 10.6
82.1 10.3
87.8 10.1
77.3 11.5
SimRegg 1
87.2 10.1
82.7 10.2
87.5 10.2
79.8 10.6
SimReg +
85.0 +1.1
81.7 +2.7
87.8 +0.0
78.4 +1.3
Unknown bias
SimReg
88.3 #0.1
83.0 +0.3
88.3 10.1
74.6 +1.
SimReg
10.0
82.3 10.6
87.7
76.5 40.3
VimReg f
80.2
80.8 10.4
85.4
73.7 40.7
SimReg $
87.3
81.8
40.1
71.6
17


Reviews:
[{"review_id": "yLF2v_SVhC", "reviewer": "Reviewer_UAbR", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a regularization method for mitigating artifacts. Their method (SimReg+) is simple: take a model that is known to be artifact-free, and train another model to be similar to that model. The idea is somewhat counter-intuitive, but works reasonably well on some datasets, and is also shown to have fewer internal biases. The authors also propose several variants of the main idea (e.g., decreasing the similarity between the model and some model that is known to be biased), but those do not work as well as the main idea. ", "summary_of_the_review": "The paper presents a simple method that seems to overall work well. However, it is not entirely clear how generalizable it is, given that it is not very intuitive and the results are sometimes mixed.", "strength_and_weaknesses": "\nStrengths:\n- A simple approach for mitigating artifacts\n- Results for SimReg+ are generally better than most baselines\n- The second-order similarity measure seems interesting\n- The internal bias representation results are interesting\n\nWeaknesses: \n- The proposed approach is somewhat counter-intuitive: why would making a model imitate a weak model (i.e., one trained on less data) result in a stronger model? This is sort of like doing knowledge distillation where the teacher is the smaller model. I see that empirically the idea sort-of works, although sometimes it does much worse (e.g., HANS non-entailment, PAWS) and in some cases the differences are small (Table 2). I wonder how generalizable this approach is.\n- Related to the previous point - for some reason the Guidance baseline is missing from the Deberta experiments (Table 12 doesn't show any baselines except the original model).   \n- Given that SimReg+ works best among all proposed methods, it might make sense to focus on it, and present the rest as ablations\n\nOther questions:\n- It is not clear why the authors chose HANS- as the X axis in Figure 3. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "qgwLjaHxKSF", "reviewer": "Reviewer_qJJe", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method to address the spurious correlations / bias problem in the context of NLU. Starting from the observation in Mendelson & Belinkov (2021) that most de-biasing methods consisting in re-weighting data result in actually more biased representations, they explicitly consider to regularize the representations (or the gradients) of the full model to be more similar to a de-biased \"guidance\" model (trained on small amounts of de-biased data found by either an oracle or by a biased model), in order to train de-biased representations. The method is tested on NLI, Fact checking and paragraph identification.\n", "summary_of_the_review": "This paper focuses on an important problem: how to estimate less biased representations rather than just creating de-biased models by re-weighting features in a biased representation. In its current form, I don't feel confident enough to state that the paper advances on its goals. Moreover, performance is on par - or lower - than standard re-weighting approaches. More empirical evidence to support the validity of the method is needed.", "strength_and_weaknesses": "Strengths:\n- *The motivation of the paper is sound*: Recent approaches (such as Deep Feature Reweighting https://arxiv.org/pdf/2204.02937.pdf) show that de-bias models just modify the last output layer, thus re-weighting robust features. It is natural to ask how to make data representations intrinsically less biased.\n- *The paper is well-written*: I found the paper easy to read and self-contained, most of the details are present either in the main text or the appendix.\n\nWeaknesses:\n- *Lack of enough empirical support*:\nThe main hypothesis of this paper is that the guidance model representations are less biased and thus can inform the de-biasing methods into learning representations invariant to the bias features. Right now, I don't feel that this hypothesis is substantiated enough by the experiments. I would first expect convincing evidence supporting the fact that the bias is less extractable from the guidance model. An analysis of bias extractability only comes in Section 6.1, Figure 3, but it is only applied to the full regularized model and not to the \"guidance\" model. I am equally perplexed by Figure 3: the model that *minimizes* the similarity w.r.t the guidance model seem to have representations that are *less* biased than the model that *maximizes* the similarity w.r.t. the guidance model, which seems against the thesis of the paper? I think that most of the paper should be focused on proving that the representations of the final de-biased model are indeed less biased on several datasets: right now this analysis is confined to one dataset (HANS). It would good to include in this study the simple method RWG where the loss is re-weighted by the size of each group.\n\n- *Results are not strong enough / missing baselines*:\nOverall, the results obtained by the proposed de-biased model are not significantly better than other baselines. This would not be such an issue if the paper was focused on training de-biased representations without caring so much for improved performance (although it would leave to the authors to justify why having less biased representations would ultimately be useful). For example, if we compare the results of the paper with https://arxiv.org/pdf/1911.03861.pdf, we see lower performance on HANS and PAWS (unknown bias mitigation setting). It would also be suitable to include results for \"Correct-n-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations\": this method also operates on the representation space, by aligning representations between groups, thus it has an inherent similarity with the proposed approach.\n\n- *Manual hyper-parameter search*:\nI am a bit uncertain what is the rationale by which the authors chose their hyper-parameters. In Section 3.2, the authors mention \"choosing the threshold c_t is performed manually by plotting the confidence of the bias model over the training set\". I am not sure what is the rationale for choosing the threshold c_t, and definitely think that an automatic method to do so should be described and analyzed.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "lgrTe4MOHkJ", "reviewer": "Reviewer_73u6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a method for debiasing internal model components, based on similarity-regularization. This is meant to encourage the internal representations to be either similar to representations of an _unbiased_ model, or dissimilar from representations of a _biased_ model. The authors apply this regularization to either the model representations or its gradients. The paper evaluates the approach on three tasks---natural language inference, fact checking, and paraphrase identification---and demonstrates that it improves performance on one out-of-distribution dataset for each task while incurring little degradation in in-distribution performance.", "summary_of_the_review": "The paper is very confusing and lacks structure. Statements are often made without providing evidence to back them up. The paper relies on several assumptions but there is nothing stated as to why these assumptions might be valid. It is not clear what the results table shows (mean and standard deviation? mean and standard error?) but it's not clear what the source of randomness is (mean over what, different samples of training data or different seeds of a model or both?) or if the differences in performance are statistically significant. The paper claims that the proposed method leads to better OOD performance, but better performance on one OOD dataset does not imply better OOD performance overall. In related work, a whole line of work on debiasing methods is missing.", "strength_and_weaknesses": "- The paper is very confusing. Exposition lacks structure, claims are made without providing evidence to back them up, and there are some hand wavy arguments in this paper. Here are several examples of these issues:\n  - \"While improving out of distribution (OOD) performance, it was recently observed that the internal representations of the presumably debiased models are actually more, rather than less biased\": Biased to what? Bias is a statistical term that's in reference to something specific. \n  - The paper relies on having a _biased_ model and an _unbiased_ model, and what those mean doesn't become clear until Section 3.2\n  - It is not clear to me how you train a biased model for the unknown biases. You mention you use TinyBert but that's it.\n  - You make a point in Section 3.2 that excluding samples on which a _biased_ model is correct and confident would result in an unbiased dataset. That makes the assumptions that (i) instances where your biased model is correct and confident are actually instances which contain confounding which is not always true (see [1], but even other than that, they could just be correct and confident for the right reason), and (ii) your biased model captured all instances that contain confounding. There's nothing stated as to why these assumptions might be valid.\n  - \"Choosing the threshold $c_t$ is performed manually by plotting the confidence of the bias model over the training set.\" Yet I can't tell how Figure 3 tells you that $c_t$ has to be 0.8 and not say 0.7 or 0.9 or something.\n  - \"Appendix A.4) shows that regularizing only $D^B$ results in better OOD performance, supporting our intuition.\" Did you use same hyperparameters when just regularizing $D^B$ versus when regularizing all examples or different? How were they chosen?\n  - \"We repeat some of the experiments using DeBERTa to verify our model is not specific to BERT.\" How did you select which experiments would be repeated and which not?\n  - The results tables show means and error terms but its not clear: (i) are these standard error terms or standard deviation?; (ii) what is the source of randomness?; (iii) are differences in performance statistically significant?\n- The paper claims that the proposed method leads to better OOD performance. Better performance on one OOD dataset does not imply better OOD performance. This is like testing the model on one example and claiming 100% accuracy if it correctly classifies that example. But the issue with that is that if the second example is incorrectly classified, it's accuracy would suddenly be 50%. Perhaps this model did better on one OOD dataset, to show that it will likely have better OOD performance overall, you need to evaluate on a battery of OOD datasets. A sound claim would be that it leads to better performance on the specific datasets (which is good).\n- In related work, a whole line of work on debiasing methods is missing. Some relevant papers below:\n  - Victor Veitch, Alexander D'Amour, Steve Yadlowsky, and Jacob Eisenstein. \"Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests.\" arXiv preprint arXiv:2106.00545 (2021).\n  - Zhao Wang and Aron Culotta. \"Robustness to spurious correlations in text classification via automatically generated counterfactuals.\" In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 16, pp. 14024-14031. 2021.\n  - Divyansh Kaushik, Amrith Setlur, Eduard H. Hovy, and Zachary Chase Lipton. \"Explaining the Efficacy of Counterfactually Augmented Data.\" In International Conference on Learning Representations. 2021.\n  - Zhao Wang and Aron Culotta. \"Identifying Spurious Correlations for Robust Text Classification.\" In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 3431-3440. 2020.\n  - Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. \"Learning The Difference That Makes A Difference With Counterfactually-Augmented Data.\" In International Conference on Learning Representations. 2020.\n\n- Errata:\n  - incorrect quotation usage in Line 5 of introduction.\n  - Third to last line of introduction: \"This is different from previous methods\" ---> \"This is different from some previous methods\" (refer to the citations above that do indeed talk about this)\n  - Section 3.1: \"In the case of decreaseing\" ---> \"In the case of decreasing\"\n  - Section 4.1.1, second paragraph: \"As OOD test sets, we use HANS\" ---> \"As OOD test set, we use HANS\"\n  - Section 4.1.3: Thorne et al. should be \\citep and not \\citet\n  - Section 4.1.4, second last line: \"bias.For\" ---> \"bias. For\"\n  - Section 5.2, second last line: \"support out hypothesis\" ---> \"support our hypothesis\"\n\n[1] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. \"Understanding deep learning (still) requires rethinking generalization.\" Communications of the ACM 64, no. 3 (2021): 107-115.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "0WHsZ3hdQoA", "reviewer": "Reviewer_2WEz", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes SimReg, a method that achieves the goal of debiasing by increasing(decreasing) the similarity between the final model and an unbiased(biased) model. They explore two different ways to impose the regularization: on model representation or on the gradients. \nExcept for similarity-based regularization, another key component of this paper is the creation of an unbiased dataset, which is critical for regularization's success. \nBy manipulating internal model representations, SimReg differs from previous methods that downweigh the importance of training samples during training. \nUpon extensive evaluation, SimReg is effective in improving models' OOD performance.\n", "summary_of_the_review": "The submission proposes a novel method that is effective in mitigating spurious correlation.  But there are some limitations and missing related works that have to be addressed before acceptance. ", "strength_and_weaknesses": "Strength:\n\n1. The problem is well-motivated and the paper is easy to follow. \n2. The method is simple yet effective.\n3. The experiments are carried over several different datasets that cover different types of bias, and the results are convincing. \n\nWeakness:\n1.  One limitation of SimReg is that exclude highly-confident examples using a biased model (e.g., tinyBERT) might also filter out some \"easy\" or \"unbiased\" examples, because neural models often get over-confident. \n2. Some related works/baselines are missing. SimReg share many similarity with knowledge distillation (eq 3 is basically a variant of the typical KD loss). So some short discussion about KD would improve the paper and KD loss might be be used as another model variant. Although \"intrinsic debiasing\"(manipulating representations) methods are novel in the area of spurious correlation mitigation, they have been widely used in debiasing hate speech. Despite the differences in terms of areas, they might also need to be included in the related works. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject