IBADR: an Iterative Bias-Aware Dataset Refinement Framework for Debiasing NLU models
Authors: Xiaoyue Wang
Xin Liu
Lijie Wang
Yaoxiang Wang
Jinsong Su
Hua Wu
Venue: EMNLP 2023 Main (2023)
Keywords: debiased models; dataset refinement; spurious correlation

Abstract: As commonly-used methods for debiasing natural language understanding (NLU) models, dataset refinement approaches heavily rely on manual data analysis, and thus maybe unable to cover all the potential biased features. In this paper, we propose IBADR, an Iterative Bias-Aware Dataset Refinement framework, which debiases NLU models without predefining biased features. We maintain an iteratively expanded sample pool. Specifically, at each iteration, we first train a shallow model to quantify the bias degree of samples in the pool. Then, we pair each sample with a bias indicator representing its bias degree, and use these extended samples to train a sample generator. In this way, this generator can effectively learn the correspondence relationship between bias indicators and samples. Furthermore, we employ the generator to produce pseudo samples with fewer biased features by feeding specific bias indicators. Finally,
we incorporate the generated pseudo samples into the pool. Experimental results and in-depth analyses on two NLU tasks show that IBADR not only significantly outperforms existing dataset refinement approaches, achieving SOTA, but also is compatible with model-centric methods.

IBADR: an Iterative Bias-Aware Dataset Refinement Framework for Debiasing NLU models
Xiaoyue Wang" Sang Xin Liu Lijie Wang", Yaoxiang Wang" Jinsong Su and Hua Wu Schooo of Informatics, Xiamen University, Xiamen 361005, China, BBaidu Inc., Beijing 100085, China,
"Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China (xiaoyuewang, liuxin]ēstu, xmu. edu. cn, jssuêxmu. eesu@xmuuedu.cm edu.
Abstract
with target labels, rather than to capture the under- lying semantics. Consequently, how to effectively debias neural networks has become a prominent research topic, attracting increasing attention re- cently.
As commonly-used methods for debiasing nat- ural language understanding (NLU) models, dataset refinement approaches heavily rely on manual data analysis, and thus maybe unable cover all the potential biased features. In this paper, propose IBADR, an Iterative Bias- Aware Dataset Refinement framework, which debiases NLU models without predefining bi- ased features. We maintain an iteratively ex- panded sample pool. Specifically, at each itera- tion, first train shallow model quantify the bias degree of samples in the pool. Then, we pair each sample with bias indicator repre- senting its bias degree, and use these extended samples to train a sample generator. In this way, this generator can effectively learn the corre- spondence relationship between bias indicators and samples. Furthermore, we employ the gen- erator produce pseudo samples with fewer biased features by feeding specific bias indi- cators. Finally, we incorporate the generated pseudo samples into the pool. Experimental re- sults and in-depth analyses on two NLU tasks show that IBADR not only significantly outper- forms existing dataset refinement approaches, achieving SOTA, but also is compatible with model-centric methods."
To alleviate this issue, researchers have pro- posed many methods that can be generally divided into two categories: model-centric miligation ap- proaches (Clark et al., 2019; Stacey et al., 2020; Utama al., 2020a; Karimi Mahabadi et al., 2020; Du al., 2021) and dataset refinement approaches (Lee al., 2021; Wu al., 2022; Ross al., 2022).
The former mainly focuses on designing model ar- chitectures or training objectives to prevent models from exploiting dataset biases, and the latter aims to adjust dataset distributions to reduce correlations between spurious features and labels. In these two types of methods, dataset refinement approaches possess the advantage of not requiring modifica- tions the model architecture and training objec- tive, while are also compatible with model-centric approaches. Therefore, in this work, we also con- centrate on dataset refinement approaches, which employ controllable generation techniques (Zhou et al., 2020; Hu et al., 2022) to refine the data dis- tribution. However, recent studies (Lee et al., 2021; Wu et al., 2022: Ross al., 2022) heavily rely on manual data analysis for debiasing models. They either define perturbation rules to generate adver- sarial examples, or generate pseudo samples and filter out samples with identified biased features. Typically, the statc-of-the-art (SOTA) method (Wu et al., 2022) first generates large amount of sam- ples and then applies z-filtering involving prede- fined set of biased features to eliminate samples with such features. However, these methods of manually predefining biased features may overlook some potential biased features, thus limiting their generalizability.
1 Introduction
Although neural models have made significant progress in many natural language understanding (NLU) tasks (Bowman et al., 2015; Gururangan al., 2018), recent studies have demonstrated that these models exhibit limited generalization to out- of-distribution data and are vulnerable to various types of adversarial attacks (Dasgupta et al., 2018; McCoy et al., 2019). This is primarily due to their tendency to rely excessively on biased features- spurious surface patterns that are falsely associated
'We release our code at http://githhu http://ggittbbbcooo com/ DeepLearnXMU/IN IBADR
In this paper, we propose IBADR, an Iterative Bias- Aware Dataset Refinement framework, which
"These authors contributed equally to this work.
"Corresponding author.

---

iteratively generates samples debias NLU mod- els without predefining biased features. Under this framework, we create sample pool initialized by the original training samples, and gradually expand it through multiple iterations. As shown Figure I, in each iteration, we first sort and group samples in the pool according to their bias degree, deter- mined by shallow model trained a limited set of training samples. Next, we concatenate samples in each group with bias indicator that represents its bias degree. These concatenated samples are then utilized to train a sample generator, which effectively learns the correspondence relationship between bias indicators and samples. Afterwards, implemented in the training phase, we feed a low-degree bias indicator to the sample generator, allowing it to generate pseudo samples with fewer biased features. Finally, add these pseudo sam- ples back into the sample pool and repeat the above process until the maximum number of iterations reached.
We propose a dataset refinement framework designed iteratively generate pscudo sam- ples without prior analysis of biased features.
We present two strategies to enhance the di- versity of the pseudo samples, which further boost the performance of NLU models.
To verify the effectiveness and generality of IBADR, we conduct experiments two NLU tasks. The experimental results show that IBADR achieves SOTA performance.
The IBADR Framework
In this section, we give detailed description of IBADR. Under this framework, we first use lim- ited set of training samples to train shallow model, which serves to measure the bias degree sam- ples. Then, we iteratively generate pseudo samples with fewer biased features, as illustrated in Figure 1. Finally, these pseudo samples are used to debias the NLU models via retraining.
Apparently, the above iterative process guides the sample generator towards samples with fewer biased features. However, observe the gener- ated pseudo samples display less diversity when we feed the lowest-degree bias indicator to the sam- ple generator. The underlying reason is that the shallow model consistently assigns a relatively low bias degree to samples with specific patterns, such as the premise directly negating the hypothesis by inserting word "not". Consequently, the sample generator learns these patterns and tends to pro- duce samples containing similar patterns, thereby limiting their diversity.
2.1 Training a Shallow Model to Measure the Bias Degree of Samples
As investigated in (Utama et al., 2020b), shallow model trained on a small portion training data tends to overfit on biased features, thus is highly confident on the samples that contain biased fea- tures. Motivated by this, we randomly select some training samples to train a shallow model, denoted as 0s, for measuring the bias degree of samples.
Let (ar(), () denote a training sample for NLU tasks, where ( is the golden label the input aff), we directly use the model confi- dence p(y p(y(0) 12(0; 0s) to quantify the bias degree of (a(t) (()... Apparently, if (1:(0)) 8.) + 1, (x(0), y()) is more likely to be biased one.
To address this issue, we further explore two strategies to diversify generations. First, instead of always using the lowest-degree bias indicator, we randomly select low-degree bias indicator. In this way, the sample generator is discouraged from continually creating pseudo samples contain- ing similar patterns, while still ensuring fewer bi- ased features in the pseudo samples. Secondly, we dynamically update the shallow model by integrat- ing the newly generated pseudo samples during the iterative generation process. By doing this, we effectively decrease the assignment of the lowest- degree bias indicator to pattern-specine samples, ultimately promoting greater diversity of the gener- ated samples.
Back our framework, our primary objective is to generate samples with a low bias degree, which can be used to reduce spurious correlations via adjusting dataset distributions.
2.2 Iterative Pseudo Sample Generation
The overview of the iterative sample generation process is shown in Figure 1. During this process, introduce a sample generator Og to iteratively generate pseudo samples, which are added into a sample pool S. Specifically, we initialize the sam- ple pool with the original training samples, the sample generator 6g with generative pretrained
To summarize, the main contributions of this paper are three-fold:

---

Label
Semeences
Riss Indiizerr &
Eabel
Semtumaas
Sentences
Label
Label
Suntennze
Label
Sentenees
Label
Sentences
Pseudo Samples
Sample Pool
Niter* x
Somple Generator
Biss Indicator
Lafiei
Settieiees
Shallow
Bias Indicator by
Labrl
Scitences
By
Sentemcen
Samples w/ Bias Indicators
Figure 1: Overview the iterative sample generation process, which consists of four key stages: Setting bias indicator; Finetuming sample generator; @ Generating pseudo samples; and Expanding sample pool. Through Niter iterations of the above steps, we continuously augmented the sample pool with pseudo samples, which can be effectively employed debias the NLU models.
language model. Then, we iteratively expand S via the following four stage:
Pg(-10. yú 0g). By repeating this sampling process. can obtain set of generated pseudo samples with fewer biased features.
Step 1: Setting Bias Indicators. First, use the mentenmed shallow model to measure the bias degree of each sample in S, as described in Section 2. 1, and sort these samples according to their bias degree and divide them into No groups with equal size. Each group is assigned with bias indicator bn, where 1 b1 represents the legeet bias indicator and bN:: denotes the highest-degree bias indicator.
Expanding Sample Pool. Subse- quently, to ensure the quality of generated pseudo samples, follow Wu et al. (2022) to filter the above generated pseudo samples with model confi- dence lower than threshold e, and incorporate the remaining pseudo samples back into S.
Step 4:
After Nater iterations of the above steps, our sample pool contains not only the original training samples, but also abundant pseudo samples with fewer biased features. Finally, we debias the NLU model via the retraining these samples.
Step 2: Finetuning Sample Generator. Then, we use the samples S to finetune the sample generator 0g via the following loss function:
2.3 Diversifying Pseudo Samples
[5 4g Σ
10 ,0000)0000 (g))
Intuitively, the most direct way is to set the above specified bias indicator 5 to bj. which denotes the lowest bias degree. However, we observe that such generated pscudo samples lack diversity and fail to cover diverse biased features. The reason behind this is that the generated pseudo samples designated with b always follow certain patterns, exhibiting less diversity compared to those assigned with other bias indicators. For example, the premise directly negates the hypothesis using the word "nor". Con- sequently, this results in spurious correlations be- tween b and these certain patterns. Hence, the gen- erator tends to generate samples following these patterns and fails to generate samples that compass broader range biased features.
where ((0) represents the bias indicator assigned for the training sample (z((), ())) Through training with this objective, the generator effectively learn the correspondence relationship between bias indicators and samples. Furthermore, the subse- quent stages, we can specify both the bias indicator and the label to control the generations of pseudo samples.
Step 3: Generating Pseudo Samples. Next, we designate a bias indicator 1 representing a low degree of bias, and then feed with a randomly- selected IL.I label ī into the generator 9g. This process allows us to form a pseudo sample (ž, 7) by sampling & from the generator output distribution
To address this issue, we employ the follow- ing two strategies: (i) Instead of using the lowest-

---

degree bias indicator by. we use randomly- selected low-degree bias indicator: b=b,, where I<rs . and feed into the generator during the iterative generation process. Upon human inspec- tion, we observe that the generated pseudo sam- ples not only become diverse but also still contain relatively few biased features. (ii) During the gen- eration process, update the shallow model 0g using randomly-extracted portion of S at each iteration. This strategy prevents the shallow model from consistently predicting a low bias degree to pscudo samples following previously-appeante pat- terns, thereby enhancing the diversity the pseudo samples.
Dataset
Original
Augmented
MNLI
393K
1.0M
Ny.
SNLI
549K
1.1M
FEVER
243K
607K
Table 1: Sample numbers of the constructed augmented datasets for MNLI, SNLI, and FEVER.
MNLI (Acc.)
Nbc
dev-m
dev-mm
3
85.03
85.63
5
85.31
85.88
7
84.64
84.35
9
84.47
84.19
3 Experiments
Table 2: Results on the development sets of MNLI with different numbers of bias indicators.
3.1 Setup
Tasks and Datasets. We conduct experiments on two NLU tasks: natural language inference and fact verification.
CrossAug (Lee et al., 2021). This method tackles negation bias in the fact verification task through contrastive data augmentation method.
Natural Language Inference (NLI). This task aims to predict the entailment relation- ship between the pair of premise and hypothe- sis. We conduct experiments using the MNLI (Williams al., 2018) and SNLI (Bowman et al., 2015) datasets. As conducted in pre- vious studies (Stacey et al., 2020; Wu et al., 2022; Lyu et al., 2023), in addition to the de- velopment sets, we evaluate IBADR on the corresponding challenge sets for MNLI and SNLI, namely HANS (McCoy et al., 2019) and the Scramble Test (Dasgupta al., 2018), respectively. These two challenge sets are specifically designed to assess whether the model relies on syntactic and word-overlap biases to make predictions.
z-filter (Wu et al., 2022). It first defines set of task-relevant biased features, and then trains generator on existing datasets gen- erate pseudo samples, where pseudo samples with these biased features are filtered. Finally, the remaining samples are used to retrain the model.
Products-of-Bel of-Experts (PoE) (He et al., 2019; Karimi Mahabadi et al., 2020). In an ensemble manner, it trains a debiased model with bias- only one, the predictions of which heavily rely on biased features. By doing so, the debi- ased model is encouraged focus on samples with fewer biased features where the bias-only model performs poorly.
Fact Verification. This task is designed de- termine whether a textual claim is supported or refuted by the provided evidence text. We select FEVER (Thorne et al., 2018) as our original dataset and evaluate the model per- formance on the development set and two challenge sets: FeverSymmetric V1 and V2 (Symm.vl and Symm.v2) (Schuster et al., 201 10),,,, both of which developed to miti- gate biases stemming from claim-only data.
Confidence Regularization (Conf-reg) (Utama et al., 2020a). This method trains a debiased model by increasing the uncertainty of samples with biased features. It first trains bias-only model to quantify the bias degree of each sample, and then scales the output distribution of teacher model based on the bias degree, where the re-scaled distribution can be used to enhance the debiased model.
Example Reweighting (Reweight) (Schuster et al., 2019b). This method aims to reduce the contribution of samples with biased features
Baselines. We compare IBADR with the fol- lowing baselines:

---

MNLI (Acc.)
SNLI (Acc.)
FEVER (Acc.)
Method
dev-m
dev-mm
HANS
dev
Scramble
dev
Symm.vl
Symm.v2
BERT-base
83.87
84.11
61.22
90.61
72.74
87.06
56.53
63.84
Reweight (Schuster al., 2019by*
82.56
66.18
86.44
80.30
83.45
61.56
67.33
Conf-reg (Utama al., 2020a)*
84.60
85.00
69.10
90.56
83.21
85.31
59.69
64.75
DCT (Lyu et al., 2023)
84.19
68.30
90.64
86.40
87.12
63.27
68.45
CrossAug (Lee al., 2021)
85.34
68.90
z-filter (Wu al., 2022)*
82.55
82.70
67.69
88.08
87.87
IBADR
85.171
85.051
71.671
90.32
92.31
89.61
70.01
72.61
Table 3: Results on the development and challenge sets (HANS, Scramble, Symm.vl, Symm.v2) of MNLI, SNLI and FEVER. * means the results are directly cited from previous studies. Note that IBADR outperforms all baselines on the challenge sets, while maintaining comparable or better performance on the development sets. f indicates the results are significantly better than the best comparison method (p < 0.001).
the training loss by assigning them with relatively small weights.
3.2
Effect of Bias Indicator Number No:
The bias indicator number No: our framework is an important the partition granularity of the sample pool. Thus, we gradually vary No, from ment of in each step, and compare the model performance on the development sets of MNLI. As shown in Table 2, when No; is set to a smaller value, such as there is significant decrease in model performance. This because in this case, the bias indicator can only be b1, which re- duces the diversity of generated pseudo samples, as discussed in Section 2.3. Conversely, when No is set to larger values, e.g., or 9, the model per- formance on both development sets also decreases. We hypothesize that this decline occurs because a larger value of N& results in a fine- fine-grained partition of the sample pool, reducing the number of sam- ples corresponding each specific bias indicator. Consequently, this weakens the correspondence relationship between bias indicators and samples, and thus harms the performance of the sample gen- erator. According to these results, we set Nis to 5 for all subsequent experiments.
which determines
Debiasing Contrastive Learning (DCT) (Lyu et al., 2023). It applies contrastive Iearn- ing to mitigate biased latent features by utiliz- ing specifically designed sampling strategy.
with an incre-
Please note that in exception to CrossAug and 2- fillter, which are dataset refinement approaches, all other approaches are model-centric.
Implementation Details. our experiments, we use GPT2-large (Radford et al., 2019) to con- struct the sample generator and the shallow model, respectively. To train the shallow models for dif- ferent NLU tasks, we randomly select 2K, 2K, and 0.5K samples from the original training sets of MNLI, SNLI, and FEVER, individually. These shallow models are trained for 3 epochs with learning rate of 5c-5. When training the sample generator, we set the learning rate to 5e-5, the num- ber of pseudo sample generated per iteration to 200K, and the iteration number Nater to 5. Partic- ularly, we train the sample generator for 3 epochs in the first iteration and for only epoch in the subsequent iterations. When updating the shallow model, we randomly select 2K samples from the sample pool to finetune it.
3.3 Main Results
Table presents the experimental results. Overall, compared with all baselines, IBADR is able to achieve the most significant improvements on the challenge sets (i.e. HANS, Scramble, Symm.vl, and Symm. v2). Specifically, IBADR achieves im- provements of 2.57, 4.44, 1.11 and 4.16 points than the previously-reported best results, respectively. Note that IBADR is effective on both the devel- opment and challenge sets of MNLI and FEVER,
For the NLU models, we train them on the aug- mented datasets of different tasks for 8 epochs us- ing learning rate of lc-5, We employ an early stop strategy during the training process. We con- duct all experiments three times, each time with different random seeds, and report the average re- sults. Sample numbers of the augmented datasets are listed in Table 1.

---

MNLI (Acc.)
SNLI (Acc.)
FEVER (Acc.)
Method
dev-m
dev-mm
HANS
dev
Scramble
dev
Symm.vl
Symm.v2
PoE (Karimi Mahabadi et al., 2020)*
84.58
84.85
66.31
83.69
79.51
82.23
62.19
67.36
P-filte+PPEE (Wu al., 2022)*
85.22
85.72
68.75
IBADR
85.31
85,88
71.67
90.32
92.31
89.61
70.01
72.61
IBADR+POE
85.11
85.07
73.80
90.79
94.76
89.23
70.99
73.10
Table Results on the development and challenge sets of MNI., SNLI, and FEVER. The combination of IBADR with PoE can significantly enhance the model performance the challenge sets, surpassing all baselines.
MNLI (Acc.)
selected low-degree bias indicator by. where 15r5
Method
Noi to generate pseudo samples. 2
dev-m
dev-mm
HANS
IBADR
85.31
85.88
71.68
w/o USM & by b, In this variant, the sample generator uses the bias indicator bj to gener- ate pscudo samples, and the shallow model remains fixed during the generation process.
w/o USM
84.08
83.94
68.90
b,
84.52
84.53
69.45
w/o USM & by
83.76
83.61
67.79
w/o bias indicator
85.21
84.89
63.07
w/o iterative generation
84.54
84.79
66.97
w/o bias indicator. This variant directly uses the samples without the bias indicator to train the sample generator.
Table 5: Ablation study on MNLI.
w/o iterative generation. Instead of generating pseudo samples iteratively, we only utilize the sample generator trained in the first iteration generate pseudo samples.
while other baselines, for example, Reweight and z- z-filter, decline on the development sets.
3.4 Compatibility of IBADR with PoE
To assess the compatibility of IBADR with model- centric debiasing methods, we report the model performance when simultaneously using IBADR and PoE (He et al., 2019), following the setting of Wu et al. (2022).
As shown Table 5, all variants exhibit perfor- mance declines on HANS, indicating the effective- ness of our special designs. Particularly, w/o bias indicator demonstrates the most significant per- formance drop, which is intuitive since the bias indicator can guide the sample generator pro- duce pseudo samples with fewer biased features. Without the bias indicators, the generated pseudo samples will contain undesired biased features, re- sulting in poorer performance on the challenge set.
As shown in Table 4, on the challenge sets, the combination of IBADR and PoE not only yields better results than using PoE or IBADR individ- ually, but also outperforms the combination of z- filter and PoE. Thus, we believe that IBADR has the potential to further enhance the performance of existing model-centric methods.
3.6 Adversarial Tests for Combating Distinct Biases in NLI
3.5 Ablation Study
As shown in (Liu et al., 2020), current debiasing approaches primarily concentrate on addressing known biases, and thus might fail to mitigate un- known biases in NLI tasks. To assess the robustness of NLI models, Liu et al. (2020) introduce sev- eral comprehensive test sets to evaluate the model performance across various types of biased fea- tures, including partial input heuristics (PI), inter- sentence heuristics (IS), logical inference ability (LI), and stress tests (ST). Morcover, they propose several data augmentation strategies to improve
To assess the effects of special designs IBADR, also report the performance of several IBADR
variants on MNLI:
w/o USM. In this variant, we do not Update the Shallow Model during the process of iterative sample generation.
by. The sample generator uses the low- est bias indicator b, rather than the randomly-

---

Method
PI-CD
PI-SP
IS-SD
IS-CS
LI-LI
LI-TS
ST
Avg.
Text Swapping*
71.7
72.8
63.5
67.4
86.3
86.8
66.3
73.6
Sub(synonym)
69.8
72.0
62.4
65.8
85.2
82.8
64.3
71,8
Sub(MLM)*
71.0
72.8
64.4
65,9
85.6
83.3
64.9
72.6
Paraphrasing*
72.1
74.6
66.5
66.4
85.7
83.1
64.8
73.3
BERT-basc(MNLI)
70.3
73.7
53.5
64.8
85.5
81.6
69.2
71.2
IBADR
72.49
76.28
71.67
68.75
91.43
82.46
71.90
76.43
Table 6: Results on the NLI adversarial test benchmark (Liu et al., 2020). We compare IBADR with the data augmentation techniques investigated by Liu al. (2020). BERT-basc(MNLII indicates the BERT-base model trained on the training data of MNLL. Note that training the IBADR's ' augmented datasets significantly improves the model performance on nearly test sets.
the model generalization: (i) Text Swapping: this strategy exchanges the premise and hypothesis in each original training sample; (ii) Sub(synnyym) words in the hypothesis are randomly replaced with synonyms; (iii) Sub(MLM): a masked language model is used to predict the randomly masked words the hypothesis; (iv) Paraphrasing: the hypotheses are paraphrased through back transla- tion.
Test data
Original
z-filter
Ours
dev-m
86.65
85.29
86.83
BERT- BERT-large Jarge
MNLI dev-mm
85.91
84.70
86.62
HANS
65.28
70.35
77.22
SNLI dev
91.74
88.71
91.22
Scramble
81.41
85.77
91.99
Adv.Test
76.66
78.10
78.71
We compare IBADR with the above data aug- mentation strategies in Table 6, Overall, IBADR achieves better results on nearly all test sets. This reveals that IBADR can effectively mitigate vari- ous biases simultaneously. We attribute this success to the fact that IBADR does not rely on predefined biased features, which enables it to better handle unknown biases.
dev-m
87.20
85.93
87.68
RoBERTa-bise base
MNLI dev-mm
87.49
86.25
87.92
HANS
72.19
75.55
76.26
SNLI dev
91.90
88.24
91.80
Scramble
83.07
90.01
93.21
Adv.Test
78.52
79.67
80.03
MNLI dev-m
89.80
88.38
90.11
RoBERTa-large large
MNLI dev-mm
90.03
88.89
90.14
3.7 Generalization on Different Sizes of Pre-trained Language Models
HANS
76.35
77.92
81.74
SNLI dev
92.87
88.30
92.70
To further explore the compatibility of IBADR with different sizes of pre-trained language mod- els, reconduct experiments by individually placing the BERT-base model with BERT-large, RoBERTa-base and RoBERTa-large. We also com- pare IBADR with z-filter, which is the current SOTA data refinement method. The comparisons are performed on the MNLI and SNLI datasets.*
Scramble
85.22
87.43
95,31
Adv.Test
82.22
81.97
82.95
Table 7: Results MNLI and SNLI when training different sizes models on augmented datasets of z- filter and IBADR.
3.34 point, compared to z-filter. These results clearly demonstrate the superior generalizability of IBADR across different sizes of pre-trained lan- guage models.
As presented in Table IBADR consistently outperforms z-ilter on all test datasets. When re- spectively using BERT-large, RoBERTa-base, and RoBERTa-laree as pretrained models, IBADR achieves average improvements 3.29, 1.88, and
3.8 Results on Different Sizes Datasets
Since the code of z-filter only supports MNLI and SNLI, we exclusively compare our results with theirs for these two datascts.
To investigate the robustness of IBADR when the number of original training samples is limited, we

---

Data Size
Test Data
BERT-base
Ours
MNLI
Data Size
dev-m 81.03
84.38
dev-m
dev-mm
HANS
100K
dev-mm
81.55
84.56
OK
83.87
84.11
61.22
HANS
54.25
67.11
10K
84.68
84.88
67.11
dev-m
82.99
84.65
100K
84.73
84.96
68.03
200K
dev-mm
83.41
85.04
300K
85.11
85.13
70.21
HANS
56.17
69.11
600K
85.17
85.05
71.67
dev-m 83.87
85.17
900K
85.24
85.03
71.60
393K
dev-mm
84.11
85.05
z-filter
82.55
82.70
67.69
HANS
61.22
71.67
Table 9: Results on MNLI when using different sizes of augmented datasets.
Table Results MNLI when using different sizes of original training samples.
MNLI
randomly select two subsets from original training samples of MNLI, with sizes 100K and 200K, re- spectively. Afterwards, we employ IBADR to aug- ment these subsets and subsequently retrain NLU
Data
dev-m
dev-mm
HANS
IBADR (GPT-2 Large)
85.17
85.05
71.67
IBADR (LLaMA-7b)
85.64
85.81
72.78
models using the augmented datasets.
Table 10: Results of IBADR with GPT-2 Large and LLaMA-7b on MNLI.
Table 8 presents the results of both the devel- opment and challenge sets. We can observe that IBADR consistently improves model performance across all test sets. Notably, even when using limited number of original training samples, e.g. 100K, the model trained the IBADR augmented dataset outperforms the full-size baseline. This sug- gests that IBADR exhibits remarkable robustness on limited original training samples.
dataset are listed in Table 10. We can observe that the performance IBADR further improved with LLaMA-7b, which indicates IBADR's 's gener- alizability.
Related Work
Our related work primarily focuses on two cate- gories of methods: model-centric and dataset finement methods.
3.9 The Effect of Augmented Dataset Size
To explore the influence of augmented dataset size, we retrain the NLU model on the MNLI dataset with different numbers of augmented sam- ples: 10K, 100K, 300K, 600K, and 900K, respec- tively. As indicated in Table 9, the performance of IBADR consistently improves. Moreover, with just 100K augmented samples, IBADR outper- forms z-iltte across dev-m, dev-mm, and HANS. It's worth mentioning that z-filter utilizes larger set of 360K augmented samples.
4.1 Model-centric Data Debiasing Methods
Numerous previous studies have adopted model- centric approaches to address biases in NLU mod- els. Informed by their deep understanding of task- specific biases, they introduce innovative model architectures and training objectives aimed at pre- venting models from exploiting these biases. For instance, Belinkov et al. (2019) introduce an ad- versarial architecture specifically designed to miti- gate hypothesis-only bias (Gururangan et al., 2018), while Stacey et al. (2020) enhance debiasing by employing multiple adversarial classifiers. Further- more, there exists complementary line of research focus debiasing the model by down-weighting the importance of biased samples during training. The typical methods include example re-weighting (Reweight) (Schuster et al., 2019a), confidence reg- ularization (Conf-reg) (Utama et al., 2020a; Du
3.10 The Compatibility with Advanced Language Models
To ensure a fair comparison with z-fittrr, we em- ploy GPT-2 Large as the sample generator in our main study. In exploring IBADR's compatibil- ity with advanced large language models (LLM), we finetune the L.L.MA-bb model (Touvron et al., 2023) using LORA (Hu et al., 2021) as an alter- native to GPT-2 Large. The results on the MNLI

---

al., 2021), and products-of-experts (POB) (Clark al., 2019; He al., 2019; Karimi Mahabadi et al., 2020). Typically, these methods follow a two-stage paradigm. In the first stage, a bias-only model is trained, either automatically (Utama et al., 2020c; Geirhos et al., 2020; Sanh et al., 2021) or by lever- aging prior knowledge about the bias (Clark et al.. 2019; He et al., 2019; Belinkov et al., 2019). Then, the second stage, the output of the bias-only model utilized adjust the loss function of the debiased model. Recently, Lyu et al. (2023) pro- pose novel approach using contrastive learning to capture the dynamic influence biases and effec- tively reduce biased features, offering an alterna- tive perspective on addressing bias in NLU models. Wang et al. (2023) observe that lower layers in Transformer models tend capture biased features. They introduce the residual connection to integrate low-layer representations with top-layer ones, thus minimizing biased feature impact on the top layer.
IBADR consistently shows superior performance compared to baseline methods. Besides, IBADR can better handle unknown biased features and has good compatibility with larger language models. In the future, we will explore the compatibility of IBADR with other large language models, such as GPT4 (OpenAI, 2023).
Limitations
The limitations of this framework are the following aspects: (i) Despite filtering the pseudo samples with low model confidence, IBADR might still pro- duce pseudo samples with incorrect labels, which limits the model performance: (ii) We only conduct experiments on NLU tasks, neglecting the explo- ration of its applicability to a wider range of tasks.
Ethics Statement
This paper proposes a dataset refinement frame- work that aims to adjust dataset distributions in order to mitigate data bias. All the datasets used in this paper are publicly available and widely adopted by researchers to test the performance of debiasing frameworks. Additionally, this pa- per does not involve any data collection or release, thus eliminating any privacy concerns. Oveall, this study will not pose any ethical issues.
4.2 Dataset Refinement
Several studies have explored generative data augmentation methods to enhance the model ro- bustness in various domains. Lee et al. (2021) train generator to generate new claims and ev- idence for debiasing fact verification datasets like FEVER. Ross et al. (2022) introduce TAILOR, perturbation method for data augmentation based on some manually defined perturbation strategies. Wu et al. (2022) identify a set of biased features by z-statistics, and then ad- just the distribution of the generated samples by post-hoc filtering to remove the generated samples with biased features.
semantically-controll
Acknowledgements
We would like to thank the anonymous reviewers for their insightful comments and suggestions. This research is supported by National Natural Science Foundation of China (No. 62276219) and Natural Science Foundation of Fujian Province of China (No. 2020J06001).
Unlike these approaches, our framework does not require data analysis to define biased features or manual perturbation rules, and hence achieves better generalizability.
References
5 Conclusions
Yonatan Belinkov, Adam Poliak, Stuart Shieber, Ben- jamin Van Durme, and Alexander Rush. 2019. On adversarial removal of hypothesis-only bias in natural language inference. In SEM 2019.
this work, we propose IBADR, an iterative dataset refinement framework for debiasing NLU models. Under this framework, we train shal- low model to quantify the bias degree of samples, and then iteratively generate pscudo samples with fewer biased features, which can be used to de- bias the model via retraining. We also incorporated two strategies to enhance the diversity of generated pseudo samples, further improving model perfor- mance. On extensive experiments of two tasks,
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. In FMNLP 2015.
Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. 2019. Don't 't take the casy way out: Ensemble based methods for avoiding known dataset biases. In EMNLP 2019.

---

Ishita Dasgupta, Demi Guo, Andreas Stuhlmüller, Samuel Gershman, and Neah D. Goodman. 2018. Evaluating compositionality in sentence embeddings. In CogSci 2018.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. 2019. Language models are unsupervised multitask learners.
Alexis Ross, Tongshuang Wu, Hao Peng, Matthew E. Peters, and Matt Gardner. 2022. Tailor: Generating and perturbing text with semantic controls. In ACI. 2022.
Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Demoncourt, Jiuxiang Gu, Tong Sun, and Xia Hu. 2021. Towards interpreting and mitigating shorteut learning behavior of nlu models. In NAACI2021.
Victor Sanh, Thomas Wolf, Yonatan Belinkov, and Alexander M. Rush. 2021. Learning from others' mistakes: Avoiding dataset biases without modeling them. In ICLR 2021.
Robert Geirhos, Jörn-Herrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. 2020. Shortcut learning in deep neural networks. Nature Machine Intelligence.
Tal Schuster, Darsh Shah, Yun Jie Serene Yeo, Daniel Roberto Filizzola Ortiz, Enrico Santus, and Regina Barzilay. 2019a. Towards dehiasing fact verification models. In EMNLP 2019.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts natural language infer- ence data. In NAACL 2018.
Schuster, Darsh Shah, Yun Serene Yeo, Daniel Roberto Filizzola Ortiz, Enrico Santus, and Regina Barzilay. 2019b. Towards debiasing fact verification models. In EMNLP 2019.
He He, Sheng Zha, Haohan Wang. 2019. Unlearn dataset bias in natural language inference by fitting the residual. In Workshop of Deeplo 2019.
Joe Stacey, Pasquale Minervini, Haim Dubossarsky, Se- bastian Riedel, and Tim Rocktäschel. 2020. Avoiding the Hypothesis-Only Bias Natural Language Infer- ence via Ensemble Adversarial Training, In EMNLP 2020.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adap- tation of large language models. arXiv preprint arXiv:2106.09685.
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and verification. In NAACL 2018
Zhe Hu, Zhiwei Cao, Hou Pong Chan, Jiachen Liu, Xinyan Xiao, Jinsong Su, and Hua Wu. 2022. Con- trollable dialogue generation with disentangled multi- grained style specification and attribute consistency reward. IEEF/ACM Transactions Audio, Speech, and Language Processing.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.13971
Rabeeh Karimi Mahabadi, Yonatan Belinkov, and James Henderson, 2020. End-to-end bias mitigation by modelling biases in corpora. In ACL 2020.
Prasetya Ajie Utama, Nafise Sadat Moosavi, and Iryna Gurevych. 2020a. Mind the trade-off: Debiasing NLU models without degrading the in-distribution performance. In ACL 2020.
Minwoo Lee, Scungpil Won. Juae Kim, Hwanhee Lee, Cheoneum Park, and Kyomin Jung. 2021. Crossaug: A contrastive data augmentation method for debias- ing fact verification models. In CIKM 2021.
Prasetya Ajic Utama, Nafise Sadat Moosavi, and Iryna Gurevych. 2020b. Towards debiasing NLU models from unknown biases. In EMNLP 2020.
Tianyu Liu, Zheng Xin, Xiaoan Ding, Baobao Chang. and Zhifang Sui. 2020. An empirical study model- agnostic debiasing strategies for robust natural lan- guage inference. In CoNLL 2020.
Prasetya Ajie Utama, Nafise Sadat Moosavi, and Iryna Gurevych. 2020c. Towards debiasing NLU models from unknown biases. In EMNLP 2020.
Yougang Lyu, Piji Li, Yechang Yang, Maarten Rijke, Pengjie Ren. Yukun Zhao, Dawei Yin, and Zhaochun Ren. 2023. Feature-leved debiased natural language understanding. AAAI 2023.
Xiaoyue Wang, Lijie Wang, Xin Liu, Suhang Wu, Jin- song Su, and Hua Wu. 2023. A simple yet effec- tive self-debiasing framework for transformer models. arXiv preprint arXiv:2306.0190)
Tom McCoy, Ellic Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactie heuris- tics in natural language inference. In ACI. 2019.
Adina Williams, Nikita Nangia, and Samuel R. Bow- man. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In NAACL 2018.
OpenAl. 2023. GPT-4 technical report. abs/2303.08774.
CoRR,

---

Yuxiang Wu, Matt Gardner, Pontus Stenetorp, and Pradeep Dasigi. 2022. Generating data to mitigate spurious correlations in natural language inference datasets. In ACL 2022.
Chulun Zhou, Liangyu Chen, Jiachen Liu, Xinyan Xiao, Jinsong Su, Sheng Guo, and Hua Wu. 2020. Explor- ing contextual word-level style relevance for unsu- pervised style transfer. In ACL 2020.


Reviews:
[{"review_id": "w7eX3GNuFY", "reviewer": "Reviewer_NJeA", "summary": "", "questions": "Q1) How sensitive the model is on the different number of augmented dataset?\n\nQ2) Do the better auto-regressive models (e.g., GPT-3) can improve the debiasing performance?\n\nQ3) Why do the authors use the same model (i.e., BERT-base) for both the main and shallow models in the experiment of Table 3, which is not consistent to the explanation?\n\nQ4) Did the authors compare the proposed method with existing and simple augmentation techniques (other than CrossAug)? because there is possibility that two time more augmented dataset can just simply mitigate the dataset bias.\n\nQ5) Are there any failure cases that the augmented text are not aligned with the given labels?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reviewing our paper and for the insightful comments. We hope our answers to the 5 questions will address the concerns and clarify the contributions of the paper.\n\n**Q1: How sensitive the model is on the different number of augmented dataset?**\n\nTo explore the influence of augmented dataset size, we retrain the NLU model on the MNLI dataset with different numbers of augmented samples: 10K, 100K, 300K, 600K, and 900K, respectively. The average results of 5 runs of experiments are reported below. Please note that the original MNLI dataset contains 393K samples, and we report the model performance with additional 600K augmented samples in the original paper. From the following table, we can observe that the performance of our model can be always improved.\n\n|                       |   dev-m   |   dev-mm  |    HANS   |\n|-----------------------|:---------:|:---------:|:---------:|\n| 0K (from the paper)   |   83.87   |   84.11   |   61.22   |\n| 10K                   |   84.68   |   84.88   |   67.11   |\n| 100K                  |   84.73   |   84.96   |   68.03   |\n| 300K                  |   85.11   | **85.13** |   70.21   |\n| 600K (from the paper) |   85.17   |   85.05   | **71.67** |\n| 900K                  | **85.24** |   85.03   |   71.60   |\n| z-filter (360K)       |   82.55   |   82.70   |   67.69   |\n\n**Q2: Do the better auto-regressive models (e.g., GPT-3) can improve the debiasing performance?**\n\nTo ensure a fair comparison with z-filter, we also use GPT2-large as the pretrained language model. Besides, since the sample generator is expected to model the co-occurrence relationships of bias indicators, labels, and sentences, we have to finetune the language model. However, finetuning large language models (LLMs) poses challenge due to cost or feasibility issues, especially in cases of commercial LLMs or GPT3 via the OpenAI API. As a workaround, researchers have proposed some finetuning-efficient techniques, such as LORA. Based on this, we explore finetuning llama-7B with the help of LORA, as a replacement for GPT2-large. The results are reported below.\n\n|                  | dev-m | dev-mm | HANS  |\n|------------------|-------|--------|-------|\n| IBADR (GPT-2)    | 85.17 | 85.05  | 71.67 |\n| IBADR (llama-7B) | **85.64** | **85.81**  | **72.78** |\n\n**Q3: Why do the authors use the same model (i.e., BERT-base) for both the main and shallow models in the experiment of Table 3, which is not consistent to the explanation?** \n\nAs mentioned in Lines 149-153, we adopt the commonly-used approach proposed by Utama et al., 2020[1] to train the shallow model. Note that this shallow model shares the same architecture as the main model, and is trained with only a small subset of the training data.\n\n**Q4: Did the authors compare the proposed method with existing and simple augmentation techniques (other than CrossAug)? because there is possibility that two time more augmented dataset can just simply mitigate the dataset bias.**\n\nAs shown in Table 6, we compare IBADR with simple augmentation techniques, including text swapping, paraphrasing and synonym substitution. Experimental results demonstrate the effectiveness of IBADR. Notably, as mentioned in the response to Q1, IBADR can maintains its effectiveness even with only 10K augmented samples.\n\n**Q5: Are there any failure cases that the augmented text are not aligned with the given labels?**\n\nYes, we have investigated the quality of the augmented samples. Specifically, we randomly select 100 samples from the augmented MNLI dataset and engage 3 human annotators to manually classify the relation between sentence pairs. We apply the majority vote to finalize annotations. The accuracy of augmented instances is about 92% based on human-annotated gold labels. We will add these details in our revised version.\n\n**References:**\n\n1.Utama P A, Moosavi N S, Gurevych I. Towards debiasing NLU models from unknown biases. EMNLP 2020.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "As I stated in the contribution parts, the strengths of this paper are as follows:\n\n(i) a new method for the dataset-refinement approach that does not require human-annotated biases. \n\n(ii) the proposed dataset-refinement methods shows better OOD and ID performance compared to model-centric and existing dataset-refinement approaches. \n\n", "reasons_to_reject": "The weakness of this paper is the weak analysis about the sample generator. While utilizing sample generator leads to better performance, the detailed analysis on the sample generators are absent. For example, (i) the qualitative analysis on the generated examples (ii) the effect of the different number of augmented samples. This is crucial because the current version requires at least 2 time (approximately 3 times) more training dataset than the original dataset.", "questions_to_authors": "Q1) How sensitive the model is on the different number of augmented dataset?\n\nQ2) Do the better auto-regressive models (e.g., GPT-3) can improve the debiasing performance?\n\nQ3) Why do the authors use the same model (i.e., BERT-base) for both the main and shallow models in the experiment of Table 3, which is not consistent to the explanation?\n\nQ4) Did the authors compare the proposed method with existing and simple augmentation techniques (other than CrossAug)? because there is possibility that two time more augmented dataset can just simply mitigate the dataset bias.\n\nQ5) Are there any failure cases that the augmented text are not aligned with the given labels?", "paper_topic_and_main_contributions": "This paper considers the dataset biases problem that impedes the model generalization ability to out-of-distribution data. There are two mainstreams to mitigate the detrimental effect of the biased samples, which are model-centric and dataset-refinement approaches. Among them, the proposed method is built on the dataset refinement approach as it does not require modification on model architectures and training objectives. Unlike the previous studies that rely on the human-annotated bias dataset, the proposed method, IBADR, iteratively refines the training dataset such that the remaining dataset only includes the bias-conflicting samples. Specifically, IBADR adopts the shallow model to identify the bias samples and fine-tunes LMs to only generate bias-conflicting samples. The above processes are iteratively repeated, and the final samples are only used to train the main networks. \n\nAs a review's viewpoint, the contribution of this paper are as follows:\n\n(i) introducing the first dataset-refinement approach which does not require human-annotated bias labels. \n\n(ii) better debiasing performance compared to existing two approaches.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "rsOHdVkz4z", "reviewer": "Reviewer_HAyt", "summary": "", "questions": "1.\tWhat is the performance variance across the five trials?\n2.\tHow does performance change with the number of augmented data? And at what point does it outperform existing bias reduction methods?\n3.\tSanh et al., 2020 show that with a small model, it is possible to train a sufficiently biased model even using full data. But why did you train a shallow model with some datasets instead of the entire dataset?\n4.\tGPT-2 is a very old model. Why did you use that model despite the fact that various generative models with better performances was recently released?\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reviewing our paper and for the insightful comments. We hope our answers to the 4 questions will address the concerns and clarify the contributions of the paper.\n\n**Q1: What is the performance variance across the five trials?**\n\nA1: Taking the MNLI dataset as an example, we try five different random seeds for the generator, shallow model, and BERT-base. Throughout the experimentation process, we observe fluctuations in the shallow model's performance on the MNLI dev-m, spanning a range of 45% to 51%. However, we also discover that these fluctuations in the shallow model do not affect the quality of samples generated by IBADR. The variances in its performance on dev-m, dev-mm, and HANS are 0.05, 0.03, and 0.17 respectively.\n\n**Q2: How does performance change with the number of augmented data? And at what point does it outperform existing bias reduction methods?**\n\nA2:  To explore the influence of augmented dataset size, we retrain the NLU model on the MNLI dataset with different numbers of augmented samples: 10K, 100K, 300K, 600K, and 900K, respectively. The average results of 5 runs experiments are reported below. Please note that the original MNLI dataset contains 393K samples, and we report the model performance with additional 600K augmented samples in the original paper. From the following table, we can observe that the performance of our model can be always improved.\n\n|                       |   dev-m   |   dev-mm  |    HANS   |\n|-----------------------|:---------:|:---------:|:---------:|\n| 0K (from the paper)   |   83.87   |   84.11   |   61.22   |\n| 10K                   |   84.68   |   84.88   |   67.11   |\n| 100K                  |   84.73   |   84.96   |   68.03   |\n| 300K                  |   85.11   | **85.13** |   70.21   |\n| 600K (from the paper) |   85.17   |   85.05   | **71.67** |\n| 900K                  | **85.24** |   85.03   |   71.60   |\n| z-filter (360K)       |   82.55   |   82.70   |   67.69   |\n\nAs shown in the above table, IBADR can surpass z-filter across dev-m, dev-mm, and HANS with only 100K augmented samples. It is noteworthy that z-filter[1] involves a larger set of 360K augmented samples.\n\n**Q3: Sanh et al., 2020 show that with a small model, it is possible to train a sufficiently biased model even using full data. But why did you train a shallow model with some datasets instead of the entire dataset?**\n\nA3: Previous studies have highlighted two primary approaches to construct biased models without manual intervention: 1. employing a reduced subset of dataset to train a biased model with the same architecture as the original one (Utama et al., 2020)[2]; 2. immobilizing a significant portion of parameters and training the model using the complete dataset (Sanh et al., 2020). Importantly, there is currently no work discussing the pros and cons of these two methods, which is also not our emphasis. For simplicity, we directly adopt the Utama's approach.\n\nTo investigate the generality of our framework, we also follow Sanh et al., (2020) to retrain the shallow model and then reconduct the experiment on MNLI. The average results of 5 runs experiments are reported below.\n\n|                            | dev-m     | dev-mm    | HANS      |\n|----------------------------|-----------|-----------|-----------|\n| BERT-base                  | 83.87     | 84.11     | 61.22     |\n| z-filter                   | 82.55     | 82.70     | 67.69     |\n| IBADR (Utama et al., 2020) | 85.17     | **85.05** | **71.67** |\n| IBADR (Sanh et al., 2020)  | **85.31** | 84.94     | 71.41     |\n\n**Q4: GPT-2 is a very old model. Why did you use that model despite the fact that various generative models with better performances was recently released?**\n\nA4: To ensure a fair comparison with z-filter, we also use GPT2-large as the pretrained language model. Besides, since the sample generator is expected to model the co-occurrence relationships of bias indicators, labels, and sentences, we have to finetune the language model. However, finetuning large language models (LLMs) poses challenge due to cost or feasibility issues, especially in cases of commercial LLMs or GPT3 via the OpenAI API. As a workaround, researchers have proposed some finetuning-efficient techniques, such as LORA. Based on this, we explore finetuning llama-7B with the help of LORA, as a replacement for GPT2-large. The results are reported below.\n\n|                  | dev-m | dev-mm | HANS  |\n|------------------|-------|--------|-------|\n| IBADR (GPT-2)    | 85.17 | 85.05  | 71.67 |\n| IBADR (llama-7B) | **85.64** | **85.81**  | **72.78** |\n\n**References:**\n\n1.Wu Y, Gardner M, Stenetorp P, et al. Generating data to mitigate spurious correlations in natural language inference datasets. ACL 2022.  \n\n2.Utama P A, Moosavi N S, Gurevych I. Towards debiasing NLU models from unknown biases. EMNLP 2020.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper addresses an important problem of existing dataset refinement debiasing methods, such as the human cost problem, by an iterative dataset augmentation approach. This approach is simple and intuitive.\n\nExperiment results in the paper show that the proposed method, IBADR, significantly improves the out-of-distribution performance of the pre-trained language model on various debiasing benchmarks. In particular, unlike most previous studies, IBADR improves both in-distribution performance and OOD performance simultaneously. \n\nThough some moderately unclear claims exist, such as on lines 55-61, the paper is generally clear and well-written. \n", "reasons_to_reject": "Since they use a small subset of training examples for biasing a shallow model, which is highly sensitive to sampled examples, the authors need to provide some analysis and experimental results on the robustness of their framework.\n\nThe authors use much more augmented data than the original data to achieve the performance, thus they need to show how much additional data is needed to achieve comparable performance with existing debiasing methods.\n", "questions_to_authors": "1.\tWhat is the performance variance across the five trials?\n2.\tHow does performance change with the number of augmented data? And at what point does it outperform existing bias reduction methods?\n3.\tSanh et al., 2020 show that with a small model, it is possible to train a sufficiently biased model even using full data. But why did you train a shallow model with some datasets instead of the entire dataset?\n4.\tGPT-2 is a very old model. Why did you use that model despite the fact that various generative models with better performances was recently released?\n", "paper_topic_and_main_contributions": "This paper proposes a new dataset refinement framework, which does not require human supervision about biases in the dataset, to debias NLU models. The main contributions of the paper are in three folds: 1) reducing the costs of human annotating, 2) may capturing overlooked biases, and 3) outperforming state-of-the-art by a large margin.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "T5kI6YI7aj", "reviewer": "Reviewer_XyJx", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reviewing our paper and for the insightful comments. We hope our answers to the 3 reject reasons will address the concerns and clarify the contributions of the paper.\n\n**R1: The authors claim that the proposed method significantly outperforms the other baselines. However, no significant tests are performed.**\n\nA1: When comparing IBADR and z-filter, we conduct the significance tests on HANS, MNLI-m hard and MNLI-mm hard, where the corresponding p-values are 0.0064, 0.0017, and 0.0031 respectively. This indicates that our framework is significantly better than z-filter across all three test sets. We will add this in our revised version.\n\n**R2: The experimental results on MNLI-hard and SNLI-hard (two widely-used OOD datasets for NLI) are lacking.**\n\nA2: Following the experimental setup of Utama et al., (2020)[1] and Lyu et al., (2023)[2], where HANS and scramble test sets are used to assess the debiasing effects of models on the MNLI and SNLI datasets. In response to your recommendation, we also conduct experiments on MNLI-hard and SNLI-hard. The average results across five runs with different random seeds are reported below. Obviously, our framework is also effective on these two datasets. Meanwhile, we conduct significance tests to investigate whether the performance improvement of IBADR over z-filter is significant.\n\n|              | SNLI-hard | MNLI-m hard | MNLI-mm hard |\n|--------------|:---------:|:-----------:|:------------:|\n| BERT-base    |   80.34   |    75.88    |     75.75    |\n| z-filter     |   82.82   |    79.19    |     78.51    |\n| IBADR        |   **83.43**   |    **79.62**    |     **80.01**    |\n| $p$-values    |   0.0052  |   0.0017   |     0.0031   |\n\n**R3: The computational cost is high due to the iterative nature of the refinement process. Each iteration involves expanding the sample pool and retraining the generator, contributing to the overall expense of the framework.**\n\nA3: Similar to prevalent data augmentation methods (e.g., Wu et al., 2022[3]; Lee et al., 2021[4]), the data augmentation process in IBADR is performed one-time in an offline manner. Once the augmented data is generated, it can serve as a valuable resource for training diverse downstream models.\n\n**References:**  \n1.Utama P A, Moosavi N S, Gurevych I. Towards debiasing NLU models from unknown biases. EMNLP 2020.   \n2.Lyu Y, Li P, Yang Y, et al. Feature-level debiased natural language understanding. AAAI 2023.   \n3.Wu Y, Gardner M, Stenetorp P, et al. Generating data to mitigate spurious correlations in natural language inference datasets. ACL 2022.   \n4.Lee M, Won S, Kim J, et al. Crossaug: A contrastive data augmentation method for debiasing fact verification models. CIKM 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed data refinement framework is simple, and does not rely on pre-defined types of biases.\n- Good experimental results.\n- The paper is well-organized and reader-friendly.", "reasons_to_reject": "- The authors claim that the proposed method significantly outperforms the other baselines. However, no significant tests are performed.\n- The experimental results on MNLI-hard and SNLI-hard (two widely-used OOD datasets for NLI) are lacking.\n- The computational cost is high due to the iterative nature of the refinement process. Each iteration involves expanding the sample pool and retraining the generator, contributing to the overall expense of the framework.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper concentrates on the dataset refinement methods to debias the NLU models and improve their generalization under out-of-distribution (OOD) settings. It proposes an iterative data refinement framework for NLU. In each iteration, a shallow model is trained on a subset of the training data to measure the bias degree without manually-defined biased features. Then a generator, trained on samples paired with their bias degree, is instructed to generate pseudo samples with fewer biased features, expanding the training sets.\n\n-------------Post-rebuttal---------------\n\nI have read the author's response, and I think they have addressed my concerns. Therefore, I will improve my score.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents the Iterative Bias-Aware Dataset Refinement (IBADR) framework, a new dataset-refinement method for debiasing NLU models. The authors introduce an iterative process involving a shallow model and a sample generator. The shallow model identifies biases in samples while the generator produces pseudo samples with fewer biased features in each iteration. Experimental results show that IBADR not only outperforms existing dataset refinement methods but also surpasses performance from model-centric approaches. However, potential weaknesses in this study include substantial data augmentation requirements and lack of comprehensive analysis surrounding the implemented sample generator.

The key novelty of this approach is its independence from predefined biased features or manual data analysis, which contributes to overcoming crucial limitations in contemporary dataset refinement approaches. Comprising a shallow model and a sample generator, the IBADR framework effectively identifies bias degrees of samples and generates pseudo samples with fewer biased features iteratively.

Some reviewers pointed out that the paper falls short in providing an exhaustive analysis on the role and impact of the sample generator. They also pointed out the lack of significance testing, computational costs and the heavy reliance on augmented samples raising questions over scalability. The authors have offered adequate analysis in author response, which has been acknowledged by the reviewers to be satisfactory.
Decision: Accept-Main