IBADR : an Iterative Bias-Aware Dataset Refinement Framework for Debiasing NLU models (2023)

Abstract: As commonly-used methods for debiasing natural language understanding (NLU) models, dataset refinement approaches heavily rely on manual data analysis, and thus maybe unable to cover all the potential biased features. In this paper, we propose IBADR, an Iterative Bias-Aware Dataset Refinement framework, which debiases NLU models without predefining biased features. We maintain an iteratively expanded sample pool. Specifically, at each iteration, we first train a shallow model to quantify the bias degree of samples in the pool. Then, we pair each sample with a bias indicator representing its bias degree, and use these extended samples to train a sample generator. In this way, this generator can effectively learn the correspondence relationship between bias indicators and samples. Furthermore, we employ the generator to produce pseudo samples with fewer biased features by feeding specific bias indicators. Finally, we incorporate the generated pseudo samples into the pool. Experimental results and in-depth analyses on two NLU tasks show that IBADR not only significantly outperforms existing dataset refinement approaches, achieving SOTA, but also is compatible with model-centric methods.

## introduction although neural models have made significant progress in many natural language understanding (nlu) tasks @xcite , recent studies have demonstrated that these models exhibit limited generalization to outof-distribution data and are vulnerable to various types of adversarial attacks @xcite . this is primarily due to their tendency to rely excessively on biased featuresspurious surface patterns that are falsely associated 1 we release our code at http://github.com/ deeplearnxmu/ibadr . with target labels, rather than to capture the underlying semantics. consequently, how to effectively debias neural networks has become a prominent research topic, attracting increasing attention recently. to alleviate this issue, researchers have proposed many methods that can be generally divided into two categories: model-centric mitigation approaches @xcite @xcite @xcite and dataset refinement approaches @xcite @xcite . the former mainly focuses on designing model architectures or training objectives to prevent models from exploiting dataset biases, and the latter aims to adjust dataset distributions to reduce correlations between spurious features and labels. in these two types of methods, dataset refinement approaches possess the advantage of not requiring modifications to the model architecture and training objective, while are also compatible with model-centric approaches. therefore, in this work, we also concentrate on dataset refinement approaches, which employ controllable generation techniques @xcite to refine the data distribution. however, recent studies @xcite @xcite heavily rely on manual data analysis for debiasing models. they either define perturbation rules to generate adversarial examples, or generate pseudo samples and filter out samples with identified biased features. typically, the state-of-the-art (sota) method @xcite first generates a large amount of samples and then applies z-filtering involving a predefined set of biased features to eliminate samples with such features. however, these methods of manually predefining biased features may overlook some potential biased features, thus limiting their generalizability. in this paper, we propose ibadr, an iterative bias-aware dataset refinement framework, which iteratively generates samples to debias nlu models without predefining biased features. under this framework, we create a sample pool initialized by the original training samples, and gradually expand it through multiple iterations. as shown in figure 1 , in each iteration, we first sort and group samples in the pool according to their bias degree, determined by a shallow model trained on a limited set of training samples. next, we concatenate samples in each group with a bias indicator that represents its bias degree. these concatenated samples are then utilized to train a sample generator, which effectively learns the correspondence relationship between bias indicators and samples. afterwards, as implemented in the training phase, we feed a low-degree bias indicator to the sample generator, allowing it to generate pseudo samples with fewer biased features. finally, we add these pseudo samples back into the sample pool and repeat the above process until the maximum number of iterations is reached. apparently, the above iterative process guides the sample generator towards samples with fewer biased features. however, we observe the generated pseudo samples display less diversity when we feed the lowest-degree bias indicator to the sample generator. the underlying reason is that the shallow model consistently assigns a relatively low bias degree to samples with specific patterns, such as the premise directly negating the hypothesis by inserting a word "not". consequently, the sample generator learns these patterns and tends to produce samples containing similar patterns, thereby limiting their diversity. to address this issue, we further explore two strategies to diversify generations. first, instead of always using the lowest-degree bias indicator, we randomly select a low-degree bias indicator. in this way, the sample generator is discouraged from continually creating pseudo samples containing similar patterns, while still ensuring fewer biased features in the pseudo samples. secondly, we dynamically update the shallow model by integrating the newly generated pseudo samples during the iterative generation process. by doing this, we effectively decrease the assignment of the lowestdegree bias indicator to pattern-specific samples, ultimately promoting greater diversity of the generated samples. to summarize, the main contributions of this paper are three-fold: ## the ibadr framework in this section, we give a detailed description of ibadr. under this framework, we first use a limited set of training samples to train a shallow model, which serves to measure the bias degree of samples. then, we iteratively generate pseudo samples with fewer biased features, as illustrated in figure 1 . finally, these pseudo samples are used to debias the nlu models via retraining. ## related work our related work primarily focuses on two categories of methods: model-centric and dataset refinement methods. ## conclusions in this work, we propose ibadr, an iterative dataset refinement framework for debiasing nlu models. under this framework, we train a shallow model to quantify the bias degree of samples, and then iteratively generate pseudo samples with fewer biased features, which can be used to debias the model via retraining. we also incorporated two strategies to enhance the diversity of generated pseudo samples, further improving model performance. on extensive experiments of two tasks, ibadr consistently shows superior performance compared to baseline methods. besides, ibadr can better handle unknown biased features and has good compatibility with larger language models. in the future, we will explore the compatibility of ibadr with other large language models, such as gpt4 @xcite .