Debiasing Masks: A New Framework for Shortcut Mitigation in NLU (2022)

Abstract: Debiasing language models from unwanted behaviors in Natural Language Understanding (NLU) tasks is a topic with rapidly increasing interest in the NLP community. Spurious statistical correlations in the data allow models to perform shortcuts and avoid uncovering more advanced and desirable linguistic features.A multitude of effective debiasing approaches has been proposed, but flexibility remains a major issue. For the most part, models must be retrained to find a new set of weights with debiased behavior.We propose a new debiasing method in which we identify debiased pruning masks that can be applied to a finetuned model. This enables the selective and conditional application of debiasing behaviors.We assume that bias is caused by a certain subset of weights in the network; our method is, in essence, a mask search to identify and remove biased weights.Our masks show equivalent or superior performance to the standard counterparts, while offering important benefits.Pruning masks can be stored with high efficiency in memory, and it becomes possible to switch among several debiasing behaviors (or revert back to the original biased model) at inference time. Finally, it opens the doors to further research on how biases are acquired by studying the generated masks. For example, we observed that the early layers and attention heads were pruned more aggressively, possibly hinting towards the location in which biases may be encoded.

## introduction the issue of spurious correlations in natural language understanding datasets has been extensively studied in recent years @xcite @xcite . in the mnli dataset @xcite , negation words such as "not" are unintended hints for the contradiction label @xcite , while a high word overlap between the premise and the hypothesis often correlates with entailment @xcite . transformer layers embedding classifier embedding classifier transformer layers embedding classifier biased behavior debiased behavior 1 (e.g. lexical overlap) debiased behavior 2 (e.g. negation words) transformer layers 1. freeze weights 2. assign debiasing loss 3. train scores (find mask) we mask individual weights in all modules, excluding embedding and classifier. different debiasing targets lead to different masks. they can be applied to the same base model. figure 1 : we find masks that remove bias from a finetuned model by using frozen-weight movement pruning. a common way to prevent the acquisition of biases (in this paper defined as unintended shortcut behaviors) is to adopt a debiasing loss function, such as product-of-experts @xcite , to discourage learning shortcuts through the use of the annotated level of bias in each sample. because manual bias annotation is difficult and expensive, the predictions of a biased model are used as bias annotations @xcite . commonly, models must be retrained to achieve debiased behavior; this limits our ability to target different biases separately, as well as choose varied levels of debiasing strength (which impacts the in-vs. out-ofdistribution trade-off). we propose a new debiasing framework that focuses on removing bias from an existing model, instead of the now-common approach of re-training from scratch. our approach is depicted in figure 1 . we find a pruning mask that zeroes out the weights that cause biased outputs, producing the desired effect without altering the original model. this approach offers several clear advantages. first, pruning masks can be stored very efficiently, only occupying a fraction of the original model size. this enables creating multiple masks for varied debiasing behaviors. secondly, masks can be effortlessly set or unset at inference time, as opposed to replacing the entire model. this allows for flexible application, as well as easy reversion to the original model when needed. finally, re-framing the debiasing process as a mask search opens the doors towards future analysis directions, helping to more deeply understand how biases are learned, and how they can be eliminated. ## masked debiasing our proposed approach to debiasing takes a unique point of view in the debiasing field by assuming that biased behavior is encoded in specific weights of the network and can be removed without altering the remaining weights. thus, we perform a mask search to identify and remove those weights, experiencing debiased behavior in the resulting model. our approach comes together by combining a debiasing loss, a biased model, and a score-based pruning technique. debiasing loss among the debiasing losses mentioned in section 2.2, none appear to be clearly superior, each offering certain strengths and weaknesses, and striking a balance between in-and outof-distribution performance. we err on the side of simplicity and run our experiments with the focal loss. we follow @xcite for the implementation. bias model to utilize a debiasing loss, we must obtain the predictions of a biased model on the training samples. we utilize weak learners, following the nomenclature introduced by @xcite ; they do not require making any assumptions on the underlying biases. while our method works with any debiasing loss and bias level source, we choose this setting due to its flexibility and adaptability to new scenarios. to date, two major learners have been proposed: undertrained @xcite and underparameterized @xcite learners. we train and extract the predictions of both of them for comparison. undertrained learners are trained by selecting a very small subset of the training data, but keeping the target model's architecture and hyperparameters. this effectively translates into a model that overfits on the selected data subset. features uncovered by the model in this manner are deemed spurious, as it is assumed that more advanced features require exploring larger quantities of data. we used 2000 samples for mnli, and 500 samples for the other datasets. underparameterized learners, on the other hand, restrict the model complexity by reducing layer size and count. the expectation is that this underparameterization is restrictive and limits the ability to find complex features. we use bert-tiny @xcite , a bert model with 2 layers and an inner size of 128. frozen movement pruning our approach is similar to zhao et al. (2020); but we implement it by leveraging the framework provided by @xcite , with the addition of weight-freezing. we use unstructured pruning, which means that each individual weight is considered for exclusion. further, we utilize a threshold approach: in each weight tensor we remove (mask out) those weights with an associated score that is lower than the threshold. a regularization term for the scores helps avoid all scores growing larger than the threshold. for further details, we refer the reader to @xcite . as the starting point, we load a model already finetuned on the target task. ## experimental setup we compare our approach against the standard approach of re-training the model with a debiasing loss. our experiments are carried out with bert-base @xcite . ## results we compare our approach against our reproduction of the two weak model approaches, and compile our results in ## mask analysis our pruning approach uses a score threshold, which allows for a dynamic pruning rate across modules. to gain a better understanding on our masks, we study the generated density distribution. specifically, we study the rate at which weights were removed in different network regions. in each layer, we obtain the pruning rate of the attention layer's query, key, value, and output modules, as well as the two linear modules that follow. we plot the results in figure 2 . we make two general observations. first, pruning rates are relatively higher in the early layers. it is known that the last few layers are very task specific @xcite , which likely implies that their modification more directly impacts performance. thus, our masks may be targeting early layers as a means to remove biases without causing a noticeable performance decrease. secondly, the attention modules are more heavily pruned than the linear layers, which could suggest that attention heads play an important role in bias encoding. ## conclusion we conclude that debiasing masks are an effective approach to mitigating shortcuts. alongside providing surprisingly well-performing debiased behaviors, masks allow to shift the way we think about debiasing: no longer should biased and unbiased models be treated as two separate models; rather, it becomes possible to "remove" biases by simply eliminating certain weights from the network.