Elastic weight consolidation for better bias inoculation (2021)

Abstract: The biases present in training datasets have been shown to affect models for sentence pair classification tasks such as natural language inference (NLI) and fact verification. While fine-tuning models on additional data has been used to mitigate them, a common issue is that of catastrophic forgetting of the original training dataset. In this paper, we show that elastic weight consolidation (EWC) allows fine-tuning of models to mitigate biases while being less susceptible to catastrophic forgetting. In our evaluation on fact verification and NLI stress tests, we show that fine-tuning with EWC dominates standard fine-tuning, yielding models with lower levels of forgetting on the original (biased) dataset for equivalent gains in accuracy on the fine-tuning (unbiased) dataset.

## introduction a number of recent works have illustrated shortcomings in sentence-pair classification models that are used for natural language inference (nli). these arise from limited or biased training data and the lack of suitable inductive bias in models. @xcite demonstrated that phenomena such as the presence of negation or a high degree of lexical overlap induce misclassifications on models trained on the multinli dataset @xcite . @xcite and @xcite identified biases introduced during dataset construction that were exploited by models to learn associations between the label and only one of the two input sentences without considering the other -known as hypothesis-only bias. such biases also affect fact verification @xcite , typically modeled as a text-pair classification between a claim and evidence retrieved from a trusted source @xcite . @xcite evidence @xcite his debut solo studio album [. to mitigate these undesirable behaviors @xcite fine-tune models with a small targeted number of labeled instances to "inoculate" the models. this can be contrasted to methods such as debiased focal loss (dfl) and product of experts (poe) @xcite which require architectural changes to separately encode and penalize biases. in related work, @xcite delexicalize instances -replacing tokens with placeholders, preventing classifiers from exploiting mutual information between domainspecific noun-phrases and class labels. finally, @xcite re-weight the loss of in-stances during training guided by the mutual information between tokens and the instance labels. each of these methods corresponds to a multiobjective optimization problem where the original dataset accuracy is often sacrificed in favor of accuracy on a different evaluation set. for fine-tuning specifically, the reduction in accuracy on the original task called catastrophic forgetting @xcite as parameters for the original task are overridden during fine-tuning. in this paper, we show that regularizing fine-tuning with elastic weight consolidation @xcite minimizes catastrophic forgetting by penalizing weight updates to parameters crucial to modeling the original dataset. ewc has previously been applied to machine translation, where @xcite and @xcite minimized catastrophic forgetting for models undergoing domain adaption. extending this line of research further, we demonstrate that ewc can be used to mitigate biases present in two sentence-pair classification tasks without architectural changes to the models. we evaluate a number of popular model architectures, trained on multinli and fever, and demonstrate that fine-tuning with ewc mitigates model bias while yielding lower reductions in accuracy on the original dataset compared to standard fine-tuning. on all experiments on the fever dataset, finetuning with symmetric counterfactual data @xcite mitigated hypothesis-only bias, increasing the absolute accuracy of a bert model by approximately 10%. without ewc, accuracy on the original dataset was reduced from 87% to 79%, whereas with ewc catastrophic forgetting was mitigated and accuracy was 82%. in all experiments with ewc, the original task accuracy was significantly higher than fine-tuning without regularization and fine-tuning with l2 regularization. these gains were attained while maintaining similar performance on the fine-tuning data. plotting the pareto frontier, we show that equivalent gains in accuracy can be made with less forgetting of the original dataset. furthermore, we demonstrate that fine-tuning methods can be combined with poe and dfl, yielding improvements on both the original task as well as fine-tuning data used for debiasing. similar patterns were observed when finetuning multinli models with lexical bias evaluation datasets @xcite . ## mitigating biases with fine-tuning fine-tuning broadly refers to approaches where a model is initially trained on one dataset and then further improved by training on another. we refer to these datasets as fine-tuning training and test data as ft-train and ft-test respectively. this technique is commonly used to mitigate model biases @xcite , where the original data, while useful in model training, often contain biases, which are addressed by further training the model on a small set of instances targeting these biases. fine-tuning to mitigate bias, however, can result in model parameters over-adjusting to the instances targeting it, reducing the accuracy on the original task, referred to as catastrophic forgetting @xcite . to ameliorate this issue, one can regularize the parameter updates so that they do not deviate too much from the original training, similar to the intuition behind multi-task training approaches @xcite . elastic weight consolidation @xcite penalizes parameter updates according to the model's sensitivity to changes in these parameters. the model sensitivity is estimated by the fisher information matrix, which describes the model's expected sensitivity to a change in parameters, and near the (local) minimum of the loss function used for training is equivalent to the second-order derivative @xmath0,y)∼d original [∇ 2 log p(y|x; θ)]. when fine-tuning with ewc (which we refer to as ft+ewc), the fisher information is used to elastically scale the cost of updating parameters θ i from the original value θ * i , controlled by the λ hyper-parameter, as follows: for efficiency, we use the empirical fisher @xcite : diagonal elements are approximated through squaring first-order gradients from a sample of instances, recomputed before each epoch. if the fisher information is not used (i.e. f i,@xmath1, eq. 1 is equivalent to l2 regularization (which we refer to as ft+l2). ## experimental setup we assess the application of ewc to minimize catastrophic forgetting when mitigating model biases in the context of two sentence-pair classi-fication tasks: fact verification and natural language inference. we compare the untreated model (original), fine-tuning (ft), ft+ewc, ft+l2, and merging instances from the ft-train dataset when training (merged). each model is first trained using the original dataset and splits from the respective task, using the allennlp implementations @xcite with default hyper-parameters and tokenize with spacy or pretrained transformer tokenizers. we train five random initializations of each model, reporting the mean accuracy, standard deviation, and p-value with an unpaired t-test. for fine-tuning, the learning rate, regularization strength λ, and number of epochs are selected through 5-fold crossvalidation on the ft-train data, selecting the model with the highest ft-train accuracy. 30 hyper parameter choices were evaluated with grid search over 10 choices for regularization strength between 10 6 and 10 8 and 3 choices of learning rates in {2 • 10 -6 , 4 • 10 -6 , 6 • 10 -6 } for transformer models and {2 • 10 - foot_3 , 4 • 10 -4 , 6 • 10 -4 } for esim models. we trained the models for a max of 8 epochs. for the transformer-based models, the highest cross validation accuracy on the ft-train dataset was achieved with @xmath2, λ = 10 7 and 6 epochs. for the esim-based models, the highest ft-train accuracy was achieved with @xmath3, λ = 10 7 and 5 epochs. full hyperparameter choices are in appendix d. mitigating hypothesis only bias in fact verification: the fever foot_0 task @xcite is to predict whether short factoid sentences called claims are supported or refuted against evidence (in the form of sentences from wikipedia) or whether there is not enough info (nei). when training the models, the nei instances by definition don't have evidence: we sample negative instances for these with random sentences from the wikipedia page closest to the claim using tf•idf. this preprocessing is the same as @xcite . @xcite identified a bias where the label for some claims can be predicted without the need for evidence. to evaluate this bias, they released foot_1 a set of 1420 symmetric counterfactual instances where each claim is supported by one wikipedia passage and refuted by another (approximately 1% of the fever dataset). this is mitigating the claim-only bias by reducing the mutual information between claims and labels. the availability of counterfactual data meant that it is possible to experiment with fine-tuning as a mitigation strategy, using the published dev and test data as ft-train and ft-test respectively. following schuster et al. (2019) 's evaluation, we train two esim (enhanced lstm) variants @xcite , and a bert @xcite transformer. we also evaluate roberta @xcite , as it has been shown to be more robust to adversarial testing @xcite . mitigating model limitations in nli stress tests: the multinli foot_2 task (williams et al., 2018) requires systems to predict whether a hypothesis is entailed by a premise. @xcite identify limitations of models trained on this dataset where 6 phenomena such as lexical overlap, numerical reasoning and presence of antonyms were evaluated with 'stress-tests'. in this paper, we report on antonyms and numerical reasoning as these stress-tests exhibited catastrophic forgetting when used to fine-tune models @xcite . to this end, we do not evaluate on hans @xcite , as high accuracies can be attained without forgetting. like @xcite , we mitigate these biases through fine-tuning both an esim @xcite model on stress-test data 4 . each contains a small number of procedurally generated instances (between 1500-9800) that specifically target one of these phenomena. we evaluate ft and ft+ewc using the same methodology, controlling the number of instances, sampled at random, in ft-train and report the change in accuracy on the ft-test and original test sets. ## conclusions fine-tuning can be used to mitigate model bias but has the risk that the model catastrophically forgets the data it was originally trained on. incorporating elastic weight consolidation (ewc) when finetuning minimizes catastrophic forgetting, yielding higher accuracy on the original task. we show this holds for both the nli stress-tests, as well debiasing fact-verification systems @xcite through fine-tuning.