Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Lists (2022)

Abstract: Natural Language Processing (NLP) models risk overfitting to specific terms in the training data, thereby reducing their performance, fairness, and generalizability. E.g., neural hate speech detection models are strongly influenced by identity terms like gay, or women, resulting in false positives, severe unintended bias, and lower performance. Most mitigation techniques use lists of identity terms or samples from the target domain during training. However, this approach requires a-priori knowledge and introduces further bias if important terms are neglected. Instead, we propose a knowledge-free Entropy-based Attention Regularization (EAR) to discourage overfitting to training-specific terms. An additional objective function penalizes tokens with low self-attention entropy. We fine-tune BERT via EAR: the resulting model matches or exceeds state-of-the-art performance for hate speech classification and bias metrics on three benchmark corpora in English and Italian.EAR also reveals overfitting terms, i.e., terms most likely to induce bias, to help identify their effect on the model, task, and predictions.

## introduction online hate speech is growing at a rapid pace, with effects that can result in dangerous criminal acts offline. due to its verbal nature, various natural language processing approaches have been proposed @xcite @xcite @xcite . recently, detection performance has significantly improved with the use of large pre-trained language models based on transformers @xcite , such as bidirectional encoder representations from transformers (bert) @xcite . however, figure 1 : false positive from bert as a hate speech detector. the darker and taller the bar, the higher the overfitting on the term. various methods have been proposed to mitigate and measure (unintended) bias @xcite @xcite @xcite . however, all those methods rely on the availability of a set of identity terms. this is a severe limitation, which hinders the generalizability and applicability of hate detection models to real-world contexts. for example, a model designed to reduce the unintended bias on genderrelated terms (such as woman, wife) will not address unintended bias for religious affiliation. so practitioners must decide a-priori "which vulnerable groups are present in our data?" we propose an entropy-based attention regularization (ear) that forces the model to build token representations by attending to a wider context, i.e., consider a larger number of tokens from the rest of the sentence. we measure the attended context as the entropy of the self-attention weight distribution over the input sequence. we use ear as a regularization term in the loss computation to maximize each token's entropy. we apply ear to bert. the resulting model (bert+ear) significantly improves performance on unintended bias mitigation in english and italian. in addition, it requires no a-priori knowledge (e.g., sets of identity terms), making it fairer and more general. the contextualized representations ear induces avoid basing the classification on individual terms and, ultimately, mitigate lexical overfitting and intrinsic bias from pre-trained weights. as a training by-product, ear lets us extract the overfitting terms, i.e., terms accounting for narrower context that most likely induce unintended bias. these terms can highlight possible weaknesses in the model: from the over-sensitivity of pre-trained weights to specific words @xcite @xcite , to overspecialization of training corpora on the keywords used for collecting data @xcite . note that while we show results on bert, ear is applicable to any attention-based architecture. contributions. ear is a novel entropy-based attention regularization method to mitigate unintended bias by reducing lexical overfitting. it is applied to all terms, so it does not need a-priori domain knowledge (e.g, predefined term lists). independent of domain-specific information, ear generalizes better to different languages and contexts compared to similar approaches. attention entropy is used to extract a list of the most likely biased terms. ear code is available at https: //github.com/g8a9/ear . ## entropy-based attention regularization attention was originally designed for aligning target and source sequences in machine translation girl is concentrated on its representation: its entropy is low. attention for you is spread: its entropy is high. @xcite . however, in the transformer architecture @xcite , it has become a means to account for lexical influence and long-range dependencies. it also provides useful information about the importance of a term for the output @xcite @xcite . here, we use the notion of attention entropy, and ear's use of it in bert. note, though, that ear can be used with any attention-based architecture. ## experimental settings in this work, we consider the problem of unintended bias @xcite : "a model contains unintended bias if it performs better for comments containing some particular identity terms than for comments containing others". ## experimental results bert+ear is overall the best debiasing model considering the proposed bias metrics. the only exception is misogyny (en), for which bert+ear has lower auc bnsp and auc bpsn than bert+soc mitigation. the latter's advantage, however, comes with high variability in the results. bert+soc mitigation seems more sensitive to random initialization. the standard deviation over 10 runs is 37%, compared to 13% of bert+ear. figure 4 shows the auc subgroup metric separately by identity term on misogyny @xcite . we compare bert and bert+ear over 10 different initialization runs. ear improves bert across all identity terms most existing models and auc-based metrics for unintended bias focus only on the false positives (i.e., hateful instances wrongly recognized as non-hateful). while correctly recognizing hateful instances is important, we believe that the problem of false negatives is equally important. since bert+ear does not rely on identity term lists, it regularizes terms that impact both the positive and negative class. bert+ear obtains an average decrease of 15.04% in false negative rate compared to bert and bert+soc mitigation. indeed, the performance difference between bert+ear vs. bert and bert+soc is mainly due to nonhateful instances (∼95% of the time). reducing the impact of overfitting terms like f*ck and p*ssy in misogyny (en) causes bert+ear to consider a larger context, and correctly labels them as non-hateful. ## extracting overfitting terms while being the core of ear, attention entropy serves another purpose. once standard fine-tuning is concluded (i.e., with no regularization involved), models have overfitted specific terms. we identify these terms using attention entropy. to extract the most indicative terms, we replicate training conditions. specifically, we run inference using all the training data using a fine-tuned checkpoint and a standard bert tokenizer. we collect attention entropy values for each term and average them over all training instances. terms with lowest average entropy show the highest overfitting as the model learned them with a narrow context. 5 retrieving these terms after training allows us to gain insights into the domain and language-specific aspects driving the outcome. ## related work the first works to study bias measurement and mitigation in neural representation aimed at removing implicit gender bias from word embeddings @xcite @xcite @xcite . more recently, researchers have started to focus on contextualized sentence representations and effective neural models for understanding the presence and resolution of bias @xcite . while the majority of proposed approaches focus on data augmentation @xcite @xcite @xcite , different approaches have been proposed for bias mitigation intervening directly in the objective function. @xcite proposed to apply regularization during training to the explanationbased importance of identity terms, obtained with sampling-and-occlusion (soc) explanations @xcite . @xcite pro-dataset overfitting terms misogyny (en) girls, womens*ck, hoes, c*ck, shut, stupid, hoe, p*ssy, trying, f*ck misogyny (ita) pezzo, bel, bellissima, scoperei, p*ttanona, zitta, sb*rro, t*ttona, bella, c*lone (piece, nice, very nice, i'd f*ck, sl*t, shut up, c*m, b*sty, beautiful, fat*ss) mlma n*gger, n*gro, shut, chong, ching, d*ke, okay, sp*c, tw*t, f*ggot the main drawback of all aforementioned works is their strict reliance on a set of predefined identity terms. this list can be either defined manually by experts or extracted a-priori from the data set. in both cases, the subsequent debiasing models will be strongly affected by these biased terms, limiting the applicability of the trained model to new data. this is a severe limitation, since it is not always possible to retrain a model on new data to reduce bias, resulting in limited use in real-world cases. ## conclusion we introduce ear, a regularization approach applicable to any attention-based model. our approach does not require any a-priori knowledge of identity terms, e.g., lists. this feature (i) allows us to generalize to different languages and contexts, and (ii) avoids neglecting important terms. thus, it prevents the introduction of further bias. as part of the training procedure, ear also discovers the impact of relevant domain-specific terms. this automatic term extraction provides researchers with an analysis tool to improve data collection and bias mitigation approaches. ear, applied to bert, reliably classifies data with competitive performance and substantially improves various bias metrics. bert+ear generalizes better to new domains and languages than similar methods. in future work, we will apply ear-based models to different downstream tasks to both improve bias mitigation and automatically extract biased terms.