Debiasing Pre-trained Contextualised Embeddings (2021)

Abstract: In comparison to the numerous debiasing methods proposed for the static non-contextualised word embeddings, the discriminative biases in contextualised embeddings have received relatively little attention. We propose a fine-tuning method that can be applied at token- or sentence-levels to debias pre-trained contextualised embeddings. Our proposed method can be applied to any pre-trained contextualised embedding model, without requiring to retrain those models. Using gender bias as an illustrative example, we then conduct a systematic study using several state-of-the-art (SoTA) contextualised representations on multiple benchmark datasets to evaluate the level of biases encoded in different contextualised embeddings before and after debiasing using the proposed method. We find that applying token-level debiasing for all tokens and across all layers of a contextualised embedding model produces the best performance. Interestingly, we observe that there is a trade-off between creating an accurate vs. unbiased contextualised embedding model, and different contextualised embedding models respond differently to this trade-off.

## introduction contextualised word embeddings have significantly improved performance in numerous natural language processing (nlp) applications @xcite @xcite and have established as the de facto standard for input text representations. compared to static word embeddings @xcite that represent a word by a single vector in all contexts it occurs, contextualised embeddings use dynamic context dependent vectors for representing a word in a specific context. unfortunately however, it has been shown that, similar to their non-contextual counterparts, contextualised text embeddings also encode various types of unfair biases @xcite @xcite @xcite . this is a worrying situation because such biases can easily propagate to the downstream nlp applications that use contextualised text embeddings. different types of unfair and discriminative biases such as gender, racial and religious biases have been observed in static word embeddings @xcite @xcite @xcite . as discussed later in § 2 different methods have been proposed for debiasing static word embeddings such as projection-based methods @xcite @xcite and adversarial methods @xcite . in contrast, despite multiple studies reporting that contextualised embeddings to be unfairly biased, methods for debiasing contextualised embeddings are relatively under explored @xcite @xcite . compared to static word embeddings, debiasing contextualised embeddings is significantly more challenging due to several reasons as we discuss next. first, compared to static word embedding models where the semantic representation of a word is limited to a single vector, contextualised embedding models have a significantly large number of parameters related in complex ways. for example, bert-large model @xcite ) contains 24 layers, 16 attention heads and 340m parameters. therefore, it is not obvious which parameters are responsible for the unfair biases related to a partic-ular word. because of this reason, projection-based methods, popularly used for debiasing pre-trained static word embeddings, cannot be directly applied to debias pre-trained contextualised word embeddings. second, in the case of contextualised embeddings, the biases associated with a particular word's representation is a function of both the target word itself and the context in which it occurs. therefore, the same word can show unfair biases in some contexts and not in the others. it is important to consider the words that co-occur with the target word in different contexts when debiasing a contextualised embedding model. third, pre-training large-scale contextualised embeddings from scratch is time consuming and require specialised hardware such as gpu/tpu clusters. on the other hand, fine-tuning a pre-trained contextualised embedding model for a particular task (possibly using labelled data for the target task) is relatively less expensive. consequently, the standard practice in the nlp community has been to share foot_0 pre-trained contextualised embedding models and fine-tune as needed. therefore, it is desirable that a debiasing method proposed for contextualised embedding models can be applied as a fine-tuning method. in this view, counterfactual data augmentation methods @xcite @xcite that swap gender pronouns in the training corpus for creating a gender balanced version of the training data are less attractive when debiasing contextualised embeddings because we must retrain those models on the balanced corpora, which is more expensive compared to fine-tuning. using gender-bias as a running example, we address the above-mentioned challenges by proposing a debiasing method that fine-tunes pre-trained contextualised word embeddings foot_1 . our proposed method retains the semantic information learnt by the contextualised embedding model with respect to gender-related words, while simultaneously removing any stereotypical biases in the pre-trained model. in particular, our proposed method is agnostic to the internal architecture of the contextualised embedding method and we apply it to debias different pre-trained embeddings such as bert, roberta @xcite , albert @xcite , distilbert @xcite and elec-tra @xcite . moreover, our proposed method can be applied at token-level or at sentencelevel, enabling us to debias at different granularities and on different layers in the pre-trained contextualised embedding model. following prior work, we compare the proposed debiasing method in two sentence-level tasks: sentence encoder association test (seat; @xcite and multi-genre co-reference-based natural language inference (mnli; @xcite . experimental results show that the proposed method not only debiases all contextualised word embedding models compared, but also preserves useful semantic information for solving downstream tasks such as sentiment classification @xcite , paraphrase detection @xcite , semantic textual similarity measurement @xcite , natural language inference @xcite and solving winograd schema @xcite . we consider gender bias as a running example throughout this paper and evaluate the proposed method with respect to its ability to overcome gender bias in contextualised word embeddings, and defer extensions to other types of biases to future work. ## related work prior work on debiasing word embeddings can be broadly categorised into two groups depending on whether they consider static or contextualised word embeddings. although we focus on contextualised embeddings in this paper, we first briefly describe prior work on debiasing static embeddings for completeness of the discussion. bias in static word embeddings @xcite proposed a post-processing approach that projects gender-neutral words into a subspace, which is orthogonal to the gender direction defined by a list of gender-definitional words. however, their method ignores gender-definitional words during the subsequent debiasing process, and focus only on words that are not predicted as gender-definitional by a classifier. therefore, if the classifier erroneously predicts a stereotypical word as gender-definitional, it would not get debiased. @xcite modified the original glove @xcite objective to learn gender-neutral word embeddings (gn-glove) from a given corpus. unlike the above-mentioned methods, @xcite proposed gp-glove, a post-processing method to preserve gender-related information with autoencoder @xcite , while removing discriminatory biases from stereotypical cases. adversarial learning @xcite @xcite for debiasing first encode the inputs and then two classifiers are jointly trained -one predicting the target task (for which we must ensure high prediction accuracy) and the other for protected attributes (that must not be easily predictable). @xcite showed that although it is possible to obtain chance-level development-set accuracy for the protected attributes during training, a post-hoc classifier trained on the encoded inputs can still manage to reach substantially high accuracies for the protected attributes. they conclude that adversarial learning alone does not guarantee invariant representations for the protected attributes. @xcite found that iteratively projecting word embeddings to the null space of the gender direction to further improve the debiasing performance. ## debiasing contextualised embeddings we propose a method for debiasing pre-trained contextualised word embeddings in a fine-tuning setting that simultaneously (a) preserves the semantic information in the pre-trained contextualised word embedding model, and (b) removes discriminative gender-related biases via an orthogonal projection in the intermediate (hidden) layers by operating at token-or sentence-levels. fine-tuning allows debiasing to be carried out without requiring large amounts of tarining data or computational resources. our debiasing method is independent of model architectures or their pre-training methods, and can be adapted to a wide range of contextualised embeddings as shown in § 4.3. let us define two types of words: attribute words (v a ) and target words (v t ). for example, in the case of gender bias, attribute words consist of multiple word sets such as feminine (e.g. she, woman, her) and masculine (e.g. he, man, him) words, whereas target words can be occupations (e.g. doctor, nurse, professor), which we expect to be gender neutral. we then extract sentences that contain an attribute or a target word. sentences contain more than one attribute (or target) words are excluded to avoid ambiguities. let us denote the set of sentences extracted for an attribute or a target word w by ω(w). moreover, let @xmath0. we require that the debiased contextualised word embeddings preserve semantic information w.r.t. the sentences in a, and remove any discriminative biases w.r.t. the sentences in t . let us consider a contextualised word embedding model e, with pre-trained model parameters θ e . for an input sentence x, let us denote the embedding of token w in the i-th layer of e by e i (w, x; θ e ). moreover, let the total number of layers in e to be n . in our experiments, we consider different types of encoder models such as e. to formalise the requirement that the debiased word embedding e i (t, x; θ e ) of a target word @xmath1, we consider the inner-product between the noncontextualised embedding v i (a) of a and e i (t, x; θ e ) as a loss l i given by (1). here, v i (a) is computed by averaging the contextualised embedding of a in the i-th layer of e over all sentences in ω(a) following @xcite and is given by (2). here, |ω(a)| denotes the total number of sentences in ω(a). if a word is split into multiple sub-tokens, we compute the contextualised embedding of the word by averaging the contextualised embeddings of its constituent sub-tokens. minimising the loss l i defined by (1) with respect to θ e forces the hidden states of e to be orthogonal to the protected attributes such as gender. although removing discriminative biases in e is our main objective, we must ensure that simultaneously we preserve as much useful information that is encoded in the pre-trained model for the downstream tasks. we model this as a regulariser where we measure the squared ℓ 2 distance between the contextualised word embedding of a word w in the i-th layer in the original model, parametrised by θ pre , and the debiased model as in (3). the overall training objective is then given by (4) as the linearly weighted sum of the two losses defined by ( 1 ) and (3). here, coefficients α, β ∈ [0, 1] satisfy α + β = 1. as shown in figure 1 , a contextualised word embedding model typically contains multiple layers. it is not obvious which hidden states of e are best for calculating l i for the purpose of debiasing. therefore, we compute l i for different layers in a particular contextualised word embedding model in our experiments. specifically, we consider three settings: debiasing only the first layer, last layer or all layers. moreover, l i can be computed only for the target words in a sentence x as in (1), or can be summed up for all words in @xmath2.e. we refer to the former as token-level debiasing and latter sentence-level debiasing. collectively this gives us six different settings for the proposed debiasing method, which we evaluate experimentally in § 4.3. ## conclusion we proposed a debiasing method for pre-trained contextualised word embeddings, operating at token-or sentence-levels. our experimental results showed that the proposed method effectively debiases discriminative gender-related biases, while preserving useful semantic information in the pretrained embeddings. the results showed that the downstream task was more effective in debias than the previous studies.