Towards Debiasing Sentence Representations (2020)

Abstract: As natural language processing methods are increasingly deployed in real-world scenarios such as healthcare, legal systems, and social science, it becomes necessary to recognize the role they potentially play in shaping social biases and stereotypes. Previous work has revealed the presence of social biases in widely used word embeddings involving gender, race, religion, and other social constructs. While some methods were proposed to debias these word-level embeddings, there is a need to perform debiasing at the sentence-level given the recent shift towards new contextualized sentence representations such as ELMo and BERT. In this paper, we investigate the presence of social biases in sentence-level representations and propose a new method, Sent-Debias, to reduce these biases. We show that Sent-Debias is effective in removing biases, and at the same time, preserves performance on sentence-level downstream tasks such as sentiment analysis, linguistic acceptability, and natural language understanding. We hope that our work will inspire future research on characterizing and removing social biases from widely adopted sentence representations for fairer NLP.

## introduction machine learning tools for learning from language are increasingly deployed in real-world scenarios such as healthcare @xcite , legal systems @xcite , and computational social science @xcite . key to the success of these models are powerful embedding layers which learn continuous representations of input information such as words, sentences, and documents from large amounts of data @xcite . although word-level embeddings @xcite are highly informative features useful for a variety of tasks in natural language processing (nlp), recent work has shown that word-level embeddings reflect and propagate social biases present in training corpora @xcite @xcite . machine learning systems that incorporate these word embeddings can further amplify biases @xcite @xcite and unfairly discriminate against users, particularly those from disadvantaged social groups. fortunately, researchers working on fairness and ethics in nlp have devised methods towards debiasing these word representations for both binary @xcite and multiclass @xcite bias attributes such as gender, race, and religion. more recently, sentence-level representations such as elmo @xcite , bert @xcite , and gpt @xcite have become the preferred choice for text sequence encoding. when compared to word-level representations, these models have achieved better performance on multiple tasks in nlp @xcite , multimodal learning @xcite , and grounded language learning @xcite . as their usage proliferates across various real-world applications @xcite , it becomes necessary to recognize the role they play in shaping social biases and stereotypes. debiasing sentence representations is difficult for two reasons. firstly, it is usually unfeasible to fully retrain many of the state-of-the-art sentencebased embedding models. in contrast with conventional word-level embeddings such as glove @xcite and word2vec @xcite which can be retrained on a single machine within a few hours, the best sentence encoders such as bert @xcite , and gpt @xcite are trained on massive amounts of text data over hundreds of machines for several weeks. as a result, it is difficult to retrain a new sentence encoder whenever a new source of bias is uncovered from data. we therefore focus on post-hoc debiasing techniques which add a post-training debiasing step to these sentence representations before they are used in downstream tasks @xcite . secondly, sentences display large variety in how they are composed from individual words. this variety is driven by many factors such as topics, individuals, settings, and even differences between spoken and written text. as a result, it is difficult to scale traditional word-level debiasing approaches (which involve bias-attribute words such as man, woman) @xcite to sentences. ## debiasing sentence representations our proposed method for debiasing sentence representations, sent-debias, consists of four steps: 1) defining the words which exhibit bias attributes, 2) contextualizing these words into bias attribute sentences and subsequently their sentence representations, 3) estimating the sentence representation bias subspace, and finally 4) debiasing general sentences by removing the projection onto this bias subspace. we summarize these steps in algorithm 1 and describe the algorithmic details in the following subsections. 1) defining bias attributes: the first step involves identifying the bias attributes and defining a set of bias attribute words that are indicative of these attributes. for example, when characterizing bias across the male and female genders, we use the word pairs (man, woman), (boy, girl) that are indicative of gender. when estimating the 3class religion subspace across the jewish, christian, and muslim religions, we use the tuples (judaism, christianity, islam), (synagogue, church, mosque). each tuple should consist of words that have an equivalent meaning except for the bias attribute. in general, for d-class bias attributes, the set of words forms a dataset @xmath0 ## experiments we test the effectiveness of sent-debias at removing biases and retaining performance on downstream tasks. all experiments are conducted on english terms and downstream tasks. we acknowledge that biases can manifest differently across different languages, in particular gendered languages @xcite , and emphasize the need for future extensions in these directions. experimental details are in the appendix and code is released at https://github.com/pliang279/ sent_debias . ## discussion and future work firstly, we would like to emphasize that both the weat, seat, and mac metrics are not perfect since they only have positive predictive ability: they can be used to detect the presence of biases but not their absence @xcite . this calls for new metrics that evaluate biases and can scale to the various types of sentences appearing across different individuals, topics, and in both spoken and written text. we believe that our positive results regarding contextualizing words into sentences implies that future work can build on our algorithms and tailor them for new metrics. secondly, a particular bias should only be removed from words and sentences that are neutral to that attribute. for example, gender bias should not be removed from the word "grandmother" or the sentence "she gave birth to me". previous work on debiasing word representations tackled this issue by listing all attribute specific words based on dictionary definitions and only debiasing the remaining words. however, given the complexity of natural sentences, it is extremely hard to identify the set of neutral sentences and its complement. thus, in downstream tasks, we removed bias from all sentences which could possibly harm downstream task performance if the dataset contains a significant number of non-neutral sentences. finally, a fundamental challenge lies in the fact that these representations are trained without explicit bias control mechanisms on large amounts of naturally occurring text. given that it becomes infeasible (in standard settings) to completely retrain these large sentence encoders for debiasing @xcite , future work should focus on developing better post-hoc debiasing techniques. in our experiments, we need to re-estimate the bias subspace and perform debiasing whenever the bert encoder was fine-tuned. it remains to be seen whether there are debiasing methods which are invariant to fine-tuning, or can be efficiently re-estimated as the encoders are fine-tuned. ## conclusion this paper investigated the post-hoc removal of social biases from pretrained sentence representations. we proposed the sent-debias method that accurately captures the bias subspace of sentence representations by using a diverse set of templates from naturally occurring text corpora. our experiments show that we can remove biases that occur in bert and elmo while preserving performance on downstream tasks. we also demonstrate the importance of using a large number of diverse sentence templates when estimating bias subspaces. leveraging these developments will allow researchers to further characterize and remove social biases from sentence representations for fairer nlp. that debiasing performance improves and stabilizes with the number of sentence templates as well as the number of domains.