Title: DOUBLE-HARD DEBIASING: TAILORING WORD EM-BEDDINGS FOR GENDER BIAS MITIGATION

Abstract: Gender bias in word embeddings has been widely investigated. However, recent work has shown that existing approaches, including the well-known Hard Debias algorithm (Bolukbasi et al., 2016) which projects word embeddings to a subspace orthogonal to an inferred gender direction, are insufficient to deliver gender-neutral word embeddings. In our work, we discover that semantic-agnostic corpus statistics such as word frequency are important factors that limit the debiasing performance. We propose a simple but effective processing technique, Double-Hard Debias, to attenuate the effect due to such noise. We experiment with Word2Vec and GloVe embeddings and demonstrate on several benchmarks that our approach preserves the distributional semantics while effectively reducing gender bias to a larger extent than previous debiasing techniques.

1. INTRODUCTION:

Word embeddings are based on the distributional hypothesis that linguistic tokens, such as words, with similar distributions in the vector space have similar meanings. This assumption is the basis for statistical semantics that captures the frequency and order of recurrence of words in context along with their meaning. Widely used word embeddings Word2Vec (Mikolov et al., 2013b) and GloVe (Pennington et al., 2014) are known to exhibit gender bias (Bolukbasi et al., 2016) which gets further amplified when these biased embeddings are used in various downstream NLP tasks (Zhao et al., 2018a; Rudinger et al., 2018) . To mitigate this effect of gender bias, most of the recent research has focused on identifying the gender dimensions in word embeddings and zeroing out the gender component in a post-hoc manner (Bolukbasi et al., 2016; Prost et al., 2019) . However, the effectiveness of these efforts has been limited as argued in Gonen & Goldberg (2019) .

We hypothesize that getting rid of the gender component from word embeddings is more complicated than it was expected. An important assumption towards the success of Hard Debias is we can effectively find the gender direction. However, this heavily relies on a set of gender word pairs which is manually defined. Although Bolukbasi et al. (2016) carefully derives 10 word pairs through crowdsourcing, it is hard to claim they can perfectly capture the gender direction in the embedding space. Even with proper pairs, as word embeddings are learned from text corpora through machine learning algorithms, statistical signals can be easily encoded into embeddings and further lead to an offset in gender direction. It has been shown (Gong et al., 2018; Mu & Viswanath, 2018 ) that frequency of words causes a twist of the geometry of word embeddings, which degrades the quality of them. We posit that frequency also contaminate the gender direction that we want to find. As a consequence, it constraints the debiasing ability of Hard Debias.

To this end, we propose a novel pre-processing algorithm called Double-Hard Debias that builds on the existing Hard Debias technique. The intuition behind our method is that word embeddings exhibit another more general type of bias in which words that have similar frequencies during training tend to be closer in the vector space even in cases when they do not have similar meanings -and this is a bias that closely intertwined with gender bias. Double-Hard Debias demonstrates that it is important to not just down weight the gender component as in previous works but also the frequency component in order to "purify" embeddings with respect to gender.

More concretely, inspired by (Mu & Viswanath, 2018) , we conduct PCA(principal component analysis) on all word embeddings to get a set of principal components which we consider as candidate directions related with frequency. We then run kmeans clustering on male and female biased words as a proxy experiment to pick a direction that is most helpful to alleviate the influence from frequency. We then finetune embeddings by projecting out the component along the selected direction. On those purified embeddings, Hard Debias manages to further reduce gender bias. This two-stage process, named as Double-Hard Debias , achieves better debiasing results and meanwhile, is capable of preserving distributional semantics.

2. BACKGROUND:

Definitions. Let W be the vocabulary of a particular language. Following Bolukbasi et al. (2016) , we assume there is a set of gender neutral words N ∈ W, such as "doctor" and "teacher", which by definition are not specific to any gender. We also assume a given set of female-male word pairs P ⊂ W × W, where the main difference between each pair of words is gender 1 .

An embedding consists of a vector -

→ w ∈ R n for each word w ∈ W. A subspace B is defined by k orthogonal unit vectors B = {b 1 , . . . , b k } ∈ R d . A subspace with k = 1 is also called a direction.

We denote the projection of vector v on

B by v B = k j=1 (v • b j )b j and the projection onto the orthogonal subspace v ⊥B = v -v B .

Hard Debias. Hard Debias is a debiasing algorithm defined in terms of word sets. The algorithm takes as input the set of words to neutralize N . and a family of equality sets E = {E 1 , E 2 , . . . , E m |E i ⊆ W, i = 1, . . . m}. First, it identifies a direction in the word embedding space that captures the bias. Then it transforms -→ w such that every word w ∈ N has zero projection in the bias direction and is equidistant to all words in each equality set.

Mathematically, the algorithm performs the following two steps.

1. Identify the bias subspace. Let

µ i := w∈Ei - → w /|E i |.

The bias subspace B is the first k (≥ 1) rows of SVD(C), where

EQUATION

2. Neutralize and equalize 2 . For each E i ∈ E, let ν i := µ i⊥B . Then update -→ w 's by

EQUATION

For mitigating gender bias 3 , we follows Bolukbasi et al. (2016) to set k = 1, N = N and E = P, where each equality set is a female-male word pair.

Clustering Male and Female Biased Words. For each word w ∈ W, we compute its bias score

B = cos( - → w , -→ he) -cos( - → w , -→ she).

We then sort the vocabulary according to B and take top k words as male biased words set W m and bottom k words as W f . For w i ∈ W m , we create ground truth gender label g i = 0. Similarly, for w i ∈ W f , annotate g i = 1. We run Kmeans to cluster the embeddings of selected words and compute the accuracy score a based on the clustering output and the created ground truth gender labels:

EQUATION

For comparison convenience, we set a = max(a, 1 -a). A higher a value shows stronger gender information in embedding which is captured by the clustering algorithm.

1 P consists of "woman"&"man", "girl"&"boy", "she"&"he", "mother"&"father", "daughter"&"son", "gal"&"guy", "female"&"male", "her"&"his", "herself"&"himself", and "Mary"&"John" (Bolukbasi et al., 2016) .

2 Originally, Hard Debias normalizes embeddings. However, we found it is unnecessary in our experiments. This is also confirmed in Ethayarajh et al. (2019) 3 Hard Debias is general and applicable to mitigating multi-class biases such as racial or religious bias.

3. DOUBLE-HARD DEBIASING:

We recall that word embeddings are representations learned from massive text corpora to capture the relations between words. The attribute "gender" is induced from a set of gender specific words, such as "he" and "she", "man" and "woman". In the process of learning word embeddings, "gender" propagates to other words that interact with gender specific ones. Gender bias happens when a word is neutral by its definition but the embedding learned is strongly associated with gender. For example, "programmer" is closer to "male" than "female" in the embedding space because it appears more frequently with male words in the training corpus. To address this, Bolukbasi et al. (2016) proposes Hard Debias which aims to eliminate the effect from gender specific words. Specifically, Hard Debias projects word embeddings into a subspace orthogonal to a gender direction. To find the gender direction, it requires a collection of gendered word pairs which were obtained using crowdsourcing.

The method of Hard Debias should work well if it can effectively find the gender direction, however we realize that there may exist factors that can potentially lessen the effectiveness of Hard Debias.

As observed in recent works (Mu & Viswanath, 2018; Gong et al., 2018) , word embeddings also encode a seemingly more pervasive type of bias with respect to word frequency. Popular words and rare words cluster in different subregions of the embedding space. This further affects the semantic properties of word embeddings, as two words with similar semantic meanings may be far apart due to frequency. We posit that frequency also interferes with the gender direction. In fact, the selected ten word pairs typically used to compute the gender direction have significantly different frequencies, e.g. "guy" and "gal" are much less frequent than "he" and "she". Moreover, the context word of each words also occurs at various frequencies, ultimately affecting the embeddings for both words.

On the other hand, as the geometry of embeddings can be affected by frequency, we speculate that the gender direction in some words is not well aligned with the direction generated by Hard Debias. This also constraints the Debias ability of Hard Debias.

Mu & Viswanath (2018) empirically found that the top D dominating directions of word embeddings encode word frequency to a significant degree. Hence, they propose a simple post-processing technique to alleviate the influence of different frequencies. In particular, they subtract the mean vector, then remove a few top dominating directions from the original word embeddings. The postprocessed embeddings consistently perform better on multiple tasks, validating the effectiveness of this operation. They argue that the benefits emerge from obtaining new word embeddings that are more "isotropic", i.e. where the words are no longer represented mostly by a few dominant dimensions. As original embeddings are far from isotropic, by eliminating the frequency information, the proposed post-processing yields more isotropic embeddings. Inspired by this, we further look into the effect of this post-processing technique on debasing algorithms. We start with the aforementioned biased words clustering experiment for bias evaluation, where we cluster top biased words and measure the alignment accuracy with respect to gender. Following steps in Mu & Viswanath (2018) , we first compute the PCA components of all word embeddings and then project embeddings away from the D th dominating direction. Unsurprisingly, we observe little changes in terms of clustering accuracy. Why does it happen? First, it should be noticed that the post-processing method is supposed to only deal with frequency features implicitly; second, although word frequency becomes similar by the implicit way, it doesn't remove the gender specific words, which still exists in the context of the word. Therefore, the gender bias doesn't disappear and the clustering accuracy is maintained, but by removing the frequency factor on gender, it would only remain the gender direction to be eliminated.

To do so, we then apply Hard Debias on the post-processed embeddings. As we can see from Figure 1(a, b ), projecting away the D th (8 for Word2Vec and 2 for GloVe ) components from word embeddings effectively helps Hard Debias on the gender evaluation task: Male and Female bias words clustering. This result is akin to experiments in Mu & Viswanath (2018) . It clearly show that there exists certain direction that can largely improve the debias result of Hard Debias. 4 Furthermore, we also try to remove the top D components in Figure 1(c, d ), clearly at D = 8 for Word2Vec it has a sudden drop, then it keeps smoothly no change or even worse which means only 8 th component works. This is different from Mu & Viswanath (2018) , where they find that a good rule of thumb is to choose top D = d/100, where d is the dimension of a word embedding. We posit that while top components are useful for improving general word embeddings, for specific words that are affected by gender bias, certain directions are more helpful to eliminate the harmful effect due to frequency. More importantly, we propose to incorporate clustering as a proxy experiment to effectively decide the optimal direction. Through elaborate experiments in Section 4.2, we demonstrate that proper preprocessing on word embeddings improves debias results and at the same time, maintains the representational power of the original word embeddings. Combining this preprocessing with Hard Debias, we name our approach as Double-Hard Debias and our detailed algorithm is presented below: 

4. EXPERIMENT:

We use off-the-shelf Word2Vec embeddings trained on the Google News Dataset and GloVe embeddings trained on the wikidump dataset in our experiments. 5 We evaluate Double-Hard Debias across multiple bias measurements. Additionally, we benchmark it on word similarity, concept categorization, word analogy and coreference resolution. We show that Double-Hard Debias is able to reduce more gender bias and at the same time, maintain the quality of word embeddings.

4.1. DEBIAS RESULTS:

Clustering Male and Female Biased Words. We first take the most biased words (500 male and 500 female) according to their cosine similarity with -→ he and -→ she in the original embedding space. We then run k-Means to cluster them into two clusters and compute their alignment accuracy with respect to gender, results are presented in Table 1 . Using the original Word2Vec and GloVe embeddings, k-Means can accurately cluster selected words into a male group and a female group. Hard Debias is able to reduce bias in some way while GN-GloVe (Zhao et al., 2018b) appears to be less effective on this test. Double-Hard Debias , however, reaches an accuracy of 59.9 and 74.1 on Word2Vec and GloVe , indicating much less gender information after Debias. We also conduct tSNE (van der Maaten & Hinton, 2008) projection of the embeddings before/after Debias. As shown in Figure 2 , from left to right, the male group and female group are closer to each other and confirming less gender cues after Double-Hard Debias .

Classifying Male and Female Biased Words. We follow the same setting in Gonen & Goldberg (2019) , where the objective is to try to predict gender from the debiased word embeddings. We train an RBF-kernel SVM classifier with embeddings of 1000 biased words (500 male and 500 female) and test it on other 4000 embeddings (2000 male and 2000 female). The result is in line with the previous clustering experiment. As we can see in Table 1 , Double-Hard Debias achieves the lowest accuracy on both Word2Vec and GloVe , suggesting a better Debias result. Note that this experiment explicitly finds common patterns between training data and test data, thus it is hard to reach an accuracy of as low as 50% and the absolute accuracy value may not be a good reflection of gender bias. The Word Embeddings Association Test (WEAT). WEAT is a permutation test used to measure the bias in word embeddins. For more details, we refer the reader to Caliskan et al. (2017) . We consider male names and females names as attribute sets and compute the differential association of the two sets of target words6 and the gender attribute sets. We report effect sizes (d) and p-values (p) in Table27 . The effect size is a normalized measure of how separated the two distributions are. A higher value of effect size indicates larger bias between target words in regards to gender. A high p-value higher (more than 0.5) indicates the lack of bias. With different target words sets, Double-Hard Debias consistently outperforms other Debias methods. 

4.2. RETAINING DESIRABLE PROPERTIES:

Word Similarity. The word similarity tasks evaluates if the similarities between word pairs in the embedding space are consistent with human judgements, in terms of Spearmans rank correlation.

Original and debiased embeddings are evaluated on multiple datasets: WS353 (Finkelstein et al., 2001) , RG65 (Rubenstein & Goodenough, 1965) , the SimLex (Hill et al., 2015) , MTurk (Radinsky et al., 2011) , MEN (Bruni et al., 2014) ), RW (Luong et al., 2013) . The detailed performan is reported in Table3, where we see that embeddings after Double-Hard Debias preserve and at times increase the semantic information in the embeddings Concept Categorization. The goal of concept categorization is to cluster a set of words into different categorical subsets. For example, "sandwich" and "hotdog" are both food and "dog" and "cat" are both animals. The clustering performance is evaluated in terms of purity (Manning et al., 2008) -the fraction of the total number of the words that are correctly classified. Experiments are conducted on four datasets: the Almuhareb-Poesio (AP) dataset (Almuhareb, 2006) ; the ESSLLI 2008 (Baroni et al., 2008) ; the Battig 1969 set (Battig & Montague, 1969) and the BLESS dataset (Baroni & Lenci, 2011) . We run classical k-Means algorithm with fixed k. Again, across four datasets, the performance after Debias is on a par with original ones, full results are in Table4.

Word Analogy. Given three words A, B and C, the analogy tasks are to find word D such that "A is to B as C is to D". In our setting, D is the word that maximize the cosine similarity between D and C -A + B. We evaluate embeddings on MSR (Mikolov et al., 2013c) dataset which contains 8000 syntactic questions and Google (Mikolov et al., 2013a) which contains 19, 544 questions, including 8, 869 semantic and 10, 675 syntactic questions. The evaluation metric for the analogy tasks is the percentage of questions for which the correct answer is assigned the maximum score by the algorithm. Double-Hard Debias achieves comparable results for both syntactic and semantic parts as shown in Table4, indicating our trick is capable of preserving proximity among words. Coreference Resolution. In addition, we examine Debias methods on a more complex downstream task. Coreference resolution aims at identifying noun phrases referring to the same entity. Zhao et al. (2018a) introduces a new benmark, WinoBias to certify gender bias in coreference resolution system. WinoBias provides pro-stereotype (PRO) and antistereotype (ANTI) subsets. In the PRO subset, gender pronouns refer to professions dominated by the same gender. For example, in the sentence "The physician hired the secretary because he was overwhelmed with clients.", the pronoun "he" refers to "physician", which is consistent with societal stereotype. On the other hand, the ANTI subset consists of the same set of sentences, but the opposite gender pronouns. As such, "he" is replaced by "she" in the aforementioned example. The hypothesis is that gender cues may distract the model. We consider a system to be gender biased if it performs more better in pro-stereotypical scenarios than in anti-stereotypical scenarios.

We train the end-to-end coreference resolution model (Lee et al., 2017) with different word embeddings on OntoNote training set and report their performance on the Ontonotes5.0 test set and the WinoBias dataset. We also include the average (Avg) and absolute difference (Diff) of F1 scores on PRO and ANTI for the WinoBias dataset. Note that a smaller Diff value indicates a less biased coreference system. Results in Table5 shows that thr proposed trick does not degrade the coreference performance but significantly reduce gender bias. Gender Bias in Word Embeddings. Word embeddings have been proven to carry social biases (e.g. gender and race). Bolukbasi et al. (2016) shows that Word2Vec (Mikolov et al., 2013b) embeddings trained on Google News dataset associate "programmer" with "man" closer than "woman". Such a bias will also propagate to downstream tasks, e.g. coreference systems (Zhao et al., 2018a) and machine translation (Stanovsky et al., 2019) . More recently, researchers also observe significant gender bias in modern contextual word embeddings (Zhao et al., 2019; Kurita et al., 2019) . To mitigate gender bais, Bolukbasi et al. (2016) introduces a post-processing method which zeros out the component along the gender direction of emebddings. Zhao et al. (2018b) ; Kaneko & Bollegala (2019) tackle this problem by proposing a new training procedure to explicitly restrict gender information in certain dimensions. While existing methods reduce gender bias in some degree, Gonen & Goldberg (2019) presents a series of experiments to show that they are far from delivering gendernetural embeddings. Our work builds on top of Bolukbasi et al. (2016) . We propose a simple but effective preprocessing trick that helps to further reduce gender bias.

Word Embedding Learning. Our work also connects with recent research on word embedding learning. Gong et al. (2018) proposes to learn frequency-free embeddings through adversarial training. They found that words with various frequencies behave differently in the embedding space, which can potentially harm the semantic meaning. Similarly, Mu & Viswanath (2018) validates that dominating directions encode frequency and by eliminating them, performance gets improved on word embedding benchmarks. More generally, Wang et al. (2018) introduces a post-processing technique to normalize the variances of word embeddings instead of removing certain directions. We draw inspiration from these works and further transfer it to reducing gender bias in word embeddings. We provide extensive experiments to show that simple operation on word embeddings can advance existing debiasing algorithms.

6. CONCLUSION:

Gender Bias has attracted lots of attentions and been widely studied in NLP. In this paper, we find out there is another important factor: words frequency, which is neglected in previous gender bias reduction works. Motivated by this observation, we proposed Double-Hard Debias method, which is composed of two stages: we first use PCA to obtain candidate components and adopt clustering on male and female biased words as a proxy method to choose the component on frequency; then we project out the frequency component and gender direction from the embeddings via Hard Debias. We experiment on several benchmarks and demonstrate that our Double-Hard Debias is more effective on gender bias reduction with semantics preserving. In future, we plan to study different way of finding the frequency component and also extend the proposed method for other types of bias reduction.

:

We show the top 20 components for readability.

The embeddings used are not centered and normalized to unit length as inBolukbasi et al. (2016).

All word lists are fromCaliskan et al. (2017)

We use lower cased names and replace "bill" with "tom" as we use uncased GloVe embeddings.

Figures and Tables:

Figure 1: Clustering accuracy after projecting out D-th (a&b) or top D (c&d) dominating directions and Hard Debias. Lower accuracy indicates less bias.

Double-Hard Debias for word embeddings. Input : Word embeddings { -→ w ∈ R d , w ∈ W}, set of Male and Female bias words W m and W f S debias = [] Decentralize -→ w : µ ← 1 |V| w∈V -→ w , for each -→ w ∈ W, w ← -→ w -µ; Compute the PCA components: {u 1 . . . u d } ← PCA({ w, w ∈ W}); //discover the component on frequency for i = 1 to d do w m ← wm -(u T i w m )u i ; w f ← wf -(u T i w f )u i ; ŵm ← HardDebias(w m ); ŵf ← HardDebias(w f ); output = Kmeans([ ŵm ŵf ]); score = eval(output, W m , W f ); S debias .append(score); end k = arg min i S debias ; //remove component on frequency w ← w -(u T k w)u k ; //remove component on gender direction ŵ ← HardDebias(w ); Output: Debiased word embeddings { ŵ, ŵ ∈ R d }

Figure 2: tSNE visualization of male and female words before/after Debias. Double-Hard Debiased embedding are more mixed up, showing less gender information encoded.

Accuracy(x100) of clustering/classifying male and female words. Lower accuracy means less gender cues can be captured. Double-Hard Debias reaches the lowest accuracy on both tasks.

Table content: <table><tr><td>Embeddings</td><td colspan="2">Clustering Classification</td></tr><tr><td>Word2Vec</td><td>99.9</td><td>99.3</td></tr><tr><td>Hard Word2Vec</td><td>89.2</td><td>90.9</td></tr><tr><td>Double-Hard Word2Vec</td><td>59.9</td><td>87.9</td></tr><tr><td>GloVe</td><td>100.0</td><td>100.0</td></tr><tr><td>Hard GloVe</td><td>77.2</td><td>92.3</td></tr><tr><td>GN-GloVe</td><td>99.5</td><td>99.5</td></tr><tr><td>Double-Hard GloVe</td><td>74.1</td><td>91.3</td></tr></table>

WEAT test of embeddings before/after Debias. The bias is significant when p-value, p < 0.05. Lower effective size (d) indicates less gender bias.

Table content: <table/>

Before/After Debias results (x100) on word similarity task on six datasets.

Table content: <table><tr><td>Embeddings</td><td colspan="4">WS353 RG65 SimLex MTurk</td><td>RW</td><td>MEN</td></tr><tr><td>Word2Vec</td><td>70.0</td><td>76.1</td><td>44.2</td><td>68.4</td><td>53.4</td><td>77.1</td></tr><tr><td>Hard Word2Vec</td><td>69.9</td><td>76.5</td><td>44.3</td><td>68.4</td><td>53.4</td><td>76.9</td></tr><tr><td>Doubel-Hard Word2Vec</td><td>69.5</td><td>77.3</td><td>44.5</td><td>68.6</td><td colspan="2">54.1 78.1</td></tr></table>

Before/After Debias results (x100) on word analogy and concept categorization task.

Table content: <table><tr><td>Embeddings</td><td colspan="3">Analogy Google-Sem Google-Syn MSR</td><td>AP</td><td colspan="4">Concept Categorization ESSLI Battig BLESS</td></tr><tr><td>Word2Vec</td><td>24.8</td><td>66.5</td><td>73.6</td><td colspan="2">64.5</td><td>75.0</td><td>46.3</td><td>78.9</td></tr><tr><td>Hard Word2Vec</td><td>23.8</td><td>66.3</td><td>73.5</td><td>62.7</td><td/><td>75.0</td><td>47.1</td><td>77.4</td></tr><tr><td>Double-Hard Word2Vec</td><td>23.5</td><td>66.3</td><td>74.0</td><td>63.2</td><td/><td>75.0</td><td>46.5</td><td>77.9</td></tr><tr><td>GloVe</td><td>80.5</td><td>62.8</td><td>54.2</td><td>55.6</td><td/><td>75.0</td><td>49.0</td><td>81.0</td></tr><tr><td>Hard GloVe</td><td>80.3</td><td>62.5</td><td>54.0</td><td>57.1</td><td/><td>70.5</td><td>48.9</td><td>78.0</td></tr><tr><td>GN-GloVe</td><td>77.7</td><td>61.6</td><td>51.9</td><td>57.6</td><td/><td>72.7</td><td>49.2</td><td>82.5</td></tr><tr><td>Double-Hard GloVe</td><td>80.9</td><td>61.6</td><td>53.8</td><td colspan="2">59.6</td><td>72.7</td><td>47.2</td><td>79.5</td></tr></table>



Table content: <table/>

Peer review (iclr2020):
The paper proposes a post-processing recipe for gender-neutralizing word embeddings. The proposed method is a combination of existing Debias Algorithm (Bolukbasi et al, 2016) and all-but-the-top model (Mu and Viswanath, 2018). The former tries to identify gender component and eliminate the effect of that and the latter tries to reduce the effect of frequencies in the quality of embedding. The proposed method, Double-hard Debias tries to reduce gender bias from embedding, in a way that embeddings remain effective. The paper is readable and addresses an important question. However, the method proposed is incremental, missing extensive technical or intuitive exploration of its effect and has limited experimental evaluation. - Debiasing experiments are limited and the task in Section 4.1 seems a toy task to me. - Showing neighborhood of problematic words, before and after the preprocessing is useful to see. - The performance of the processed embeddings in downstream NLP tasks could be reported. Decision: I vote for a weak reject.