SlowGuess/ACL_2020_OCR huggingface dataset:
Table 2: WEAT test of embeddings before/after Debiasing. The bias is insignificant when p-value,  $p > 0.05$ . Lower effective size  $(d)$  indicates less gender bias. Significant gender bias related to Career & Family and Science & Arts words is effectively reduced by Double-Hard GloVe. Note for Math & Arts words, gender bias is insignificant in original GloVe.
Embeddings	Top 100	Top 500	Top 1000
GloVe	100.0	100.0	100.0
GN-GloVe	100.0	100.0	99.9
GN-GloVe(wa)	100.0	99.7	88.5
GP-GloVe	100.0	100.0	100.0
GP-GN-GloVe	100.0	100.0	99.4
(Strong) Hard GloVe	59.0	62.1	68.1
Double-Hard GloVe	51.5	55.5	59.5
Table 3: Clustering Accuracy (%) of top 100/500/1000 male and female words. Lower accuracy means less gender cues can be captured. Double-Hard GloVe consistently achieves the lowest accuracy.
4.4 Analysis of Retaining Word Semantics
Word Analogy. Given three words  $A$ ,  $B$  and  $C$ , the analogy task is to find word  $D$  such that “ $A$  is to  $B$  as  $C$  is to  $D$ ”. In our experiments,  $D$  is the word that maximizes the cosine similarity between  $D$  and  $C - A + B$ . We evaluate all non-debiased and debiased embeddings on the MSR (Mikolov et al., 2013c) word analogy task, which contains 8000 syntactic questions, and on a second Google word analogy (Mikolov et al., 2013a) dataset that contains 19,544 (Total) questions, including 8,869 semantic (Sem) and 10,675 syntactic (Syn) questions. The evaluation metric is the percentage of questions for which the correct answer is assigned
the maximum score by the algorithm. Results are shown in Table4. Double-Hard GloVe achieves comparable good results as GloVe and slightly outperforms some other debiased embeddings. This proves that Double-Hard Debias is capable of preserving proximity among words.
Concept Categorization. The goal of concept categorization is to cluster a set of words into different categorical subsets. For example, "sandwich" and "hotdog" are both food and "dog" and "cat" are animals. The clustering performance is evaluated in terms of purity (Manning et al., 2008) - the fraction of the total number of the words that are correctly classified. Experiments are conducted on four benchmark datasets: the Almuhareb-Poesio (AP) dataset (Almuhareb, 2006); the ESSLLI 2008 (Baroni et al., 2008); the Battig 1969 set (Battig and Montague, 1969) and the BLESS dataset (Baroni and Lenci, 2011). We run classical Kmeans algorithm with fixed  $k$ . Across four datasets, the performance of Double-Hard GloVe is on a par with GloVe embeddings, showing that the proposed debiasing method preserves useful semantic information in word embeddings. Full results can be found in Table4.
Gender Bias in Word Embeddings. Word embeddings have been criticized for carrying gender bias. Bolukbasi et al. (2016) show that word2vec (Mikolov et al., 2013b) embeddings trained on the Google News dataset exhibit occupational stereotypes, e.g. "programmer" is closer to "man" and "homemaker" is closer to "woman". More recent works (Zhao et al., 2019; Kurita et al., 2019; Basta
(a) GloVe
(e) GP-GN-GloVe
Figure 3: tSNE visualization of top 500 most male and female embeddings. Double-Hard GloVe mixes up two groups to the maximum extent, showing less gender information is encoded.
(b) GN-GloVe
(f) Hard-GloVe
(c) GN-GloVe(wa)
(g) Strong Hard-GloVe
(d) GP-GloVe
(h) Double-Hard GloVe
Embeddings	Sem	Analogy	Concept Categorization
Syn	Total	MSR	AP	ESSLI	Battig	BLESS
GloVe	80.5	62.8	70.8	54.2	55.6	72.7	51.2	81.0
GN-GloVe	77.7	61.6	68.9	51.9	56.9	70.5	49.5	85.0
GN-GloVe(wa)	77.7	61.6	68.9	51.9	56.9	75.0	51.3	82.5
GP-GloVe	80.6	61.7	70.3	51.3	56.1	75.0	49.0	78.5
GP-GN-GloVe	77.7	61.7	68.9	51.8	61.1	72.7	50.9	77.5
Hard-GloVe	80.3	62.5	70.6	54.0	62.3	79.5	50.0	84.5
Strong Hard-GloVe	78.6	62.4	69.8	53.9	64.1	79.5	49.2	84.5
Double-Hard GloVe	80.9	61.6	70.4	53.8	59.6	72.7	46.7	79.5
Table 4: Results of word embeddings on word analogy and concept categorization benchmark datasets. Performance (x100) is measured in accuracy and purity, respectively. On both tasks, there is no significant degradation of performance due to applying the proposed method.
et al., 2019) demonstrate that contextualized word embeddings also inherit gender bias.
Gender bias in word embeddings also propagate to downstream tasks, which substantially affects predictions. Zhao et al. (2018a) show that coreference systems tend to link occupations to their stereotypical gender, e.g. linking "doctor" to "he" and "nurse" to "she". Stanovsky et al. (2019) observe that popular industrial and academic machine translation systems are prone to gender biased translation errors.
Recently, Vig et al. (2020) proposed causal mediation analysis as a way to interpret and analyze gender bias in neural models.
Debiasing Word Embeddings. For contextualized embeddings, existing works propose task-specific debiasing methods, while in this paper we focus on more generic ones. To mitigate gender bias, Zhao et al. (2018a) propose a new training approach which explicitly restricts gender information in certain dimensions during training. While this method separates gender information from em
beddings, retraining word embeddings on massive corpus requires an undesirably large amount of resources. Kaneko and Bollegala (2019) tackles this problem by adopting an encoder-decoder model to re-embed word embeddings. This can be applied to existing pre-trained embeddings, but it still requires train different encoder-decoders for different embeddings.
Bolukbasi et al. (2016) introduce a more simple and direct post-processing method which zeros out the component along the gender direction. This method reduces gender bias to some degree, however, Gonen and Goldberg (2019) present a series of experiments to show that they are far from delivering gender-neutral embeddings. Our work builds on top of Bolukbasi et al. (2016). We discover the important factor – word frequency – that limits the effectiveness of existing methods. By carefully eliminating the effect of word frequency, our method is able to significantly improve debiasing performance.
We have discovered that simple changes in word frequency statistics can have an undesirable impact on the debiasing methods used to remove gender bias from word embeddings. Though word frequency statistics have until now been neglected in previous gender bias reduction work, we propose Double-Hard Debias, which mitigates the negative effects that word frequency features can have on debiasing algorithms. We experiment on several benchmarks and demonstrate that our Double-Hard Debias is more effective on gender bias reduction than other methods while also preserving the quality of word embeddings suitable for the downstream applications and embedding-based word analogy tasks. While we have shown that this method significantly reduces gender bias while preserving quality, we hope that this work encourages further research into debiasing along other dimensions of word embeddings in the future.
Abdulrahman Almuhareb. 2006. Attributes in lexical acquisition. Ph.D. thesis, University of Essex, Colchester, UK.
Marco Baroni, Stefan Evert, and Alessandro Lenci. 2008. Bridging the gap between semantic theory and computational simulations: Proceedings of the essli workshop on distributional lexical semantics.
Marco Baroni and Alessandro Lenci. 2011. How we blessed distributional semantic evaluation. In Proceedings of the GEMS 2011 Workshop on Geometrical Models of Natural Language Semantics, GEMS '11, pages 1-10, Stroudsburg, PA, USA. Association for Computational Linguistics.
Christine Basta, Marta Ruiz Costa-jussa, and Noe Casas. 2019. Evaluating the underlying gender bias in contextualized word embeddings. CoRR, abs/1904.08783.
William F. Battig and William E. Montague. 1969. Category norms of verbal items in 56 categories: a replication and extension of the connecticut category norms. Journal of Experimental Psychology, 80(3p2):1.
Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligram, and Adam Tauman Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In NIPS.
Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically
from language corpora contain human-like biases. Science, 356(6334):183-186.
Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillip Koehn, and Tony Robinson. 2013. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005.
Kawin Ethayarajh, David Duvenaud, and Graeme Hirst. 2019. Understanding undesirable word embedding associations. arXiv preprint arXiv:1908.06361.
Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In *NAACL-HLT*.
Chengyue Gong, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tie-Yan Liu. 2018. Frage: Frequency-agnostic word representation. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 1334–1345. Curran Associates, Inc.
Masahiro Kaneko and Danushka Bollegala. 2019. Gender-preserving debiasing for pre-trained word embeddings. CoRR, abs/1906.00742.
Keita Kurita, Nidhi Vyas, Ayush Parek, Alan W. Black, and Yulia Tsvetkov. 2019. Measuring bias in contextualized word representations. CoRR, abs/1906.07337.
Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 188-197. Association for Computational Linguistics.
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579-2605.
Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schtze. 2008. Introduction to Information Retrieval. Cambridge University Press, Cambridge, UK.
Tomas Mikolov, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 3111-3119. Curran Associates, Inc.
Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013c. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746-751, Atlanta, Georgia. Association for Computational Linguistics.
Jiaqi Mu and Pramod Viswanath. 2018. All-but-thetop: Simple and effective postprocessing for word representations. In International Conference on Learning Representations.
Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. arXiv preprint arXiv:1804.09301.
Gabriel Stanovsky, Noah A Smith, and Luke Zettle-moyer. 2019. Evaluating gender bias in machine translation. arXiv preprint arXiv:1906.00591.
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Causal mediation analysis for interpreting neural nlp: The case of gender bias.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Coterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In North American Chapter of the Association for Computational Linguistics (NAACL).
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender bias in coreference resolution: Evaluation and debiasing methods. In North American Chapter of the Association for Computational Linguistics (NAACL).
Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning gender-neutral word embeddings. In EMNLP.
Figure 4: Clustering accuracy after projecting out D-th dominating direction and applying Hard Debias. Lower accuracy indicates less bias.
Embeddings	Top 100	Top 500	Top 1000
Word2Vec	100.0	99.3	99.3
Hard-Word2Vec	79.5	74.3	79.8
Double-Hard Word2Vec	71.0	52.3	56.7
Table 5: Clustering Accuracy(%) of top 100/500/1000 male and female words. Lower accuracy means less gender cues captured. Double-Hard Word2Vec consistently achieves the lowest accuracy.
We also apply Double-Hard Debias on Word2Vec embeddings (Mikolov et al., 2013b) which have been widely used by many NLP applications. As shown in Figure 4, our algorithm is able to identify that the eighth principal component significantly affects the debiasing performance.
Similarly, we first project away the identified direction  $\mathbf{u}$  from the original Word2Vec embeddings and then apply Hard Debias algorithm. We compare embeddings debiased by our method with the original Word2Vec embeddings and HardWord2Vec embeddings.
Table 5 reports the experimental result using the neighborhood metric. Across three experiments where we cluster top 100/500/1000 male and female words, Double-Hard Word2Vec consistently achieves the lowest accuracy. Note that neighborhood metric reflects gender information that can be captured by the clustering algorithm. Experimental result validates that our method can further improve Hard Debias algorithm. This is also verified in Figure 5 where we conduct tSNE visualization of top 500 male and female embeddings. While the original Word2Vec embeddings clearly locate separately into two groups corresponding to different genders, this phenomenon becomes less obvious after applying our debiasing method.
We further evaluate the debiasing outcome with WEAT test. Similar to experiments on GloVe em
Embeddings	Career & Family	Math & Arts	Science & Arts
d	p	d	p	d	p
Word2Vec	1.89	0.0	1.82	0.0	1.57	2e-4
Hard-Word2Vec	1.80	0.0	1.57	7e-5	0.83	0.05
Double-Hard Word2Vec	1.73	0.0	1.51	5e-4	0.68	0.09
Table 6: WEAT test of embeddings before/after Debiasing. The bias is insignificant when p-value,  $p > 0.05$ . Lower effective size  $(d)$  indicates less gender bias. Across all target words sets, Double-Hard Word2Vec leads to the smallest effective size. Specifically, for Science & Arts words, Double-Hard Word2Vec successfully reaches a bias insignificant state  $(p = 0.09)$ .
Embeddings	Sem	Analogy	Concept Categorization
Syn	Total	MSR	AP	ESSLI	Battig	BLESS
Word2Vec	24.8	66.5	55.3	73.6	64.5	75.0	46.3	78.9
Hard-Word2Vec	23.8	66.3	54.9	73.5	62.7	75.0	47.1	77.4
Double-Hard Word2Vec	23.5	66.3	54.9	74.0	63.2	75.0	46.5	77.9
Table 7: Results of word embeddings on word analogy and concept categorization benchmark datasets. Performance (x100) is measured in accuracy and purity, respectively. On both tasks, there is no significant degradation of performance due to applying the proposed method.
beddings, we use male names and female names as attribute sets and analyze the association between attribute sets and three target sets. We report effective size and p-value in Table 6. Across three target sets, Double-Hard Word2Vec is able to consistently reduce the effect size. More importantly, the bias related to Science & Arts words becomes insignificant after applying our debiasing method.
To test the functionality of debiased embeddings, we again conduct experiments on word analogy and concept categorization tasks. Results are included in Table 7. We demonstrate that our proposed debiasing method brings no significant performance degradation in these two tasks.
To summarize, experiments on Word2Vec embeddings also support our conclusion that the proposed Double-Hard Debiasing reduces gender bias to a larger degree while is able to maintain the semantic information in word embeddings.
(a) Word2Vec
(b) Hard-Word2Vec
(c) Double-Hard Word2Vec
Figure 5: tSNE visualization of top 500 most male and female embeddings. Double-Hard Word2Vec mixes up two groups to the maximum extent, showing less gender information encoded.