SlowGuess/ACL_2020_OCR huggingface dataset:
De-Arteaga et al. (2019) built an English BiosBias dataset to evaluate the bias in predicting the occupations of people when provided with a short biography on the bio of the person written in third person. To evaluate the bias in cross-lingual transfer settings, we build the Multilingual BiosBias (MLBs) Dataset which contains bios in different languages.
Dataset Collection Procedure We collect a list of common occupations for each language and follow the data collection procedure used for the English dataset (De-Arteaga et al., 2019). To identify bio paragraphs, we use the pattern "NAME is an OCCUPATION-TITLE" where name is recognized in each language by using the corresponding Named Entity Recognition model from spaCy. To control for the same time period for datasets across languages, we process the same set of Common crawl dumps ranging from the year 2014 to 2018. For the occupations, we use both the feminine and masculine versions of the word in the gender-rich languages. For EN, we use the existing BiosBias dataset.
The number of occupations in each language is shown in Table 5. As the bios are written in third person, similar to De-Arteaga et al. (2019), we extract the binary genders based on the gendered pronouns in each language, such as "he" and "she".
$^{5}$ https://spacy.io/usage/models
(a) EN
(b) ES
Figure 2: Gender statistics of MLBs dataset for different occupations where each occupation has at least 200 instances. X-axis here stands for the occupation index and y-axis is the number of instances for each occupation. Among all the languages, EN corpus is the most gender balanced one. All the corresponding occupations will be provided in the appendix.
(d) FR
Bias Evaluation
We follow the method in Zhao et al. (2018a) to measure the extrinsic bias: using the performance gap between different gender groups as a metric to evaluate the bias in the MLBs dataset. We split the dataset based on the gender attribute. A gender-agnostic model should have similar performance in each group. To be specific, we use the average performance gap across each occupation in the male and female groups aggregated across all occupations (|Diff| in Table 6) to measure the bias. However, as described in Swinger et al. (2019), people's names are potentially indicative of their genders. To eliminate the influence of names as well as the gender pronouns on the model predictions, we use a "scrubbed" version of the MLBs dataset by removing the names and some gender indicators (e.g., gendered pronouns and prefixes such as "Mr." or "Ms").
To make predictions of the occupations, we adopt the model used in De-Arteaga et al. (2019) by taking the fastText embeddings as the input and encoding the bio text with bi-directional GRU units following by an attention mechanism. The predictions are generated by a softmax layer. We train such models using standard cross-entropy loss and keep the embeddings frozen during the training.
4.2 Characterizing Bias in Multilingual Models
In this section, we analyze the bias in the multilingual word embeddings from the extrinsic perspective. We show that bias exists in cross-lingual transfer learning and the bias in multilingual word embeddings contributes to such bias.
The gender distribution of the MLBs dataset is shown in Fig. 2. Among the three languages, EN corpus is most gender neutral one where the ratio between male and female instances is around
MLBs	Emb.	Avg.	Female	Male	|Diff|
EN	en	82.82	84.69	80.70	7.26
endeb	83.00	84.71	81.06	6.09 ↓
en-es	83.43	85.14	81.51	6.72 ↓
en-de	82.85	84.64	80.84	6.37 ↓
en-fr	82.66	84.34	80.78	5.87 ↓
ES	es	63.83	64.47	63.56	6.56
es-en	61.47	61.42	61.49	7.13 ↑
es-endeb	61.91	62.98	61.45	5.61 ↓
es-de	61.61	62.82	61.11	5.51 ↓
es-fr	62.91	63.31	62.73	4.32 ↓
Table 6: Results on scrubbed MLBs. "Emb." stands for the embeddings used in model training. "Avg.", "Female" and "Male" refer to the overall average accuracy  $(\%)$ , and average accuracy for different genders respectively. " |Diff|" stands for the average absolute accuracy gap between each occupation in the male and female groups aggregated across all the occupations. The results of FR and DE are in the appendix.
1.2 : 1. For all the other languages, male instances are far larger than female ones. In ES, the ratio between male and female is  $2.7:1$ , in DE it is  $3.53:1$ , and in FR, it is  $2.5:1$ ; all are biased towards the male gender.
Bias in Monolingual BiosBias We first evaluate the bias in the MLBs monolingual dataset by predicting the occupations of the bios in each language. From Table 6 we observe that: 1) Bias commonly exists across all languages (  $|\mathrm{Diff}| > 0$ ) when using different aligned embeddings, meaning that the model works differently for male and female groups. 2) When training the model using different aligned embeddings, it does not affect the overall average performance significantly ("Avg." column in the table). 3) The alignment direction influences the bias. On training the model based on the embeddings aligned to different target space, we find that aligning the embeddings to ENDEB
The results of DE and FR are in the appendix.
Trans.	Src.	Tgt.	Avg.	Female	Male	|Diff|
EN→ES	en	es-en	41.68	42.29	41.42	2.83
en-es	es	34.15	33.97	34.22	3.49
ES→EN	es	en-es	57.33	59.61	54.75	8.33
es-en	en	57.05	59.32	54.47	10.13
Table 7: Results of transfer learning on the scrubbed MLBs. "Src." and "Tgt." stand for the embeddings in source model and fine tuning procedure respectively.
Trans.	Src.	Tgt.	Avg.	Female	Male	|Diff|
EN→ES	en	es-en	39.17	41.30	38.70	7.97
en-es	es	35.66	36.11	35.47	4.53
en-de	es-de	34.12	34.46	33.98	4.07
en-fr	es-fr	37.63	38.75	37.16	4.87
ES→EN	es	en-es	58.41	61.78	54.60	9.03
es-en	en	55.62	58.00	52.93	9.52
es-de	en-de	57.98	60.47	55.17	9.13
es-fr	en-fr	55.04	57.85	51.86	8.47
Table 8: Results of transfer learning on gender balanced scrubbed MLBs. The bias in the last column demonstrates that the bias in the multilingual word embeddings also influences bias in transfer learning.
Trans.	Src.	Tgt.	Avg.	Female	Male	Diff
EN→ES	endeb	es-endeb	37.44	39.90	36.40	5.93
ES→EN	es-endeb	endeb	52.51	54.45	50.03	9.06
or a gender-rich language reduces the bias in the downstream task. This is aligned with our previous observation in Section 3.
Bias in Transfer Learning Multilingual word embeddings are widely used in cross-lingual transfer learning (Ruder et al., 2019). In this section, we conduct experiments to understand how the bias in multilingual word embeddings impacts the bias in transfer learning. To do this, we train our model in one language (i.e., source language) and transfer it to another language based on the aligned embeddings obtained in Section 3.2. For the transfer learning, we train the model on the training corpus of the source language and randomly choose  $20\%$  of the dataset from the target language and use them to fine-tune the model. Here, we do not aim at achieving state-of-the-art transfer learning performance but pay more attention to the bias analysis. Table 7 shows that the bias is present when we do the transfer learning regardless of the direction of transfer learning.
Table 9: Bias mitigation results of transfer learning when we aligned the embeddings to the ENDEB space on gender balanced scrubbed MLBs.
MLBs	Avg.	Female	Male	|Diff|
EN	84.35	85.54	83.01	7.31
ES	67.93	65.79	68.82	4.16
DE	72.68	73.68	72.28	4.89
FR	79.18	78.80	79.35	8.75
Table 10: Bias in monolingual MLBs using M-BERT.
Trans.	Avg.	Female	Male	|Diff|
EN→ES	66.56	65.70	66.92	5.48
EN→DE	76.21	75.66	76.42	7.51
EN→FR	76.46	75.73	76.81	8.97
Table 11: Bias in MLBs using M-BERT when transferring from EN to other languages. Comparing to multilingual word embeddings, M-BERT achieves better transfer performance on the MLBs dataset across different languages. But the bias can be higher comparing to the multilingual word embeddings.
Bias from Multilingual Word Embeddings The transfer learning bias in Table 7 is a combined consequence of both corpus bias and the multilingual word embedding bias. To better understand the influence of the bias in multilingual word embeddings on the transfer learning, we make the training corpus gender balanced for each occupation by upsampling to approximately make the model free of the corpus bias. We then test the bias for different languages with differently aligned embeddings. The results are shown in Table 8. When we adopt the embeddings aligned to gender-rich languages, we could reduce the bias in the transfer learning, whereas adopting the embeddings aligned to EN results in an increased bias.
Bias after Mitigation Inspired by the method in Zhao et al. (2018a), we mitigate the bias in the downstream tasks by adopting the bias-mitigated word embeddings. To get the less biased multilingual word embeddings, we align other embeddings to the ENDEB space previously obtained in Section 3. Table 9 demonstrates that by adopting such less biased embeddings, we can reduce the bias in transfer learning. Comparing to Table 8, aligning the embeddings to a gender-rich language achieves better bias mitigation and, at the same time, remains the overall performance.
4.3 Bias Analysis Using Contextualized Embeddings
Contextualized embeddings such as ELMo (Peters et al., 2018), BERT (Devlin et al., 2018) and XLNet (Yang et al., 2019) have shown significant performance improvement in various NLP applica
As there are fewer examples in DE, we use the whole datasets for transfer learning.
tions. Multilingual BERT (M-BERT) has shown its great ability for the transfer learning. As M-BERT provides one single language model trained on multiple languages, there is no longer a need for alignment procedure. In this section, we analyze the bias in monolingual MLBs dataset as well as in transfer learning by replacing the fastText embeddings with M-BERT embeddings. Similar to previous experiments, we train the model on the English dataset and transfer to other languages. Table 10 and 11 summarizes our results: comparing to results by fastText embeddings in Table 6, M-BERT improves the performance on monolingual MLBs dataset as well as the transfer learning tasks. When it comes to the bias, using M-BERT gets similar or lower bias in the monolingual datasets, but sometimes achieves higher bias than the multilingual word embeddings in transfer learning tasks such as the EN  $\rightarrow$  ES (in Table 7).
Recently bias in embeddings has attracted much attention. However, most of the work only focuses on English corpora and little is known about the bias in multilingual embeddings. In this work, we build different metrics and datasets to analyze gender bias in the multilingual embeddings from both the intrinsic and extrinsic perspectives. We show that gender bias commonly exists across different languages and the alignment target for generating multilingual word embeddings also affects such bias. In practice, we can choose the embeddings aligned to a gender-rich language to reduce the bias.
However, due to the limitation of available resources, this study is limited to the European languages. We hope this study can work as a foundation to motivate future research about the analysis and mitigation of bias in multilingual embeddings. We encourage researchers to look at languages with different grammatical gender (such as Czech and Slovak) and propose new methods to reduce the bias in multilingual embeddings as well as in cross-lingual transfer learning.
This work was supported in part by NSF Grant IIS-1927554. We would like to thank Maria De-Arteaga and Andi Peng for the helpful discussion, and thank all the reviewers for their feedback.
Wasi Ahmad, Zhisong Zhang, Xuezhe Ma, Eduard Hovy, Kai-Wei Chang, and Nanyun Peng. 2019a. On difficulties of cross-lingual transfer with order differences: A case study on dependency parsing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2440-2452.
Wasi Uddin Ahmad, Zhisong Zhang, Xuezhe Ma, Kai-Wei Chang, and Nanyun Peng. 2019b. Cross-lingual dependency parsing with unlabeled auxiliary languages. In Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 372-382.
Waleed Ammar, George Mulcaire, Yulia Tsvetkov, Guillaume Lample, Chris Dyer, and Noah A Smith. 2016. Massively multilingual word embeddings. arXiv preprint arXiv:1602.01925.
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135-146.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems, pages 4349-4357.
Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183-186.
Xilun Chen, Ahmed Hassan, Hany Hassan, Wei Wang, and Claire Cardie. 2019. Multi-source cross-lingual model transfer: Learning what to share. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3098-3112.
Alexis Conneau, Guillaume Lample, Marc'Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2017. Word translation without parallel data. arXiv preprint arXiv:1710.04087.
Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 120-128. ACM.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Long Duong, Trevor Cohn, Steven Bird, and Paul Cook. 2015. Low resource dependency parsing: Crosslingual parameter sharing in a neural network parser. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 845-850.
Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 609-614.
Jiang Guo, Wanxiang Che, David Yarowsky, Haifeng Wang, and Ting Liu. 2016. A representation learning framework for multi-source transfer parsing. In Thirtieth AAAI Conference on Artificial Intelligence.
Armand Joulin, Piotr Bojanowski, Tomas Mikolov, Hervé Jégou, and Edouard Grave. 2018. Loss in translation: Learning bilingual word mapping with a retrieval criterion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.
Svetlana Kiritchenko and Saif M Mohammad. 2018. Examining gender and race bias in two hundred sentiment analysis systems. *NAACL HLT* 2018, page 43.
Keita Kurita, Nidhi Vyas, Ayush Parek, Alan W Black, and Yulia Tsvetkov. 2019. Quantifying social biases in contextual word representations. In 1st ACL Workshop on Gender Bias for Natural Language Processing.
Anne Lauscher and Goran Glavaš. 2019. Are we consistently biased? multidimensional analysis of biases in distributional word vectors. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM* 2019), pages 85-91.
Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, and Alan W Black. 2019. Black is to criminal as Caucasian is to police: Detecting and removing multiclass bias in word embeddings. pages 615-621.
Katherine McCurdy and Oguz Serbeti. 2017. Grammatical gender associations outweigh topical gender bias in crosslinguistic word embeddings. Proceedings of WiNLP.
Ninareh Mehrabi, Thamme Gowda, Fred Morstatter, Nanyun Peng, and Aram Galstyan. 2019. Man is to person as woman is to location: Measuring gender bias in named entity recognition. arXiv preprint arXiv:1910.10872.
Tao Meng, Nanyun Peng, and Kai-Wei Chang. 2019. Target language-aware constrained inference for cross-lingual dependency parsing. In Proceedings of the 2019 Conference on Empirical Methods in
Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1117-1128.
Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168.
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2227-2237.
Sebastian Ruder, Ivan Vulic, and Anders Søgaard. 2019. A survey of cross-lingual word embedding models. Journal of Artificial Intelligence Research, 65(1):569-630.
Rachel Rudinger, Chandler May, and Benjamin Van Durme. 2017. Social bias in elicited natural language inferences. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 74-79.
Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 8-14.
Nathaniel Swinger, Maria De-Arteaga, Neil Thomas Heffernan IV, Mark DM Leiserson, and Adam Tauman Kalai. 2019. What are the biases in my word embedding? In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 305-311. ACM.
Chao Xing, Dong Wang, Chao Liu, and Yiye Lin. 2015. Normalized word embedding and orthogonal transform for bilingual word translation. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1006-1011.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems, pages 5754-5764.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Coterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 629-634.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of
the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 15-20.
Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning gender-neutral word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847–4853.
Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, and Kai-Wei Chang. 2019. Examining gender bias in languages with grammatical gender. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5279-5287.
A.1 Multilingual Word Embeddings Alignment
We use the default hyperparameters in the RC-SLS alignment model (https://github.com/facebookresearch/fastText) but change batch size to 5000 and set "sgd" to true to make sure the batch size is used. The "maxsup" is set to the same as "maxneg" with 200000.
A.2 Intrinsic Bias Analysis
Category	Target
EN	ES	DE	FR
Strong-gendered	0.1138	0.0848	0.0935	0.0833
Weak-gendered	0.0477	0.0400	0.0430	0.0395
Table 12: Bias in EN before and after alignment to different languages for different word categories. For different situation, again we see the bias will reduce when we align the words to gender rich languages.
Category	Target
ENDEB	ES	DE	FR
Strong-gendered	0.0830	0.0683	0.0747	0.0685
Weak-gendered	0.0126	0.0201	0.0269	0.0162
Table 13: Bias in ENDEB before and after alignment to different languages for different word categories. When aligning to a gender rich language, the bias in those strong-gendered words reduces.
A.3 Transfer Learning Setting
For the transfer learning, we filter some occupations that commonly occur across all languages and manually make the distribution of each occupation similar in each language. For each corpus, we use  $60\%$  of the corpus for training,  $20\%$  for validation and  $20\%$  for testing.
MLBs	Emb.	Avg.	Female	Male	|Diff|
DE	de	55.4	59.87	53.63	10.42
de-en	56.88	61.84	54.92	15.41
de-endeb	54.09	55.26	53.63	6.54
de-es	54.46	56.58	53.63	9.51
de-fr	55.8	57.50	55.18	10.43
FR	fr	76.52	76.24	76.65	11.58
fr-en	74.13	74.87	73.79	12.96
fr-endeb	73.92	74.19	73.79	10.84
fr-es	74.57	74.19	74.74	11.23
fr-de	75.11	75.56	74.90	12.07
Table 14: Results on the scrubbed BiosBias dataset in DE and FR.
Trans.	Src.	Tgt.	Avg.	Female	Male	|Diff|
EN→DE	en	de-en	37.55	39.47	36.79	16.52
en-de	es-de	34.57	32.89	35.23	13.58
DE→EN	de	en-de	42.47	45.76	38.77	6.46
de-en	en	38.55	41.25	35.51	7.12
Table 15: Results of transfer learning between EN and DE on MLBs dataset.
A.4 Occupation Lists for MLBs Gender Statistics
We list all the occupations for each language in Fig. 2.
EN: professor, accountant, journalist, architect, photographer, psychologist, teacher, nurse, attorney, software engineer, painter, physician, chiropractor, personal trainer, surgeon, filmmaker, dietitian, dentist, dj, model, composer, poet, comedian, yoga teacher, interior designer, pastor, rapper, paralegal
ES: student, model, teacher, cook, musician, artist, painter, professor, administrator, scientist, writer, nurse, hotelier, lawyer, coach, computer-programmer, doctor, journalist, architect, soldier, pharmacist, poet, dancer, engineer, farmer, pianist, pilot, psychologist, surgeon, athlete, mechanic, driver, accountant, rapper, photographer, filmmaker, attorney, physician, dj, comedian, composer
DE: journalist, teacher, psychologist, attorney, dj, photographer, nurse, professor, pastor, architect, filmmaker, composer, painter, software engineer
FR: filmmaker, teacher, composer, painter, journalist, physician, attorney, poet, photographer, pastor, rapper, architect, dj, comedian, psychologist, accountant, nurse, model, surgeon, dietitian
A.5 Extrinsic Bias Results in DE and FR
We show the bias in monolingual DE and FR datasets in Table 14 and in the transfer learning
Trans.	Src.	Tgt.	Avg.	Female	Male	|Diff|
EN→FR	en	fr-en	41.43	41.03	41.62	5.96
en-fr	en	43.12	44.96	42.26	8.33
FR→EN	fr	en-fr	57.81	62.02	51.94	9.79
fr-en	fr	55.15	58.83	50.0	8.3
between EN and them in Table 15 and 16 respectively.
Table 17 and 18 is the bias result of the transfer learning between EN and DE, FR when we manually make the gender ratio balanced for each occupation in the corpus. We also show the mitigation results when we align all the embeddings to the ENDEB space.
Table 16: Results of transfer learning between EN and FR on MLBs dataset.
Trans.	Src.	Tgt.	Avg.	Female	Male	|Diff|
EN→DE	en	de-en	39.40	38.28	39.82	10.65
endeb	de-endeb	33.51	31.37	34.42	8.9
en-es	de-es	33.16	32.21	33.50	9.31
en-de	de	33.96	31.02	35.03	9.13
en-fr	de-fr	38.31	34.17	39.82	11.04
DE→EN	de	en-de	46.43	48.83	43.72	7.93
de-en	en	50.48	53.91	46.58	8.10
de-endeb	endeb	44.44	46.84	41.73	7.16
de-es	en-es	44.04	47.54	40.09	7.29
de-fr	en-fr	46.01	47.57	44.25	7.03