Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniques
Authors: Manon Reusens
Philipp Borchert
Margot Mieskes
Jochen De Weerdt
Bart Baesens
Venue: EMNLP 2023 Main (2023)
Keywords: bias mitigation
cross-lingual transferability
multilingual BERT
TL;DR: This paper shows that cross-lingual transfer of debiasing is possible and yields promising results.

Abstract: This paper investigates the transferability of debiasing techniques across different languages within multilingual models. We examine the applicability of these techniques in English, French, German, and Dutch. Using multilingual BERT (mBERT), we demonstrate that cross-lingual transfer of debiasing techniques is not only feasible but also yields promising results. Surprisingly, our findings reveal no performance disadvantages when applying these techniques to non-English languages. Using translations of the CrowS-Pairs dataset, our analysis identifies SentenceDebias as the best technique across different languages, reducing bias in mBERT by an average of 13%. We also find that debiasing techniques with additional pretraining exhibit enhanced cross-lingual effectiveness for the languages included in the analyses, particularly in lower-resource languages. These novel insights contribute to a deeper understanding of bias mitigation in multilingual language models and provide practical guidance for debiasing techniques in different language contexts.

Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniques
Manon Reusens', Philipp Borchert Margot Mieskes", Jochen De Weerdt Bart Baesens 1,4
Research Centre for Information Systems Engineering (LIRIS), KU Leuven IESEG School of Management, 3 Rue de la Digue, 59000 Lille, France
3 University of Applied Sciences Darmstadt
"Department of Decision Analytics and Risk, University of Southampton mmanon.reusens, borcher,, joccheeed.jcchh...eeeeee jochen.deweerde bart.baesens) | @ @kuleuven.be margot.mieskes @ h-da.de
This paper contains explicit state- ments that are potentially offensive. Abstract
DISCLAIMER:
Debissing
Prebubility
Language
English
Sher 20:46
He:
This paper investigates the transferability of de- biasing techniques across different languages within multilingual models. We examine the applicability of these techniques in English, French, German, and Dutch. Using multilin- gual BERT (mBERT), we demonstrate that cross-lingual transfer of debiasing techniques not only feasible but also yields promising results. Surprisingly. findings reveal no per- formance disadvantages when applying these techniques non -ongllilshh languages. Using translations of the CrowS dataset, our analysis identifies SentenceDebias as the best technique across different languages, reducing bias in mBERT by average of 13%. We also find that debiasing techniques with additional pretraining exhibit enhanced cross- cross-lingual ef- fectiveness for the languages included in the analyses, particularly
Freuch
-0,33
She was always nagging
was always nagging.
Ho. -0.00
Dutch
lel
Figure 1: The example of the English CrowS-Pairs dataset illustrates sentence probabilities after debias- ing mBERT with SentenceDebias English, French. German, and Dutch.
and projection- based methods that assume a bias direction (Liang et al., 2020a; Ravfogel et al., 2020; Liang et al., 2020b).
lower-resource lan- guages. These novel insights contribute deeper understanding of bias mitigation in mul- tilingual language models and provide practical guidance for debiasing techniques in different language contexts.
While debiasing techniques have been developed and evaluated for monolingual, and mostly En- glish models, the effectiveness and transferability of these techniques to diverse languages within multilingual models remain largely unexplored (Stanczak and Augenstein, 2021; Sun et al., 2019). Our research aims to bridge this gap by examin- ing the potential of debiasing techniques applied to one language effectively mitigate bias in other languages within multilingual large language mod- cls. We examine English (EN), French (FR), Ger- man (DE), and Dutch (NL). Figure illustrates an example sentence pair included in the English CrowS- S-Parrs dataset 1 where the unmodified and modified parts are highlighted in blue and yellow respectively. It shows the predicted probabilities of
1 Introduction
There has been a growing interest in addressing bias detection and mitigation Natural Language Processing (NLP) due to their societal implica- tions. Initially, research focused on debiasing word embeddings (Bolukbasi et al., 2016; Zhao et al., 2018b), but recent studies found that pretrained language models also capture social biases present in training data (Meade et al., 2022). Hence, atten- tion has shifted towards debiasing techniques that target sentence representations. These techniques include additional pretraining steps (Zhao et al., 2019; Webster et al., 2020; Zmigrod et al., 2019)
'This example assumes gender to be binary. We acknow]- edge that this fails to capture the full range of gender identitics.

---

the modified part occurring given the unmodified part across different debiasing languages.
niques were benchmarked and their effectiveness demonstrated on BERT for gender, race, and reli- gion, we focus on these three types of bias. Névéol et al. (2022) translated the dataset in French. To the best of our knowledge, there are currently no peer- reviewed variants of CrowS-Pain available in other languages. Therefore, we used three samples of the full dataset and translated them into the respective language to evaluate our experiments.
This study examines the cross-lingual trans- ferability of debiasing techniques using mBERT. mBERT, trained on Wikipedia data from diverse languages, possesses the capability to process and generate text in various linguistic contexts. De- spite balaneing efforts, it still performs worse on low-resource languages (Wu and Dredze, 2020; Devlin, 2018). We investigate whether this perfor- mance disparity extends to gender, religious, and racial biases. Related studies demonstrate the effec- tiveness of cross-lingual debiasing for individual techniques and selected bias scopes (Liang et al., 2020b; Lauscher et al., 2021). We show how to re- duce bias in mBERT across different languages by conducting a benchmark of state-of-the-arr (SOTA) debiasing techniques and providing guidance on its implementation. To facilitate further research and reproducibility, we make the code and additional data available to the research community?.
To create an evaluation set for our experiments, we started from the English CrowS-Pairs dataset (Nangia et al., 2020). We randomly sampled N instances, where N E {20, 30, 40, 50). and mea- sured the performance differences on mBERT and BERT. Through three random seeds, we found that a sample size of 40 resulted in an average perfor- mance correlation of more than 75% with the full dataset for both models. Thus, we conclude that us- ing 40 instances with three random seeds provides representative dataset for our evaluation. Further details are shown in Appendix A. Subsequently, we included the translated samples from each language into dataset, either the corresponding sentences from the French CrowS-Pain or a translation.
Our contributions can be summarized as follows: 1) We provide benchmark of different SOTA de- biasing techniques across multiple languages in a multilingual large language model. 2) We find that SentenceDebias the most effective for cross- lingual debiasing, reducing the bias in mBERT by 13%. 3) We provide implementation guidelines for debiasing multilingual models and highlight the differences in the cross-lingual transfcrability of different debiasing techniques. We find that most projection- based techniques applied to one language yield similar predictions across evalua- tion languages. We also recommend performing the techniques with an additional pretraining step on the lowest resource language within the multi- lingual model for optimal results.
2.2 Debiasing techniques
Next, the different debiasing techniques are ex- plained. For more information on the attribute lists used, we refer to Appendix B.
Counterfactual Data Augmentation (CDA) is a debiasing technique that trains the model on an augmented training set (Zhao et al., 2019; Webster et al., 2020; Zmigrod et al., 2019). First, the cor- pus is augmented by duplicating sentences that in- clude words from predefined attribute list. Next, counterfactual sentences generated by swap- ping these attribute words with other variants in the list, for example, swapping he by she. We augment 10% of the Wikipedia corpus of the re- spective language and use an additional pretraining step to debias the model for three random seeds and average the results.
2 Methodology
This section introduces the data, debiasing tech- niques, and experimental setup respectively.
2.1 CrowS-Pairs
Dropout Regularization (DO) is introduced by Webster al. (2020) as a debiasing technique by implementing an additional pretraining step. We execute this pretraining step while training on 10% of the Wikipedia corpus of the respective language using three random seeds and averaging the results. SentenceDebias (SenDeb) introduced by Liang et al. (2020a) is a projection- based debiasing technique extending debiasing word embeddings
CrowS-Pairs is a benchmark dataset comprising 1508 examples that address stereotypes associated with historically disadvantaged groups in the US. encompassing various types of bias, such age and religion (Nangia et al., 2020). Following Meade et al. (2022), where different debiasing tech-
htt Poattps///iiie http//gttttttttte/eeeeeeteeeeebt http///ttuu..c//mooooooooe/ reusens/ multillngual_bbig

---

(Bolukbasi al., 2016) to sentence representa- tions. Attribute words from a predefined list are contextualized by retrieving their occurrences from a corpus and augmented with CDA. Next, the bias subspace is computed using the representations of these sentences through principal component anal- ysis (PCA). The first K dimensions of PCA are assumed to define the bias subspace they capture the principle directions of variation of the repre- sentations. We debias the last hidden state of the mBERT model and implement SenDeb using 2.5% of the Wikipedia text in the respective language.
of sentences where the model prefers the more bi- ased sentence over the less biased sentence, with an optimal performance of 50%. All experiments are performed on PIO0 GPUs.
Results
Table 1 shows the performance of the different debiasing techniques when debiasing in English in terms of the absolute deviation of the ideal unbiased model. This is an average score for all bias types and models trained for the respective evaluation language. Base represents the score that is achieved by mBERT on the respective evaluation language dataset before debiasing. More results are shown in Appendices C and D.
Iterative Nullspace Projection (INLP) is projection-based debiasing technique in which mul- tiple linear classifiers are trained predict biases, such as gender, that are to be removed from the sen- tence representations (Ravfogel et al., 2020). After training a single classifier, the representations are debiased by projecting them onto the learned linear classifier's iss weight matrix to gather the rowspace projection. We implement this technique using the 2.5% of the Wikipedia text in each language.
As shown in Table 1, English is relatively unbi- ased compared to the other languages and shows a small bias increase after debiasing. This observa- tion aligns with the findings of Ahn and Oh 12,21), who propose mBERT as a debiasing technique. In cases where the initial bias score is already close to the optimal level, further debiasing can lead to overcompensation, consequently amplifying the to- tal bias. We assume that unbiased model should equally prioritize both biased and unbiased sen- tences. However, when debiasing techniques tend to overcorrect, they skew the balance towards fa- voring the prediction of unbiased sentences over biased ones. Addressing this challenge necessitates the adoption of specialized techniques to effectively mitigate any residual bias.
DensRay (DR) is a inricction-based debias- ing technique first implemented by (Dufter and Schlitze, 2019) and adapted for contextualized word embeddings by (Liang et al., 2020b). This technique is similar SenDeb, but the bias direc- tion is calculated differently. This method aims to find an optimal orthogonal matrix so that the first K dimensions correlate well with the linguistic fea- tures in the rotated space. The second dimension is assumed to be orthogonal to the first one. The bias direction is considered to correspond to the cigenvector of the highest eigenvalue of the matrix. DR only implemented for binary bias type and using it for multiclass bias types requires modify- ing the technique. Therefore, we only apply it to the gender bias type. We implement DR debiasing the last hidden state mBERT and using 2.5% of the Wikipedia text in the respective language.
This phenomenon overcompensation occurs in several underperforming techniques, as illus- trated in Table 1. Notably, we find instances of overcompensation for gender when debiasing us- ing INLP for French and using CDA for German, as well as for race when debiasing using DO for German. Another contributing factor to the poor performance of certain techniques within specific debiasing and evaluation language combinations lies in the inherent ineffectiveness of the debiasing method itself, exemplified by the cases of gender debiasing using CDA for French and religion de- biasing using CDA for German. In Tables 5, 6, and 7, we find overcompensation for gender when debiasing with INLP in German and French, evalu- ating in German, debiasing with Sendeb and DR in French, and evaluating in French, as well when debiasing in Dutch with INLP and evaluating in French. Moreover, overcompensation for race is also observed when debiasing with CDA in French
2.3 Experimental setup
We debias mBERT using language X and evaluating it on language Y with X,Y E (EN, FR, DE, NT). In essence, we debiased the model using one language and evaluated it on another, covering all language combinations in our experiments. We implement mBERT in its base configuration (uncased, 12 layers, 768 hidden size) and utilize the bias score as implemented Meade et al. (2022). This metric evaluates the percentage

---

Base
INLP
Sendeb
DR
CDA
DO
Evalosiin langugge
Best debiasing langiage
want debossng language
EN
6.11
8.70 7.
1.78
6,34 F
13.43
8.70
English
Dutch
French
FR
11111
11.20
10
10.28 4
12.6
9,44
Franch
DE
7.52
L
10.75
9.75
German
ML
17.66
13.96
15.14
16.54 0
16.84
17:40
Dunch
French
Faglish
Table 2: Overview best- and pors--perfonning debias- ing languages per evaluation language.
Table I: Overall performance score per evaluation lan- guage and debiasing technique averaged over the three random seeds after debiasing in English.
milac mLl w2x wix
and evaluating in German.
Is cross-lingual transferability of debiasing techniques possible? Table shows that cross- lingual transfer is possible using English as debi- asing language. Figure 2 confirms this, depicting the bias scores averaged over all debiasing tech- niques. As discussed, for English, these techniques increase the bias contained in the model due to its already close to optimal performance. For the other evaluation languages, we find better performance after debiasing. Therefore, we conclude that for these four languages, it is possible to debias the mBERT model to some extent using different de- biasing language, except when the bias contained in the model is already relatively low.
PN
FR
DF
N0.
Praitation bingsage
Figure 2: Average bias scores per evaluation and debi- asing language.
compensated. For English, both racial and gender bias are overcompensated. The German evaluation shows lower overall performance due to already two ineffective methods (INLP and CDA), which were also due to overcompensating racial bias. Fi- nally, for Dutch, we find that debiasing with French overcompensates gender bias less than Dutch and, therefore, is the best-performinn method. As Dutch has the second highest gender bias score before de- biasing, it also benefits from strong debiasing and therefore both French and Dutch perform well.
To shed some light on the insights that can be gathered from Figure 2. Table offers an overview of the best- and worst-perfomne worst-performing techniques per evaluation language. As shown, Dutch is found to be the best debiasing language for English. This is because this debiasing language has shown to overcompensate the gender bias category the least, therefore, resulting in the best performance. In general, find that using the same debiasing language as evaluation language often results in an overcompensation of the bias, therefore turn- ing around the bias direction. This means that the best- best-performing debiasing language is often not the same the evaluation language. However, Ger- man is the exception. As this language already has the highest bias score for gender before debiasing, strong debiasing is beneficial and therefore does not result in overeompensation. Besides German being the best-performing debiasing language for German, it also shows the best performance for French because it achieves the best performance on all different evaluation sets. Moreover, it also shows less overcompensation for the gender bias present in the model than other languages such as Dutch.
We believe that these results influenced by the fact that both German and French have a gram- matical gender distinction, which may impact debi- asing gender to a greater extent. This grammatical gender distinction is not embedded in English and Dutch. Moreover, as the religion category regu- larly shows limited bias decrease, we find that the performance in the gender and race category often determines whether technique works well or not. How are the different techniques affected by cross-lingual debiasing? Table 3 shows the over- all percentage increase of the bias score per tech- nique. From this, we conclude that SenDeb is the best-performing technique and reduces bias in mBERT on average by 13%. DO is the second best- performing method reducing bias on average by 10%. However, Figure shows that DO performs well for all debiasing languages except English, while SenDeb performs consistently well for all languages. The other techniques perform worse overall. Hence, we suggest using SenDeb as cross- lingual debiasing technique for these languages.
French is the worst-performing debiasing lan- guage for evaluation languages except for Dutch, where it is the best-perfone perstteerfrning one. We find that when evaluating in French, the gender bias is over-

---

INLP
Sendch
DR
CDA
Dropout
L.IT
13.21
-0.16
10.21
Table 3: Average percentage difference in bias scores of each technique compared to the base model.
WINUP woodd #DX #CDA Diogpet:
Doloning Hol-iie
Figure 4: Average bias score per debiasing languages for both CDA and DO.
in mBERT for gender. Similarly, Zhao al. (2020)
analyze the cross-lingual transfer of gender bias mitigation using one method. Lauscher et al. (2021) also find that their proposed technique, ADELE, can transfer debiasing across six languages. Other studies analyze biases contained in multilingual language models. Kaneko et al. (2022) evaluate bias across multiple languages in masked language models using new metric. Ahn and Oh (2021) study ethnic bias and its variability over languages proposing mBERT debiasing technique. Finally, some studies also explore the cross- lingull trans- ferability of downstream tasks (Levy al., 2023).
Figure 3: Average bias score per technique per debiasing language.
When zooming in the projection-based tech- niques, i.c. INLP, SenDeb, and DR, high perfor- mance variation shown in Table and Figure While SenDeb offers consistent performance for all different debiasing languages, we see more vari- ation and a lower bias decrease for INLP. This is due to the high variation in performance, resulting in higher overall average. As INLP uses multi- ple linear classifiers to define the projection matrix, high variability introduced. Since DR was only implemented for gender, no performance gains can be obtained from the other bias types, therefore resulting in a lower overall performance increase. Techniques using an additional pretraining step obtain the best results when debiasing in Dutch, as illustrated in Figure 4. Notably, Dutch is the lowest resource language of these four lan- guages during pretraining (Wu and Dredze, 2020). This additional pretraining step lets the model learn unbiased associations between words while becom- ing familiar with the lower-resource language re- sulting in lower overall bias. Therefore, we con- clude that, for our set of languages, these tech- niques most effective when applied to low- resource languages.
5 Conclusion
Most studies focus on debiasing techniques for large language models, but rarely explore their cross-lingual transferability. Therefore, offer benchmark for SOTA debiasing techniques on mBERT across multiple languages (EN, FR, DE, NL) and show that debiasing transferable across languages, yielding promising results. We provide guidance for cross-lingual debiasing, highlighting SenDeb as the best-performing method, reducing bias in mBERT by 13%. Additionally, we find that, for the studied languages, debiasing with the lowest resource language is effective for techniques involy- ing an additional training step (CDA and DO). This research is first step into the cross- cross-lingual trans- ferability of debiasing techniques. Further studies should include languages from different cultures and other multilingual large language models to assess generalizability.
4 Related work
Significant research focuses on the linssll performance of mBERT (Wu and Dredze, 2020; Pires et al., 2019; Libovický al., 2019). Limited research focuses on the cross-lingual transferability of debiasing techniques in mBERT (Stanezak and Augenstein, 2021; Sun et al., 2019). Liang et al. (2020b) use DensRay in English to debias Chinese
Limitations
A first limitation concerns the analysis focused on four closely related languages from similar culture. broader range of languages should be explored to ensure the generalizability of the find- ings. Our research was conducted employing a

---

single multilingual model, mBERT. Extending this other multilingual language models would pro- vide valuable insights into the wider applicability of the results. Moreover, the evaluation of out- comes relied primarily on the CrowS-Pairs met- ric, although efforts were made to enhance the understanding by examining the absolute differ- ence compared to the optimal model. Next, the consideration of gender was limited binary clas- sification, overlooking non-binary gender identities. This should be further addressed in future research. Furthermore, a comprehensive multilingual dataset to assess stereotypes across different languages is not available, and thus, the English CrowS-Pairs dataset was translated and corresponding sentences of the French dataset were used. Nevertheless, typ- ical stereotypes for other languages were not ade- quately represented. Furthermore, the dataset used in the study exhibited certain flaws highlighted by Blodgett et al. (2021), such as the influence of se- lected names on predictions, which was observed to have a significant impact. This needs to be in- vestigated further. Additionally, attribute lists for languages other than English were not available to the same extent. We tried to compile lists for French, Dutch, and German, excluding words with multiple meanings to minimize noise. However, our lists were not exhaustive, and therefore the omission of relevant attributes is possible. is also worth noting that, in certain cases, the generic mas- culine form was considered the preferred answer, despite it being included in the attribute lists. Fi- nally, the applicability of downstream tasks should be investigated (e.g. (Levy et al., 2023)). Hence, future research should encompass wider language scope, include multiple models, address existing dataset flaws, and develop more comprehensive attribute lists for various languages.
efforts to debias the multilingual large language model, it important note that not all forms of bias are completely mitigated. The applied debi- asing techniques do lower the bias present in the model, however, there is still bias present in the model both within and outside the targeted bias types. Finally, we recognize that our evaluation datasets do not encompass all the different biases that might be present in the model. Therefore, even if model would obtain a perfect score, is still possible that other forms of bias persist.
Acknowledgements
This research was funded by the Statistics Flanders research cooperation agreement on Data Science for Official Statistics. The resources and services used in this work were provided by the VSC (Flem- ish Supercomputer Center).
References
Jaimeen Ahn and Alice Oh. 2021. Mitigating language- dependent ethnic bias in BERT. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 533-549, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. 2021. Stercotyping Norwegian salmon: An inventory of pitfalls in fair- ness benchmark datasets. Annual Meeting of the Association for Computational Linguisties and the 11th International Joint Confer- ence on Natural Language Processing (Volume 1: Long Papers), pages 1004-1015, Online. Association for Computational Linguistics.
Proceedings of the 59th
Tolga Bolukbasi, Kai- Cai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 4356-4364.
Ethics Statement
We would like to address three key ethical consid- erations in this study that highlight ongoing chal- lenges and complexities associated with mitigating bias in large language models. First, it is important to acknowledge that the gender bias examined in this paper is approached from binary perspective. However, this does not capture the full range of gen- der identities present in reality. While we recognize this limitation, was necessary to simplify the anal- ysis for experimental purposes. In future research, we hope to address this limitation. Second, despite
Jacob Devlin. 2018. Multilingual bert readme docu- ment.
Philipp Dufter and Hinrich Schütze. 2019. Analytical methods for interpretable ultradense word embed- dings. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCN) 1185-1191, Hong Kong, China. Association for Com- putational Linguistics.
pages
Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki. 2022. Gender bias in meta-embeddings.

---

In Findings of the Association for Computational Linguistics: EMNLP 2022, pages $118-3133, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Telmo Pires, Eva Schlinger. and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceed- ings of the 57th Anmual Meeting of the Association for Computational Linguistics. pages 4996-5001, Flo- rence, Italy. Association for Computational Linguis- tics.
Anne Lauscher, Tobias Lucken, and Goran Glavaš. 2021. Sustainable modular debiasing of language models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4782-4799 1782-4797,11 Punta Cana, Dominican Republic. Association for Compu- tational Linguistics.
Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldherg. 2020. Null out: Guard- ing protected attributes by iterative nullspace projec- tion. In Proceedings of the 58th Annual Meefing of the Association for Computational Linguistics, pages 7237-7256, Online. Association for Computational Linguisties.
Sharon Levy, Neha Anna John, Ling Liu, Yogarshi Vyas, Jie Ma, Yoshinari Fujinuma, Miguel Balles- teros, Vittorio Castelli, and Dan Roth. 2023. Com- paring biases and the impact of multilingual train- ing across multiple languages. arXiv preprint arXiv:2305.11242
Karolina Stanezak and Isabelle Augenstein. 2021. A survey on gender bias in natural language processing. arXiv preprint arXiv: 212.14688.
Tony Sun. Andrew Gaut, Shirlyn Tang. Yuxin Huang. Mai EISherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai- Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. Proceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics, pages 1630-1640, Florence, Italy. Association for Computational Linguistics.
Paul Pu Liang. Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis- Philipppe Morency. 2020a. Towards debiasing sen- tence representations. In Proceddings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 5502-5515, Online. Association for Computational Linguistics.
Sheng Liang. Philipp Dufter, and Hinrich Schüitze. 2020b. Monolingual and multilingual reduction of gender bias in contextualized representations. In Proceedings of the 28th International Conference on Computational Linguistics. pages 5082-5093, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint arXiv:2010.06032.
Shijie Wu and Mark Dredze. 2020. Are all languages created equal in multilingual BERT? In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 120-130, Online. Association for Com- putational Linguistics.
Jindfich Libovický, Rudolf Rosa, and Alexander Fraser. 2019. How lantuale is multilingual bert? arXiv preprint a..01:1911.0311..
Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai- Kai-Wei Chang. and Ahmed Hassan Awadallah. 2020. Gender bias in multilingual embeddings and cross- lingual transfer. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guisties, pages 2896-2907, Online. Association for Computational Linguisties.
Nicholas Meade, Elinor Poole- Poole-Dayan, and Siva Reddy. 2022. An empirical survey of the effectiveness of debiasing techniques for pre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume
1: Long Papers), pages 1878 -1988 Dublin, Ireland. Association for Computational Linguistics.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell. Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceed- ings of the 2019 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume (Long and Short Papers), pages 229-634, Min- neapolis, Minnesota. Association for Computational Linguistics.
Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020. CrowS-pairs: A chal- lenge datasct for measuring social biases in masked language models. Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 1953-1967, Online. As- sociation for Computational Linguistics.
Aurélie Névéol, Yoann Dupont, Julien Bezançon, and Karën Fort. 2022. French CrowS-pairs; Extending a challenge dataset for measuring social bias in masked language models to language other than English. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume Long Pupers), pages 8521-8533 Dublin, Ireland. Association for Computational Linguistics.
Jieyu Zhao, Tianlu Wang. Mark Yatskar, Vicente Or- donez, and Kai-Wei Chang. 2018a. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume (Short Pupers), pages 15-20, New

---

Orleans, Louisiana. Association for Computational Linguistics.
Gender:(Zhao et al., 2018a) actor, actress, ac- tors, actresses, airman, airwoman, airmen, air- women, uncle, aunt, uncles, aunts, boy, girl, boys, girls, groom, bride, grooms, brides, brother, sis- ter, brothers, sisters, businessman, businesswoman,, businessmen, businesswomen, chairman, chair- woman, chairmen, chairwomen, dude, chick, dudes, chicks, dad, mom, dads, moms, daddy, mommy, daddies, mommies, son, daughter, sons, daugh- ters. father, mother, fathers, mothers, male, female, males, females, guy, gal, guys, gals, gentleman, lady, gentlemen, ladies, grandson, granddaughter, grandsons, granddaughters, guy, girl, guys, girls, he, she, himself, herself, him, her, his, her, hus- band, wife, husbands, wives, king, queen, kings, queens, lord, lady, lords, ladies, sir, ma' am, man, woman, men, women, sir, miss, mr, mrs., mr. ms., policeman, policewoman, prince, princess, princes, princesses, spokesman, spokeswoman, spokesmen, spokeswomen.
Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang. and Kai- Wei Chang. 2018b. Learning gender-neutral word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Process- ing, pages 4847-4853, Brussels, Belgium. Associa- tion for Computational Linguisties.
Ran Zmigrod, Sabrina J. Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmenta- tion for mitigating gender stereotypes in languages with rich morphology. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1651-1661, Florence, Italy. Asso- ciation for Computational Linguistics.
A Correlation samples and full dataset
Table 4 shows the overall correlation between the performance of the respective model on the full dataset and on the sample over three random seeds. Full represents that we looked at all different bias types in the CrowS-Paie dataset and (3) refers to the three bias types used in this study (gender, re- ligion, and race). We used a threshold of 75% to decide on the sample size.
Race: (Meade et al., 2022) black, caucasian, asian, african, caucasian, asian, black, white, asian, africa, america, asia, africa, america, china, africa, europe, asia.
Religion: (Liang et al., 2020a) jewish, christian, muslim, jews, christians, muslims, torah, bible, quran, synagogue, church, mosque, rabbi, priest, imam, judaism, christianity, islam.
MBERT full
mBERT (3)
BERT full
BERT (3)
size 20
85.71
66.58
74.40
70.58
30
88.54
66.80
80.14
70.66
size 40
93.11
86.02
83.78
75.88
50
90.52
87.32
French
Table 4: Correlation between the sample and full re- spective model's performance over three random seeds before debiasing.
Gender: acteur, actrice, acteurs, actrices, avi- ateur, aviatrice, aviateurs, aviatrices, oncle, tante, oncle, tantine, tonton, tata, tontons, tatas, on- cles, tantes, oncles, tantines, garçon, fille, garçons, filles, marié, mariée, époux, épouse, mariés, mariées, époux, épouses, brother, sister, frère, soeur, frères, soeurs, entrepreneur, entrepreneuse, entrepreneur, entrepreneure, entrepreneurs, en- trepreneures, entrepreneurs, entrepreneuses, prési- dent, présidente, présidents, présidentes, mec, meuf. gamin, gamine, mecs, meufs, père, mère, pères, mères, papa, maman, papas, mamans, fils, fille, fils, filles, abbé, abbesse, abbés, abbesses, masculin, féminin, mâle, femelle, mâles, femelles, gars, fille, gars, filles, monsieur; dame, messieurs, dames, petit-fils, petite-fille, petit-fils, petites-filles, il, elle, lui-même, elle-même, lui, elle, mari, femme, maris, femmes, roi, reine, rois, reines, seigneur, seigneuresse, seigneurs, seigneuresses, monsieur, dame, monsieur, madame, homme, femme, hommes, femmes, monsieur, mademoiselle, mr, min,, mr, mile, policier, policière, prince,
B Attribute Lists
The different lists of attribute words per language used for CDA, SenDeb, INLP, and DR are listed below. We started from the lists in (Meade et al., 2022; Zhao al., 2018a; 8a; Liang al., 2020a) and extended these to other languages. For the non- English lists, we first translated the words in the English attribute list similar relation con- tained in the translated words. Subsequently, we asked a native speaker of the respective language whether there are extra words that contain similar relations which do not exist in English. We added these to the list. However, note that these lists not exhaustive, as many other words still exist that express the same relationship. Nevertheless, this list already provides more extensive attribute list than by only translating the English attribute list. English

---

princesse, princes, princesses, copain, copine, co- pains, copines, ami, amie, amis, amies, voisin, voisine, docteur, doctoresse, docteur, docteure, boulanger, boulangère, héros, héroine, employé, employée, employés, employées, chef, cheffe, chefs, cheffes, cousin, cousine. grand-pêre, grand-mère, expert, experie, pompier, pompière, pompiers, pompières, agriculteur, agricultrice, agriculteurs, agricultrices, travailleur, travailleuse, infirmier, in- firmière infirmiers, infirmières, patron, patronne, patrons, patronnes.
oom, lante, ooms, lantes, nonkel, tante, nonkels, tantes, jongen, meisje, jongens, meisjes, bruide- gom, bruid, bruidegommen, bruiden, broer, zus, broers, zussen, zakenman, zakenvrouw, zakenman- nen, zakenvrouwen, kerel, griet, kerels, grieten, vader, moeder, vaders, moeders, papa, mama, papa's, s, mama's, zoon, dochier, zonen, dochters, man, vrouw, mannen, vrouwen, gast, wijf. gasten, wijven, heer, dame, heren, dames, kleinzoon, klein- dochter, kleinzonen, kleindochters, vent, vrouw, venten, vrouwen, hij, zij. hemzelf. haarzelf, hem, haar, zijn, haar, mannelijk. vrouwelijk, vriend, vriendin, vrienden, vriendinnen, koning, koningin, koningen, koninginnen, heer, dame, heren, dames, meneer, mevrouw, jongeheer, jongedame, jonge- heren, jongedames, jongeheer, juffrouw, jonge- heren, juffrouwen, politieagent, politieagente, prins, prinses, prinsen, prinsessen, woordvoerder, woordvoerster, woordvoerders, woordvoersters, brandweerman, brandweervrouw, brandweermun- nen, brandweervrouwer, timmerman, timmervrouw, timmermannen, timmervrouwen, meester, juf, meesters, juffen, verpleger, verpleegster, verplegers, verpleegsters, bestuurder, bestuurster, bestuur- ders, bestuursters, kuisman, kuisvrouw, kuisman- nen, kuisvrouwen, kok, kokkin, kokken, kokkin- nen, leraar, lerares, directeur, directrice, directeurs, directrices, secretaris, secretaresse, secretaris- sen, secretaressen, boer, boerin, boeren, boerin- nen, held, heldin, gastheer, gastvrouw, gastheren, gastvrouwen, opa, oma, opa's, 's, oma's 's grootvader, grootmoeder, grootvaders, grootmoeders. Race: afrikauns, amerikaans, aziatisch, afrikauns, europees, aziatisch, zwart, blank, aziatisch, afrika, amerika, azië, afrika, amerika, china, afrika, eu- ropa, azië.
Race: noir, blanc, asiatique, black, blanc, asia- tique, noir, caucasien, asiatique, africain, européen, asienne, africain, américain, asiatique, afrique, amérique, asie, afrique, amérique, chine, afrique, europe, asie.
Religion: juif. chrétten, musulman, juifs, chrétiens, musulmans, torah, bible, coran, synagogue, église, mosquée, rabbin, prêtre, imam, judaisme, chris- tianisme, islam.
German
Gender: schauspieler, schauspielerin, koch, köchin, lehrer, lehrerin, schüler, schülerin, student, studentin, pilot, pilotin, onkel, tante, junge, mädchen, bräutigam, braut, bruder, schwester, geschäftsmann, geschäfisfrau, vorsitzender, vorsitzende, vater, mutter, papa, mama, sohn, tochter, mann, frau, kerl, mädel, herr, dame, enkel, enkelin, groBvater, großmutter, cousin, cousine, er, sie, ihm, ihr, sein, ihr, seine, ihre, ehemann, ehefrau, feuerwehrmann, fenerwehrfrau, könig, königin, fürst, fürstin, herzog, herzogin, mann, frau, männer, frauen, hr. ft., polizist, polizistin, prinz, prinzessin, sprecher, sprecherin, kollege, kollegin, mitarbeiter. mitarbeiterin, helfer, helferin, anwalt, anwältin. bauarbeiter, bauarbeiterin, krankempfleger, krankenpflegerin, chef, chefin, vorgesetzter, vorgesetzte, sänger, sängerin, kunde, kundin, besucher, besucherin, freund, freundin, arzt, ärztin, verkäufer, verkäuferin, kanzler, kanzlerin, geschäjtsleiter, geschäftsleiterin, pfleger, pflegerin, kellner, kellnerin. Race: dunkelhäutiig hellhäutig, asiatisch, afrikaner, europäer, asiate, amerikaner, afrika, amerika, asien, china.
Religion: joods, christen, moslim, joden, christe- nen, moslims, thora, bijbel, koran, synagoge, kerk, moskee, rabbijn, priester, imam, jodendom, chris- tendom, islam.
C Averaged results
Tables 5, 6, and 7 show the different averaged re- sults are the debiasing languages other than En- glish, namely French, German, and Dutch.
Religion: jüdisch, christlich, muslimisch, jude, christ, muslim, torah, bibel, koran, synagoge, kirche, moschee, rabbiner, pfarrer, imam, judentum, christentum, islam.
D Breakdown results
In this section, breakdown of the different scores per category shown terms of the bias metric established in (Nangia et al., 2020) in Tables 8, 9, 10, 11. For brevity, we employ the following
Dutch
Gender: acteur, actrice, acteurs, actrices,

---

mBERT
INLP
SenDeb
DR
CDA
DO
mBRIT
INCF
Sedt
DRC
CITA
1X0
6.11
10.93
1
7,50
9.44
9.07
EN
51.11
54.44
50.28
50,00*
52.59
52.13
FR
1111
9.91 6
12.22
11.67 1
10.00
10.74
89.17
89.17
44.17
44.17
43.06
9.33
64-17
NL
17.06
14.96
14.86
16.26
15.05
12.94
60.56
57.5
5083
50.00
50,00
53.89
55.83
Table 5: Overall performance score per evaluation lan- guage and debiasing technique averaged over the three random seeds after debiasing in French.
Ra
90.17
60.56
58.33
44.44
45.85
Re
71.67
61.39
61.17
63.06
59 19.17
DE
59,05
57.64
53.75
55,707
55.72
53.35
G 60.90
5000
56.46
58.41
53.47
35.00
54.17
65.14"
mBERT
INLP
SenDeb
DR
CDA
DO
49.38
48.76
56
EN
6.11
9.44
6.94
7.22
10.19
8.43
Ra
65.83
53.80
64.17
69.44
67.78
11.11
9.17
7.5
10.28
8.43
7.13
Re
80.83
79.72
77.50
75.56
75, 36
9.33
NL
17.66
14.31
14.99
16.25
17.10
16.65
Table 10: Overall performance score per evaluation language and debiasing technique averaged over the three random seeds after debiasing in German.
Table 6: Overall performance per evaluation lan- guage and debiasing technique averaged over the three random seeds after debiasing in German.
mBERT
INL
SenDJch
D3K
CDA
DO
31.11
MBERT
INLP
3023
SeaDes
50.564
DR
CDA
47.69
51.20
DO
6.11
8,00 7
6.94
1.22
7.09
7.30
11.11
11.30
10.28
10.56
9.07
9.17
44.17
9.33
8.71
6.04
5,37
60.00
62.78
6000
51.5
60.55
NL
17.66
14.68
15.71
16.54
14.69
14.41
FR
60.55
60.37
61.28
60.567
55.37
52.50
G
X083
50.28
3035
20.83
35.85
33.89
Ra
59.17
61.44
61.67
44.44
41.11
Table 7: Overall performance score per evaluation lan- guage and debiasing technique averaged over the three random seeds after debiasing in Dutch.
Re
71.67
68.39
68.33
65.83
62.50
59.05
56,251
50.86
60.90
$3.04
54-71
46.47
51
NL 67:65
69.21
63.78
$4,57
61.31
63.07
G 56. 32
$2.92
47.05
47.05
$2.97
53.22
Ra 65.83
59.22
67.50
61.67
57.78
abbreviations: Gender (G), Race (Ra), and Religion (Re).
Re 80.83
80.00
76.67
78.89
80.00
Table 11: Overall performance score per evaluation language and debiasing technique averaged over the three random seeds after debiasing Dutch.
INLP
EN
51.11
31.30
31.11
50.25
35.28
30.37
#9.17
49.17
19.17
46.57
50.83
51.11
Ra
44.17
41.94
45.00
39.72
39.17
Re
60.00
62.78
59.17
75.28
60.83
FR
DB
39.55
55.93
$6.56
56,55
37.06
35.50
60.00
$4.50
51.71
51.42
53.59
56 76.70
Ra ST.07
61.35
57.05
31.51
48.12
Re 59.17
$1.94
25.83
66.39
61.39
NL
78.61
79.17
Table 8: Overall performance score per evaluation lan- guage, debiasing technique, and category averaged over the three random seeds after debiasing in Finglis..
49.17
52.7%
47.50
46.67
40.72
53.06
Ra
44.17
45.83
43.35
46.11
38.80
Re
60.00
68.61
60.00
66.94
63.06
FR
60.55
54.54
60.00
58 80
58.15
$7.59
47.22
47.33
60.90
$2.47
5035
51.71
35.24
55.83
Ra
$7.07
59.91
58.76
$2.05
47.04
Re
59.17
66.67
36.67
61.56
56.39
NI.
67.66
64.31
64.31
65.71A
65.05
62.94
55.12
49.5 59
49.59
50.45
56.32
55.26
Table Overall performance score per evaluation lan- guage and debiasing technique averaged over the three random seeds after debiasing in French.


Reviews:
[{"review_id": "XDWEgT42eY", "reviewer": "Reviewer_pZuG", "summary": "", "questions": "NA.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your time and valuable feedback.  \nWe agree that this paper covers an important research problem in NLP, as most research into debiasing has been done in a monolingual setting (e.g, English). Below, we elaborate on your comments and feedback.\n\n>**R1A: There is still room to improve the analysis and understanding of bias in multilingual models. For example, looking at individual debiasing methods' performance in Table 1 and the appendix show that not all of them are effective in debiasing across languages**  \n\nWe agree that the paper benefits from further analysis providing additional insights into cross-lingual debiasing. Therefore, we include further analyses into the debiasing methods that are not effectively debiasing in the camera-ready version as per below. \nDue to the almost perfect bias score for English in the mBERT, debiasing techniques tend to increase the bias score. Ahn and Oh (2021) confirm this by proposing mBERT as debiasing technique. However, there are also other techniques that do not effectively debias the results. Many of these techniques show bad performance because of **overcompensation** after debiasing the model. We assume that an unbiased model has an equal chance of preferring the biased sentence over the unbiased sentence.  However, when the debiasing technique overcompensates the bias, it makes the prediction of the unbiased sentence more likely than the biased one. We saw this phenomenon occurring in many of the bad-performing techniques shown in Table 1:\n* When debiasing mBERT using **INLP with English and Dutch as debiasing languages and French as evaluation language**, we see that overall bias does not decrease due to gender bias being overcompensated. \n* When debiasing mBERT using **INLP with German as the debiasing language and evaluating the results on German**, we see again that the bias for gender is overcompensated. On top of that, we also find that when debiasing race, the bias increases.\n* When debiasing mBERT using **INLP with French as the debiasing language and evaluating the results in German**, we also find an overcompensation of the gender bias category, combined with an increased bias for religion.\n* When debiasing mBERT using **SenDeb and DR with French as the debiasing language and evaluating the results in French**, we find overcompensation for the gender bias category.\n* When debiasing mBERT using **CDA with French as the debiasing language and evaluating the results in German**, we find an overcompensation of the bias present in the race category.\n* When debiasing mBERT using **DO with English as the debiasing language and evaluating the results in German**, we also find overcompensation of the bias present in the race category and a small increase in bias for religion.\n\n**Actions taken:** The additional analysis is added to the camera-ready version of the paper.  \n**References:** Jaimeen Ahn and Alice Oh. 2021. Mitigating language-dependent ethnic bias in BERT. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 533–549, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.  \n\n>R1B: Moreover, the interesting results in Figure 2 can benefit from further analysis. Based on the results for some languages (e.g., NL), using another language (e.g., DE) is more effective in decreasing the bias.\n\nTo further clarify the figure, we added a **table** to show the best debiasing language for every evaluation language in our study.  This table is discussed in detail in the paper. We highlight the most important findings below. \n\n| **Evaluation Language** | **Best Debiasing Language** | **Worst Debiasing Language**  |\n|---------------------|-------------------------|---------------------------|\n| **English**             | Dutch                   | French                    |\n| **French**              | German                  | French                    |\n| **German**              | German                  | French                    |\n| **Dutch**               | French                  | English                   |  \n\n* **Dutch is shown to be the best-performing debiasing language for English** because this debiasing language overcompensates gender bias the least. \n* In general, using the **same debiasing as evaluation language results in overcompensation of the bias**, turning around the bias direction. Therefore it is the best debiasing language is often not the same as the evaluation language. However, German has the highest bias score for gender before debiasing, and therefore strong debiasing is beneficial. \n* **German is the best-performing debiasing language for French** since it shows the best performance on all different evaluation sets. Moreover, it also shows less overcompensation for the gender bias present in the model than other languages such as Dutch\n* **French is the worst-performing debiasing language for all languages except for Dutch, where it is the best-performing one**. We find that when evaluating in French, the gender bias is overcompensated. For English, both racial and gender bias are overcompensated. The German evaluation shows lower overall performance because of already two ineffective methods (INLP and CDA), which were also due to overcompensating racial bias. Finally, for Dutch, we find that debiasing with French overcompensated gender bias less than Dutch and therefore is the best-performing method.\n\nWe believe that these results are influenced by the fact that both German and French have a grammatical gender distinction, which could have an impact on debiasing gender to a greater extent. This grammatical gender distinction is not the case for English and Dutch.\n\n**Actions taken:** The table and additional explanations are added to the camera-ready version of the paper.\n\n>**R1C: Line 78: an extra '.' .**\n\nWe thank the reviewer for pointing this out and have deleted the extra '.' in the camera-ready version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper studies an important research problem in NLP. While most of the bias analysis and debiasing methods have been done in the monolingual area and mostly in English, this is important extend the research to other languages. ", "reasons_to_reject": "Although the question is clear and the results to some extent support it, there is still room to improve the analysis and understanding of bias in multilingual models. For example, looking at individual debiasing methods' performance in Table 1 and the appendix show that not all of them are effective in debiasing across languages. Moreover, the interesting results in Figure 2 can benefit from further analysis. Based on the results for some languages (e.g., NL), using another language (e.g., DE) is more effective in decreasing the bias. ", "questions_to_authors": "NA.", "paper_topic_and_main_contributions": "The paper studies the effectiveness of existing debiasing techniques in multilingual models.  Using BERT and translating the CrosS-Pairs dataset, the authors apply Counterfactual Data Augmentation (CDA), Dropout Regularization (DO), SentenceDebias (SenDeb), Iterative Nullspace Projection (INLP), and DensRay (DR) debiasing methods for English, French, German, and Dutch languages. The authors evaluate the debiasing methods' performance under a zero-shot setting, where they are applied to a language while being evaluated in other languages. The proposed results suggest that the debiasing techniques are effective in addressing social biases across the selected languages. The exceptional case is English, where applying debiasing methods increases the bias in mBERT.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 78: an extra '.' ."}, "extra_text": {}}, {"review_id": "xJMVKV0vv1", "reviewer": "Reviewer_4hPj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the comments and your perspective on our research and as per your request elaborated  on the experimental setup and model evaluation conducted in the paper.   \n> R2A: The experimental setup boils down to the following sentence: \"We debias mBERT using language X and evaluating it on language Y\" (Line 195). I'm not sure what this means exactly or how to reproduce the results  \n  \nOur paper investigates the **transferability of debiasing techniques across different languages within a multilingual model, mBERT**. We thoroughly investigate the effectiveness of these techniques across four languages: English, French, German, and Dutch. Our findings demonstrate the feasibility of  cross-lingual transfer of debiasing  and show promising results. More specifically, we debias the model  **using one language and subsequently evaluate its performance in another language**. We did this **for all possible combinations of the set of languages** within our research scope. Our paper highlights an important research problem as most research into bias mitigation has been done in a monolingual context, and mostly for English.  \n\n**Actions taken:** Thanks to the reviewer comment, we revised our paper to improve the clarity. An example of this is adding the following sentence for extra clarification regarding the experimental setup: *This means that we debiased the model using one language and then evaluated the results on another language, and this for all different experimental languages.*  \n\n> R2B: furthermore, I couldn't understand how the results on languages other than English were obtained, since the paper itself mentions that the dataset used isn't available in other languages (other than French?).  \n\nAs noted by the reviewer and mentioned in our paper, there is no other translated version available of the CrowS-Pairs dataset except for French. Therefore, we have **indicated in the paper that we have translated samples of the dataset** for the other respective languages. Moreover, we do wish to point out that **all datasets including translations are provided in the supplementary materials** and therefore contribute to the reproducibility of our research.  \n**Actions taken:** We added extra clarification about our experimental setup to line 111: *Therefore, we used three samples of the full dataset and translated them to the respective language to evaluate our experiments.*\n\n> R2C: Following, I think that an analysis of the results would help shed light on what exactly changes when applying debiasing cross-lingualy, e.g., via a manual error analysis  \n\nThank you for the suggestion to manually investigate the model results. While we believe this is interesting to **distill potential linguistic patterns** that impact the bias prediction results, we do not agree with the premise that this manual error analysis will *“shed light on what exactly changes when applying debiasing cross-lingualy”*. During the evaluation, we compare the differences between the predicted probabilities for two tokens. Hence, the evaluation metrics capture the changes in predicted probabilities when applying the debiasing techniques cross-lingually.  \n\nNevertheless, we acknowledge that the paper benefits from **manually investigating examples where cross-lingual bias mitigation is most and least effective**. We randomly sample 10 sentences for each pair of debiasing and evaluation language, where the predicted token probabilities deviated most/least. We include a short analysis of selected samples below.  \n\nUpon analyzing the best-performing sentences, we observe a variety of results present, with gender and race predominately present. **The worst-performing sentences are primarily associated to the religion bias category.** This is in line with the findings in Tables 7- 10, where often the bias for religion shows limited decrease in bias. For all different techniques, we find that more than 70% of the 100 worst-performing sentences are part of the religion category. Therefore, the gender category greatly influences the final result of the debiasing technique. That is why, when looking further into the results of Tables 1, 4, 5, and 6 and Figure 2, we find that often the reason for a technique to not perform well is that it overcompensates the bias for either gender or race when debiasing. Overcompensation occurs when a model is too much debiased, shifting the bias direction to the opposite direction.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The topic and idea of the paper is interesting and may benefit debiasing efforts in low resource languages.", "reasons_to_reject": "* Many details missing - how was the debiasing carried out exactly, how were the evaluations carried out on the different languages.", "questions_to_authors": "", "paper_topic_and_main_contributions": "Edit following author response:\nI think that the author response helped clarify some details. Thank you very much! \nI think that a dedicated methodology section with the details mentioned here. \nI raised my score, but still am missing crucial details - most notably how were the translations obtained? I reread the paper and couldn't find additional details. Were they automatic? manual? Either case would require more elaboration on how this was done. What system / human translated the sentences? what was the quality of the translations?\n\n\n-\n\n\nPopular debiasing techniques are applied on mBERT on the English tokens, and the effect is tested on other languages. \n\nI had a hard time understanding many of the details in the paper. The experimental setup boils down to the following sentence: \"We debias mBERT using language X and evaluating it on language Y\" (Line 195). I'm not sure what this means exactly or how to reproduce the results, furthermore, I couldn't understand how the results on languages other than English were obtained, since the paper itself mentions that the dataset used isn't available in other languages (other than French?). \n\nI think that adding more details about the exact debiasing process, and which datasets are used is crucial for making the paper clearer. Following, I think that an analysis of the results would help shed light on what exactly changes when applying debiasing cross-lingualy, e.g., via a manual error analysis.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "50yP9EJZKs", "reviewer": "Reviewer_RwvA", "summary": "", "questions": "Can you include a more diverse set of evaluation to prove the points the study makes or presents any argument that supports the reason of not including additional languages? Is these four languages enough to reach a conclusion about the general nature of cross-lingual transfer and debiasing, if yes, then why?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your time and valuable insights that contributed to the paper’s quality. We agree that this paper studies a diverse list of debiasing techniques and reports results regarding the numerous experiments we carried out in a straightforward manner. In the following paragraphs we elaborate on your feedback and remarks:  \n>R3A: What happens when we change this route and evaluate on a non-Latin script language? R3C: Can you include a more diverse set of evaluation to prove the points the study makes or presents any argument that supports the reason of not including additional languages? Is these four languages enough to reach a conclusion about the general nature of cross-lingual transfer and debiasing, if yes, then why?  \n\nWe thank the reviewer for these interesting remarks. As we think these two comments are related, we provide a combined answer to both remarks.  \n\nCross-lingual debiasing is a very challenging and complex topic, with little research available. The debiasing techniques we are testing are made for English and research has mostly focused on this language. Research also mostly focuses on gender bias, neglecting other forms of bias, which already increases complexity in a monolingual context, so even more in a cross-lingual context. We acknowledge the keen observation that the languages included in the analysis are latin-script languages. However, we carefully selected  these four languages for our short paper format to **balance adding diversity and not overreaching conclusions that can be made with debiasing techniques made for English**. We included languages from **similar cultures as English** and cover both **Roman and Germanic languages**. Moreover, the chosen languages differ in the amount of data that was used to train mBERT as shown by Wu & Dredze (2020).  English is the highest resource language. French and German show similar amounts of resources, and is then followed by Dutch.  Hence, these **languages represent a diverse set for initial cross-lingual transferability analysis**. Furthermore, this offers a good first step to **stimulate research into this area** and show its importance.  \n \nIncluding **non-Latin languages** is a very interesting idea that triggered our research curiosity that includes similar challenges as were present for some languages in our study due to the lack of an available dataset. However, including these languages **requires in-depth analysis that reaches beyond the short paper format**, since we do not wish to provide premature insights in an ethically sensitive research domain. \n\nNevertheless, we **extended the evaluation setup** to show the results when debiasing in Russian by translating samples of the CrowS-Pairs dataset, despite the cultural differences for which extended analysis is necessary.  As expected, debiasing in Russian does not help to debias English in mBERT. For the other three languages, we could say that in general, Russian can be used to debias French, German, and Dutch in mBERT. However, **for Russian, the debiasing is not effective**.  The results suggest that this is due to overcompensating when debiasing gender bias. However, this requires **more in-depth analysis**, as observed variation can also be due to cultural differences, thereby potentially requiring an alternative experimental set-up. Given the ethical intricacies inherent to this research domain, we refrain from presenting these preliminary findings, as this requires in-depth analysis that goes beyond the scope of the short paper format.  Once again, thanks for this really interesting suggestion! \n\n| Evaluation language | mBERT | INLP  | SenDeb | DR*   | CDA   | DO    |\n|---------------------|-------|-------|--------|-------|-------|-------|\n| **EN**                  | 6.11  | 10.19 | 7.5    | 7.22  | 10.09 | 9.17  |\n| **FR**                  | 11.11 | 10.93 | 12.78  | 10.83 | 10.74 | 8.98  |\n| **DE**                  | 9.49  | 7.93  | 8.98   | 7.59  | 10.64 | 11.37 |\n| **NL**                  | 17.66 | 15.05 | 16.53  | 16.53 | 17.09 | 16.08 |\n| **RU**                  | 14.60 | 15.63 | 15.16  | 15.43 | 12.28 | 14.05 |\n\n|       |     | mBERT | INLP  | SenDeb | DR*    | CDA   | DO    |\n|-----|-----|-------|-------|--------|--------|-------|-------|\n| **EN**  |   | 51.11 | 51.85 | 51.94  | 51.67* | 58.80 | 52.31 |\n|    |Gender  |   49.17 |   45.83 |   50     |   50.83  |   48.61 |   49.72 |\n|   |Race | 44.17 | 43.06 | 45     | -      | 58.89 | 41.11 |\n|   |Religion | 60.00 | 66.67 | 60.83  | -      | 68.89 | 66.11 |\n| **FR**   |      | 60.56 | 55.19 | 61.11  | 59.72* | 59.26 | 57.69 |\n|   |Gender  | 50.83 | 43.89 | 47.50  | 48.33  | 52.50 | 56.11 |\n|   |Race | 59.17 | 53.06 | 62.50  | -      | 60.83 | 50.28 |\n|   |Religion | 71.67 | 68.61 | 73.33  | -      | 64.44 | 66.67 |\n| **DE**  |       | 59.21 | 55.57 | 58.13  | 56.45* | 57.61 | 56.46 |\n|   |Gender  | 61.39 | 48.73 | 54.77  | 53.10  | 56.91 | 54.00 |\n|   |Race | 57.07 | 62.41 | 58.78  | -      | 48.43 | 47.88 |\n|   |Religion | 59.17 | 55.56 | 60.83  | -      | 67.50 | 67.50 |\n| **NL**  |       | 67.66 | 64.87 | 64.58  | 64.58* | 66.34 | 65.89 |\n|   |Gender  | 56.32 | 52.66 | 47.07  | 47.07  | 50.97 | 54.35 |\n|   |Race | 65.83 | 61.94 | 65.83  | -      | 70.28 | 64.72 |\n|   |Religion | 80.83 | 80    | 80.83  | -      | 77.78 | 78.61 |\n| **RU**  |       | 63.18 | 61.41 | 60.95  | 61.24* | 59.20 | 61.88 |\n|   |Gender  | 52.03 | 45.06 | 45.34  | 46.22  | 45.93 | 47.02 |\n|   |Race | 79.17 | 78.89 | 78.33  | -      | 77.50 | 81.11 |\n|   |Religion | 58.33 | 60.28 | 59.17  | -      | 54.17 | 57.50 |\n\n>R3B: Also, to reach a conclusion about low-resource language, I feel more experiments need to be done on other true low-resource languages such as not seen by mbert languages.    \n\nWe thank the reviewer for this correct remark, we can indeed solely address the languages that have been investigated in our study. Therefore, we agree that more experiments have to be executed on other low-resource languages and encourage research into this direction with our paper. To adequately address this, we modified the sentences mentioned below.  \n\n**Actions taken:** We changed the sentences on lines 18-21 as follows (changes in bold): *We also find that debiasing techniques with additional pretraining exhibit enhanced cross-lingual effectiveness **for the languages included in the analyses**, particularly in lower-resource languages.*, Lines 271-273: *Therefore we conclude that, **for our set of languages**, these techniques are most effective when applied to low-resource languages.* Lines  305-308: *Additionally, we find that, **for the studied languages**, debiasing with the lowest resource language is effective for techniques involving an additional training step (CDA and DO).*   \n**References:** Shijie Wu and Mark Dredze. 2020. Are All Languages Created Equal in Multilingual BERT?. In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 120–130, Online. Association for Computational Linguistics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Covering a diverse list of debiasing techniques.\n2. Straightforward and substantial result reporting based on the experiments the study covers.", "reasons_to_reject": "The main problem, I feel like not having enough languages for evaluation to reach toward a conclusion. All 4 languages in this study are in latin script. What happens when we change this route and evaluate on a non-Latin script language? Also, to reach a conclusion about low-resource language, I feel more experiments need to be done on other true low-resource languages such as not seen by mbert languages.", "questions_to_authors": "Can you include a more diverse set of evaluation to prove the points the study makes or presents any argument that supports the reason of not including additional languages? Is these four languages enough to reach a conclusion about the general nature of cross-lingual transfer and debiasing, if yes, then why?", "paper_topic_and_main_contributions": "This paper examines the effect of cross-lingual transfer on various debiasing techniques and finds, it is possible to perform debiasing while following the route of cross-lingual transfer. Moreover, the research identifies SentenceDebias as the best-performing debiasing technique in this specific setting. Additional findings include obtaining better results for lower-resourced languages.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper investigates the transferability of several debiasing techniques across languages, when applied to multilingual models.

The reviewers appreciated that this work investigates debiasing in multilingual models, particularly since model debiasing work has centered on English. However, the reviewers felt that the paper would benefit from additional experiments (i.e., on a larger and more diverse set of languages), and more clarity, particular in the analysis.
Decision: Accept-Main