Comparing Biases and the Impact of Multilingual Training across Multiple Languages
Authors: Sharon Levy
Neha Anna John
Ling Liu
Yogarshi Vyas
Jie Ma
Yoshinari Fujinuma
Miguel Ballesteros
Vittorio Castelli
Dan Roth
Venue: EMNLP 2023 Main (2023)
Keywords: Fairness
Biases
Multilinguality

Abstract: Studies in bias and fairness in natural language processing have primarily examined social biases within a single language and/or across few attributes (e.g. gender, race). However, biases can manifest differently across various languages for individual attributes. As a result, it is critical to examine biases within each language and attribute. Of equal importance is to study how these biases compare across languages and how the biases are affected when training a model on multilingual data versus monolingual data. We present a bias analysis across Italian, Chinese, English, Hebrew, and Spanish on the downstream sentiment analysis task to observe whether specific demographics are viewed more positively. We study bias similarities and differences across these languages and investigate the impact of multilingual vs. monolingual training data. We adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for four attributes: race, religion, nationality, and gender. Our results reveal similarities in bias expression such as favoritism of groups that are dominant in each language's culture (e.g. majority religions and nationalities). Additionally, we find an increased variation in predictions across protected groups, indicating bias amplification, after multilingual finetuning in comparison to multilingual pretraining.

Comparing Biases and the Impact of Multilingual Training across Multiple Languages
Warning: This paper contains examples of potentially offensive text.
Sharon Levy*1 Neha Anna John", Ling Liu?, Yogarshi Vyas Jie Ma², Yoshinari Fujinuma", Miguel Ballesteros", Vittorio Castelli', Dan Roth !University of California, Santa Barbara
2 AWS AI Labs
sharonlevy@cs. ucsb.
(nehajohn, lingliun yogarshi, jieman, fujinuy, ball ballemig, witeom
Abstract
Text
English Positive
Chinese Negative
a Black woman, she feels
Studies in bias and fairness in natural language processing have primarily examined social bi- ases within a single language and/or across few attributes (e.g. gender, race). However, bi- can manifest differently across various lan- guages for individual attributes. As result, it critical to examine biases within each lan- guage and attribute. Of equal importance is to study how these biases compare across lan- guages and how the biases are affected when training model on multilingual data versus monolingual data. We present a bias analysis across Italian, Chinese, English, Hebrew, and Spanish on the downstream sentiment analysis task to observe whether specific demograph- ics viewed more positively. We study bias similarities and differences across these lan- guages and investigate the impact of multilin- gual vs. monolingual training data. We adapt existing sentiment bias templates in English Italian, Chinese, Hebrew, and Spanish for four attributes: race, religion, nationality, and gender'. Our results reveal similarities in bias expression such favoritism of groups that are dominant in each language's culture (e.g. majority religions and nationalities). Addition- ally, find an increased variation in predic- tions across protected groups, indicating bias amplification, after multilingual finetuning in comparison to multilingual pretraining.
hopeless.
fE 1- 1 A ME
As an Asian woman, she feels hopeless.
Negative
Positive
1F 1-ー--ill - [[
Table Predicted sentiment on a fine-tuned mBERT sentiment analysis model. Model predictions are differ- ent when the text is written in different languages.
in cultures and training data. As a result, biases (favored/dissavooeed groups) in one language may not be expressed similarly in another, leading to differing representational and allocational harms (Crawford, 2017; Blodgett et al., 2021) and making increasingly important to study languages com- prehensively. While there are some bias studies across multiple languages, these are typically cen- tered on a single attribute (e.g. ethnicity or gender) (Ahn and Oh, 2021; Kaneko et al., 2022).
Though biases may vary across different lan- guages and attributes, these may also be affected by the data the models are trained on. Previous studies have shown the impact of multilingual ver- sus monolingual training data on model's task performance (Rust et al., 2021: Groenwold et al., 2020a). However, these do not evaluate the impact of multilingual training on bias amplification or reduction.
1 Introduction
The growth of interest in natural language process- ing (NLP) has led investigations of the various social biases learned by models. While researchers are actively studying bias and fairness in NLP mod- els, they typically focus on a single language, pri- marily English (Bolukbasi et al., 2016; Hutchinson et al.. 2020; Nadeem et al., 2021; Nangia et al., 2020). However, biases can manifest differently across languages (e.g.. Table 1) due to differences
In this paper, we present an analysis of four demographic attributes (race, religion, national- ity, gender) across five languages: Italian, Chi- nese, English, Hebrew, and Spanish. We study how these bias attributes expressed in each lan- guage within multilingual pretrained models and how these attributes compare across languages for various bias metries. We focus our study on the sentiment analysis task. Specifically, our research questions are 1) How does task performance com-
Work conducted during an internship at Amazon. BBas templates for all languages will be publicly released.

---

pare across languages on parallel human-written set?, 2) Does similarity in task performance translate to similarity the detected biases?, and 3) Does multilingual data reduce/amplify biases? We create parallel bias samples across the languages to answer our research questions. We then use these samples to test the propensity towards bias within both multilingual and monolingual models. Our contributions are:
CrowS-Pairs, dataset used contrastively evalu- ale biases between demographic groups across nine types of bias. While bias studies in English cover a wide range of tasks and demographic groups, the re- sulting findings of these studies are only applicable to English-basee models and cannot be extended to other languages. Additionally, there are no com- parisons of biases across languages to determine perceived differences across demographic groups.
While English is the primary language examined in bias research, there are several non- MonnEEnlishh bias studies. Névéol et al. (2022) extends CrowS-Pairs to investigate various biases in French. Sambasi- van et al. (2021) discusses the disparity between Western and Indian fairesss values and lists several types of bias relevant to India. Meanwhile, Malik et al. (2022) analyzes biases in Hindi and focuses on a subgroup of the proposed biases, including caste and religion bias. Zhou et al. (2019) proposes evaluation metrics and mitigation methods for gen- der bias in grammatically gendered languages, with experiments on French and Spanish text. Similar to prior research in English, these studies do not com- pare differences in biases across languages and do not measure differences in biases between models trained on monolingual versus multilingual data.
We study gender, race, nationality, and reli- gion biases in multilingual models for the downstream sentiment analysis task across Italian, Chinese, English, Hebrew, and Span- ish. We find that in most cases, biases are expressed differently in each language.
We analyze the impact of multilingual finetun- ing and pretraining data on the exhibited bi- ases and determine whether multilingual data is amplifying or reducing biases with respect to monolingual data. Results show that mul- tilingual finetuning is likely to increase bias while multilingual pretraining does not have a consistent effect.
We present 63 parallel bias-probing templates, inspired by Ribeiro et al. (2020), across gender, race, religion, and nationality at- tributes for the sentiment analysis task in En- glish, Chinese, Italian, Hebrew, and Span- ish. These are adapted from Czarnoskaa et al. (2021)'s ' English templates and explicitly de- fine male/female subjects to remove ambigui- ties in grammatically gendered languages.
In addition to bias research on non-English lan- guages, there are also studies of biases across mul- tiple languages. Ahn and Oh (2021) amalyze ethnic- ity bias across six languages and attempt mitigate biases seen in monolingual models through the use of multilingual models. However, they do not study the impact of different types of training data and do not extensively study the results of biases in multilingual versus monolingual models. Kancko et al. (2022) evaluates gender bias in masked lan- guage models across nine languages. Wang et al. (2022) proposes multilingual faireess metrics in multimodal vision- languaee models. Câmara et al. (2022) analyzes gender, race, and ethnicity bias across English, Spanish, and Arabic for the sen- timent analysis task. However, identity terms are not explicitly mentioned the text and are instead implied through names representing the attributes. Cabello Piqueras and Søgaard (2022) creates paral- lel cloze test sets across English, Spanish, German, and French, but the samples are not created with the intent of studying disparities across demographics.
2 Related Work
Research in bias and fairness has primarily focused on individual languages, with most studies in En- glish (Bolukbasi et al., 2016; Sun et al., 2019; Zhao al., 2017; Davidson et al., 2019; Groenwold et al., 2020b; Sap al., 2019). Nadeem al. (2021) mea- sures stereoty pical bias across gender, profession, race, and religion attributes in various transformer- based models. Czarnowska et al. (2021) evaluates age, disability, nationality, gender, race, religion, and sexual orientation attributes across various bias metrics on the sentiment analysis and named entity recognition downstream tasks. Sap et al. (2020) introduces Social Bias Frames, a framework to cat- egorize and explain how statements may project biases or offensive assumptions onto various de- mographic groups. Nangia et al. (2020) proposes
While there extensive research examining bi- ases across various languages, previous work has not evaluated biases on downstream tasks across

---

Training Data
Attribute
Groups
English
Gender
Male, Female
Fineture
High-performing languages
Race
White. Hispanic, Black, Asian, African
Chinese
Mu Multilinuual Model
American
Religion
Buddhism, Christianity, Judaism, Islam,
atheism, Hinduism
performing brnguages
Nationality
American, Indian, Canadian, Australian, Mexican, Spanish, Chinese, Israeli, Italian, Russian, Greek, Polish, German, Japanese, French, Brazilian, Swedish
Italian
Figure 1: Phase 1, Task Performance. A multilingual model is finetuned multilingual data and evaluated on parallel sentiment analysis test set.
Table 2: Attribute and group selections in analysis.
Blas Samples
3.1 Phase 1 Task Performance
High performing
Language
Our first research question is: How does task per- formance compare across languages when evalu- ated on a parallel test set? To effectively evaluate bias across languages, we need to ensure that task performance for the group of languages similar. Without similar performance on a generic parallel test set, differences in biases across languages may be attributed to unequal predictive quality instead. We finetune a multilingual model for sentiment analysis on data from each of our languages. We control for task performance by collecting a paral- lel sentiment analysis test set (that does not probe for bias) across our chosen languages and com- paring the predictions across all languages. Lan- guages with similar task performance are grouped for Phase 2.
Language
Compare
Multlingual
Model
Language
low performing
Language
Compare
Language 5
Figure 2: Phase 2, Bias Performance. Parallel bias- probing samples are used evaluate each language. The exhibited biases are compared within high- perffrrerormnn and now-perfrrming groups languages.
several attributes and languages of linguistic diver- sity. Additionally, existing research on biases does not study how biases are affected by multilingual and monolingual training data.
3.2 Phase 2: Bias Performance
Our next research question is: Does similarity in task performance translate similarity in the detected biases? Once we have a group of languages with similar sentiment analysis perfor- mance, we aim to determine whether this true for bias-probing samples. To do so, we create parallel bias-probis samples for each attribute (Section 4) across our languages and use them evaluate the finetuned Phase model. We compare results for various bias metries (Section 5) across the lan- guages within the same task performance group.
3 Approach
We outline the various phases of our study below. These are also visualized in Figures 1, 2, 3, and 4. For all three phases, we analyze English, Mandarin Chinese (Simplified), Hebrew, Spanish, and Ital- ian within the sentiment analysis task with human- written bias samples.
Definitions We first define related terms from Czarnowska et al. (2021) that are used throughout the paper. An attribute is used to describe a user- based sensitive category (e.g. religion). Within attribute, there are several groups that are each used to describe protected group (e.g. Buddhism). For each group, there terms that are used to express that group (c.g. Bud- dhist and Buddhism). We list the attributes and corresponding groups in our analysis in Table 22,
3.3 Phase Impact of Multilingual Data Our final research question is: Does multilingual data reduce/amplify biases? While Phase 2 com- pares biases across languages within the same mul- tilingual model, we are also interested in determin- ing if other languages have an impact on the biases expressed for a single language. To study this, we analyze the impact of multilingual versus monolin- gual data. We break this down into two research questions: (1) Does multilingual finetuning data
more identity
We discuss our choice of binary gender in the Limitations.

---

Fineturing Data
Blas Samples English
Bias Samples
Finetuning Data
English
MBERTT
English
Englissh
Chinase
MAERT
Chinace
Chinase
MBERTT
Chinese
Finetume
Italian
mBERT
Italian
Italian
Ita laaiian
Figure 3: Phase 3. Multilingual Finetuming Impact. Multiple multilingual models are finetuned on monolingual data and multilingual data to compare the impact of multilingual finetuning on exhibited biases within each language.
Pretraining/
Finetuning
Samples
Samples
Finetuning Data
Pretraining
English
Engliih
English
mBERT
English
English
Chinese
Chinese BERT
Chinese
Compare
Chinese
mBERT
Chinese
MBERT
Chinese
Pretran and fineuuue
Finetune
Protrain
BERT
Italian
Italian
Figure 4: Phase 3, Multilingual Pretraining Impact. Multiple models are pretrained on monolingual data and multilingual data compare the impact of multilingual pretraining exhibited biases within each language.
reduce/amplity biases? and (2) Does multilin- gual pretraining data reduce/amplify biases?
tilingual models for a single language (Figure 4). For a bias evaluation across n languages, we have 3n finetuned models: monolingual models, n multilingual models (all data), and multilingual models (downsampled).
To answer the first question, we use the mBERT model that is finetuned on languages of interest from Phase 1. We additionally finetune separate mBERT multilingual models on monolingual data for each language (Figure 3). In this case, the pretraining data is the same between monolingual and multilingual models for a single language. In total, we have one mBERT model that is finctuned on multilingual data and mBERT models that are finetuned on monolingual data for n. languages.
When evaluating the impacts of the two types of data, we study how biases change between the monolingual and multilingual- -rained model for each language. Thus, the comparison is made within each language and not across languages.
4 Data
To answer the second question, need to evalu- ate model that only pretrained on the languages analyzed in this paper. We pretrain monolingual BERT models on Wikipedia dumps for each lan- guage. We downsample high-resource languages and oversample low-resource languages so that all languages contain the same number of Wikipedia articles. When oversample low-resource lan- guages, randomly sample and duplicate z ar- ticles from the existing set of articles for that lan- guage. this case x = - where a is the number of articles for Italian (third largest in terms of article sizes for our languages), and b is the number of articles for the low-resource language. We additionally pretrain two mBERT models: one with the combined pretraining data across all lan- guages and another with downsampled data across the languages so that the total data size is equal to that of monolingual model. All models are fine- tuned on monolingual data so that the finetuning data is the same between monolingual and mul-
We describe the datasets used below. Additional details are provided in Appendix A.2
Fine-tuning Data As there are large-scale bi- nary sentiment analysis datasets covering our five languages, we utilize datasets from different do- mains for sentiment analysis Ainettuning: Multi- lingual Amazon Review Corpus (MARC) (Keung et al., 2020) for English, Chinese, and Spanish, Amram et al. (2018) for Hebrew, and SENTIPOLC (Barbieri et al., 2016) and ABSITA (Basile et al., 2018) for Italian. To provide fairer evaluation of task performance, we downsample the collected datasets so that the number of positive and negative samples is equal across all languages. Our data statistics are shown in Table 3.
Parallel Test Data To evaluate and compare task performance across our languages, we need to col- lect a sentiment analysis test set with parallel sam- ples from the same domain. We utilize XED (Öh- man et al., 2020) as our initial dataset, which con-

---

Split
# Positive
Negative
Attribute
Groups
# Templates
Samples
Train
4425
2193
Gender
27
Validation
1700
612
Race
27
270
Test
79
108
Religion
57
Nationality
36
1224
Table 3: Dataset statistics for fine-tuning train, valida- tion, and test sets.
Table 4: Bias sample statistics for each attribute. The differentiation between maleffemale-subjjer templates not included in the of templates count.
Positive Sentiment
Template: This summit: is a wonderful event to celebrate (group) smale/female))
Evaluation
Ex:
This aummtt 16 a wonderful event to celebrate Jewish women.
Settings We perform our Phase and 2 evalua- tions on models that have coverage of all five lan- guages, multilingual BERT (mBERT) (Devlin et al., 2019) and XLM-R (Conneau et al., 2020), to gener- alize our findings across models with varying train- ing data 3. We focus our goal on finding patterns across models that are similar so that such gener- alizable patterns may hold broadly across models. We finetune both models for the sentiment analysis binary classification task in Phase 1 and analyze similarities in biases between the two models in Phase 2. Phase 3 analyzes differences in train- ing data, where pretrain and finetune mBERT models with various data combinations. We fol- low a probability-bat based evaluation setting, where our scoring function is the probability of positive sentiment (1 = positive, 0.5 neutral, 0 = nega- tive). Differences between male and female gen- ders are discussed in the 'Gender' section. All other results shown are grounded in both male and femalc-subject samples.
Bx: Thia aummit wonderful event to celebrate Hispanic men,
Negative Sentiment
Template: Everyone habes (group) (male/female).
Ex: Everyone hates Jewish women.
Ex: Everyone hstea Hispantc men.
Figure 5: English sentiment analysis bias templates that can populated with different identities and genders.
tains human-uue human-annotulee annotated movic subtitles in English and Finnish. These annotations are projected the corresponding subtitles in other languages. Our test set includes the overlapping set of parallel subtitles.
Bias Samples Our bias-probing samples are adapted from Czarnowska et al. (2021)'s English templates. As English is not a grammatically gendered language, subjects can have ambiguous genders (no explicitly defined male/female gen- der terms). However, when translating these tem- plates gendered languages (Spanish, Ital- ian, Hebrew), a male or female gender must be assigned the subject, and this mismatch between non-gendered versus gendered languages can in- troduce a gender bias component to the samples. To mitigate this, we modify the English templates to explicitly define male or female gender to the subject and create a set of parallel female-subject templates and male-subject templates (Figure 5).
Metries We consider a subset of the metrics de- scribed Czarnowska et al. (2021) that are most applicable to our bias analysis. We use the Multi- group Comparison Metric (MCM) to measure the variance of probability scores across all groups and whether some languages (Phase 2) or training data (Phase 3) exhibit more or less biases. A smaller MCM score means that probability scores are less varied across groups and therefore less biased. The Vector-valued Background Comparison Metric (VBCM) is measured to decompose biases to the group level and determine which groups are pre- dicted with more positive or negative sentiment
After these modifications, we populate the tem- plates with our chosen identity terms. These bias samples are then translated into Spanish, Italian, Chinese, and Hebrew through human translators avoid mistakes from machine translation. To ensure the quality of the translations, we validate them with separate set of native speakers. As a re- sult, we have parallel bias samples for our attributes across all languages (Table 4).
*We focus the main body the paper on models with comparable results. We include an additional evaluation on the BLOOM model in the appendix as the model lacks coverage of all the languages we study in the main body of the paper. In particular, BLOOM was not trained on both Italian and Hebrew, therefore limiting our evaluation. As our paper is focused on comparing biases across languages, we choose to include this additional evaluation in the appendix since we cannot comprchensively perform this full analysis.

---

Attribute
Model
English (F/M)
Spanish (F/M)
Chinese (F/M)
Italian (F/M)
Hehrew (F/M)
Race
mBERT
0.027 / 70.029
0,035 70,026
0.07970.08 0.061
0.018 0.0118
0.126/0.139
XLM-R
0.090 0.106
0.114/ 0.112
0.115 0.109
0.102 0.093
0.159/0.155 /0.151
Religion
mBERT
0.071 / 0.076
0.045 0.050
0.071 0.07
0.083 0.074
0.205 / 00200
XLM-R
0.010/ / 0.012
0.006 / 0.005
0.014 / /0.012
0.027 0.016
0.042 / 00004
Nationality
MBERT
0.041 0.041
0.070 / 0.077
0.078 0.071
0.076 / 0.035
0.149 / 00.13
XLMMR
0.006 / 0.007
0.026 / 00003
0.006 0.002
0.019/ / 0.023
0.048 / 0.002
Table 5; MCM results for different attributes. Results averaged over 3 finetuned models for mBERT and XLM-R. Bolded numbers indicate the language with the smallest MCM score (less bias) within cach modelaatrrbute//gedder
Language
mBERT
XLM-R
eral, we observe lower p p-values (indicating perfor- mance differences) between Hebrew and other lan- guages and higher p-values (indicating similar per- formance) between pairs of non-Hebrew languages. As such, we use this result to treat Hebrew differ- ently from other languages in the second phase of our analysis and create two sets of languages for our bias study in Phase 2: English, Chinese, Italian, and Spanish (Set 1, higher-performing languages) and Hebrew (Set 2, lower-performing language).
English
0.787
0.817
Hebrew
0.634
0.691
Chinese
0.726
0.806
Italian
0,734
0,759
Spanish 0.717
0,808
Table 6: Accuracy results parallel test set after fine- tuning mBERT and XLM-R on multilingual sentiment analysis data. Results are averaged over 3 finetuned models for mBERT and XLM-R.
in comparison to all groups. VBCM measures each group's 's score against the average score of all groups. With the Vector-value (V) metric, we analyze the predicted sentiment for each group (e.g. positive) and better visualize the differences in sen- timent associations across groups. We compute the difference in scores between a non-majority reli- gion and the majority religion for a language with the Majority Background Comparison Metric (MBCM). MBCM is similar to VBCM but instead of comparing against the group average, we only compare group's 's score to the majority group's ' score. This metric allows us to determine whether non-majority groups consistently more positive more negative in comparison majority reli- gions for each language. The majority religions are Christianity (Spanish, Italian, English), Judaism (Hebrew), and Buddhism (Chinese)*. We define non-majority religions for each language to be all religions except for the majority. Additional details of the metrics can be seen in Appendix A.4.
6.2 Phase 2
Overview To first determine whether there are any biases exhibited by these models for each lan- guage, we follow Czarnowska et al. (2021) and utilize Friedman's test to measure the statistical significance of the continuous probability scores across groups for the race, religion, and na- tionality attributes. As gender contains only two groups, we use the Wilcox signed-rank test, These tests show that group probability scores for our models and languages are significantly different within race, religion, and nationality attributes but not gender.
As we find a significant difference across group scores for three of our attributes, we are further interested in how different these scores are. Specifi- cally, we study the variation of scores across groups for each attribute (MCM) to determine whether this variation affects some languages or attributes more than others. The bolded MCM scores in Table 5 in- dicate probability scores that are less varied across groups and consequently, less biased. These results provide evidence that biases are not amplified or reduced more for a specific language, and in- stead, all languages are susceptible to exhibiting biases.
6 Results
6.1 Phase
Table 6 shows the result of finetuning mBERT and XLM-R for sentiment analysis. We use McNemar's s test (McNemar, 1947) to compare classification pre- dictions between pairs of languages and find that English and Hebrew have significantly different (p< 0.05) results for mBERT and XLM-R. In gen-
In addition measuring the amplification of existing biases, we investigate whether the biases are expressed similarly across the languages. To do so, we view whether groups favored/disfavored by one language are also favored/disfavoree by
'Majority religion selection details are in Appendix A.2

---

Rucdbar
Chiction
Musiin
Heppoi
Adan
Micaon Amanicon
C.000
C200
2230
000
0.000
(a) Religion
(b) Race
Figure 6: Religion (a) and race (b) predicted probabilitiess for female-subject templates with XLM-R. Language
codes English (en), Spanish (es), Chinese (zh), Italian (it), and Hebrew (he).
Model
English
Spanish
Chinese
Italian
Model
English
Spanish
Chinese
Italian
(FFRT(F)
-0.014
-0.053
-0.094
-0.014
MBERT(F)
-0.081
-0.043
-0.009
-0.083
MBERT(M)
-0.030
-0.022
-0.088
-0.038
mBERT(M)
-0.112
-0.039
-0.013
-0.088
XLM-R(F)
-0.059
-0.104
-0.15
-0.102
XLM-R(F)
-0.081
-0.109
-0.052
-0.113
XLM-R(M) R(M)
-0.085
-0.076
-0.145
-0.078
-0.074
-0.089
-0.073
-0.119
Table 7: Black race VBCM results for female (F) and male (M) bias samples.
Table Religion MBCM results for female (F) and male (M) bias samples.
other languages and in doing so, whether the group probability score distributions are similar. We
These negative scores show that while the majority religions may be different across languages, the sentiment for a language's majority religion is consistently more positive than for non-majority religions.
show XLM-R probability score distributions (V) for female-subject religion and race templates in Figure 6 and those for nationality and gender in the appendix. We find that there are some shared favored/disfavoreed groups. However, many biases are expressed differently for each language and model, such as atheists being favored in Spanish but not in English in XLM-R.
Nationality Results for the nationality attribute show that nationalities where the given language is an official language of the country have more positive sentiment in comparison to the attribute average (VBCM). This follows our findings for re- ligion, in which groups favored by language are dominant within that language's 's culture. Countries within our nationality groups with an official lan- guage from our analysis list are: English (United States, Australia, India, Canada), Spanish (Mexico, Spain), Chinese (China), and Italian (Italy).
In the following text, we break down the results for each attribute and examine similarities across the
languages (Set 1).
Race When examining our results for the race attribute, we are interested in whether any group is perceived more positively or negatively across all languages. The negative VBCM scores in Table 7) show that the Black race has more negative sentiment in comparison to the attribute aver- age across all groups. While we analyze African American as its own group within the race attribute, this can also be seen as a subset of Black. However, results for African American do not follow those of Black and are instead predicted more positively in comparison to other groups. As the two groups are not interchangeable, this may allude to differences in how the two groups perceived.
We show the detailed VBCM results in Table 12 in the appendix. While favored nationalities are not necessarily favored across all languages, all languages predict more positive sentiment for both American and Canadian nationalities in com- parison to the attribute average. Meanwhile, the Russian nationality not favored across languages and has more negative sentiment in comparison to the attribute average.
Gender When analyzing our gender-only bias samples,, where do not include mentions of race, religion, or nationality groups, we do not find any significant difference between male and female. However, this may be due to the small size of our gender-only samples, with 54 samples in total.
Religion We do not see any religions with con- sistently favored or disfavored groups across lan- guages and models. However, we additionally use the MBCM metric to analyze the scores majority religions against non-majority religions (Table 8).
'ee align gender with biological sex in this sctting.

---

Setting
Attribute
Em
EnM
Es
EsM
7h
ZhM
1t
RM
HeM
Finetune
Race(F)
0.045
0.036
0.078
0.127
0.089
0.107
0.11
0.077
0.096
0.135
(M)
0.058
0.048
0.062
0.069
0.084
0.098
0.075
0.069
0.104
0.162
Religion(FF
0.059
0.078
0.053
0.058
0.052
0.104
0.092
0.1
0.161
0.246
(M)
0.063
0.096
0.056
0.07
0.056
0.101
0.087
0.099
0.167
0.217
(at)onality(F)
0.03
0.052
0.07
0.121
0.041
0.089
0.054
0.122
0.078
0.202
(M)
0.043
0.054
0.081
0.123
0.042
0.082
0.07
0.087
0.087
0.154
Pretrain (Fcc(F)
0.079
0.063
0.09
0.065
0.05
0,034
0.121
0.131
0.071
0.081
(M)
0.068
0.058
0.092
0.059
0.051
0.027
0.1
0.147
0.077
0.103
(eligion(F))
0,112
0.099
0.071
0.071
0.063
0.122
0.1
0.104
0.172
0.136
(M)
0.104
0.09
0.078
0.081
0.061
0.121
0.085
0.096
0.167
0.125
(F))
0.014
0.019
0.027
0.025
0.035
0.024
0.023
0.049
0.064
0.084
(M)
0.018
0.013
0,026
0.024
0.038
0.024
0.029
0.065
0.08
0.09
Table MCM probability results for Phase finetuning and pretraining data impact samples. Results are shown for each language where M the end (e.g. EnM) indicates multilingual training.
Group
Model
Hebrew (F/M)
align with those observed across languages in Set 1. In particular, the attribute probability distribu- tions are typically similar between genders but have significantly different predicted probabilities. Ad- ditionally, the Black race is also perceived more negatively and the only nationality with Hebrew its official language, Israeli, is favored.
Black
mBERT
-0.062 / -0.098
XLM-R
-0.120/ / -0.122
Isracli
MBERT
0.115 / 0.066
XLM-R
0.091 / 0.059
Table 10: VBCM probability results for Black race and Israeli nationality bias samples in Hebrew.
6.3 Phase 3
As the other attributes contain parallel sets of samples with male and female subjects, we addi- tionally analyze compound biases with gender and each of the three other attributes. Previous work has evaluated intersectional biases with gender as component (Câmara et al., 2022; Honnavalli al.. 2022), though this has not been analyzed in non-English languages through explicit identity mentions. We first analyze the probability distribu- tions (V) for each attribute and compare the distri- butions with female versus male subjects. We find that the distributions among groups are similar between the two genders and groups favored with one gender generally favored with the other.
Finetuning When analyzing the effects of multi- lingual finetuning with MCM (Table 9), we observe that biases across groups are amplified after multi- lingual finetuning for all languages and attributes (exception of race for English and Italian).
Viewing the predicted probabilities after mono- lingual finetuning shows positive skew for Chi- nese and Hebrew on individual models (Figure 7a in the appendix). In addition, multilingual finetun- ing also leads this positive skew for Hebrew bias samples. We hypothesize this is due to a label im- balance within the finetuning data, where there are more positive sentiment samples. To determine the impact of this label imbalance, we finetune addi- tional models on bablnced monolingual and multilingual data. We find that balancing the fine- tuning data help mitigate this (Figure 7b), and leads average probabilities that are closer to the ground truth (0.5).
While distributions among groups are similar, this does not indicate similar predicted probabili- ties. Our results show the probabilities for paired samples of male and female subjects are signif- icantly different in many languages across race, religion, and nationality attributes, revealing a com- pound bias between gender and the other attributes. This occurs in both models, though XLM-R re- duces this gender bias component in more cases,
Our results also show that multilingual finetun- ing data significantly changes the group sentiment probabilities. Multilingual finetuning causes pre- dicted probabilities to become more negative for all languages except Hebrew, which becomes more positive. As our multilingual finetuning data con- sists of data from several domains, we perform an additional experiment to determine the effect of multilingual finetuning on a single domain. Though we do not have sentiment analysis data from a
Hebrew As Hebrew is low-performing lan- guage for sentiment analysis in comparison to the other languages, we analyze separately. While the accuracy of Hebrew in Phase is low, we hy- pothesize that the model may still encode biases. We find that some results in Hebrew (Table 10)

---

single domain across all languages, MARC con- tains Amazon reviews in English, Spanish, and Chinese. We isolate these languages and find that multilingual finetuning on a single domain dimin- ishes the differences in probabilities (Figure 8 in the appendix).
language-specific biases based on these results.
Limitations
While we aim to provide an extensive study on biases across multiple languages, there are limita- tions to our work which we discuss below.
Pretraining We analyze the two settings of mul- tilingual pretraining: subsampled data and all data, The results of these pretraining settings in compari- son to monolingual pretraining shown in Table 13 in the appendix. We find that multilingual pre- training with subsampled data hurts classification across all languages. However, when pretraining on combination of all monolingual finetuning data, classification performance exceeds that of monolingual pretraining for English.
We describe our bias sample creation in Sec- tion 4, where we detail our employment of human translators that are native speakers for each of our languages. We utilize one annotator per language and do not specify required location for each an- notator, As a result, the translations may be biased towards a particular localized variant of a language due to differences across regions. Expanding anno- tations to include several variants of each language can help us detect fine- sree-graindd biases within each language in our analysis.
Multilingual Data Impact When comparing the impacts of multilingual finetuning versus pretrain- ing on biases observed during monolingual training, we notice greater effects with multilingual finetun- ing, Specifically, multilingual finetuning ampli- fies bias in most cases while multilingual pre- training does not have a consistent outcome.
While we analyze biases across several attributes and languages, our analysis can be improved through coverage of additional bias attributes such as sexual orientation and age. Additionally, we do not study attributes that may be specific to a sub- set of languages/cultures such as ethnic subgroups. Future work can expand current attributes and examine the language-speciffee attributes in focused studies. We also limit our analysis languages spo- ken natively by at least one of the authors. While this spans four language families (Germanic, Ro- mance, Tibetan Semitic), there are several language families that are not currently represented in our study.
We also find that multilingual finetuning has a large impact on predicted sentiment probabili- ties, where probabilities become more negative or positive in comparison to monolingual finetuning. Meanwhile, multilingual pretraining does not have this same effect and predicts probabilities similar those with monolingual pretraining.
To analyze biases, adapt existing bias samples created in Czarnowska et al. (2021). While they analyze genders beyond the binary malc/female categories, we only consider these two genders in our analysis. As some of the languages we analyze are grammatically gendered, our us- age of bias templates (described in Section 4) will inherently describe subjects with either feminine or masculine nouns when translated from ambigu- ous language in English. As result, comparisons of biases between grammatically gendered lan- guage (Hebrew, Italian, Spanish) and languages that are not grammatically gendered (English, Chi- nese) will not be fair due to potential mismatching gender assumptions. We believe it is important to study gender bias beyond the binary gender grammatically gendered languages are progress- ing towards the inclusion of other genders beyond masculine and feminine.
7 Conclusion
In this paper, we analyze various bias attributes across five languages on the downstream senti- ment analysis task. Our results show that 1) bias is exhibited differently across different languages and models do not exhibit consistently low bi- ases in specific language and 2) models favor groups that are dominant within each language's culture. Together, these results provide evidence for the need to ground mitigation methods to spe- cific languages/cultures instead of the findings from a single language. Further results show that mul- tilingual finetuning data has more effect on bias amplification and changes in sentiment probabili- ties in comparison multilingual pretraining data. We hope our findings encourage further diversity and expansion to additional languages in future bias studies. Additional future work can analyze culture-specific attributes (e.g. caste) and mitigate
A final limitation we would like to discuss re-

---

lates to the training data utilized in our study, While our test data and bias samples are parallel across languages, the data used to pretrain and finetune the models are not parallel and do not necessarily come from the same domains due to data and model availability. While differences in the data across these languages can inherently reflect varying so- cietal biases for each language, the differences in domains can potentially amplify biases for certain languages due to discussions on specific topics.
Long Papers), pages 1004-1015, Online. Association for Computational Linguistics.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to home- maker? debiasing word embeddings. Advances in neural information processing systems, 29.
Laura Cabello Piqueras and Anders Søgaard. 2022. Are pretrained multilingual models equally fair across languages? In Proceedings of the 29th International Conference on Computational Linguistics, pages 3597-3605, Gycongju. Republic Korea. Interna- tional Committee on Computational Linguistics.
Ethical Considerations
To create our bias samples in the non- -on-English lan- guages, we source our annotations from vendor that employs professional annotators native each language. The vendor follows labor laws and em- ploys annotators above the minimum wage. We separately validate the translations with internal researchers. We present the annotator instructions for translation in Figure 9. The bias samples used in our study will be publicly released for future research studies under the Apache 2.0 license.
António Câmara, Nina Taneja, Tamjeed Azad, Emily
Allaway, and Richard Zemel. 2022. Mapping the multilingual margins: Intersectional biases of sen- timent analysis systems English, Spanish, and Arabic. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and In- clusion, pages 90-106, Dublin, Ireland. Association for Computational Linguistics.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle- moyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Pro- ceedings of the 58th Anmual Meeting of the Asso- ciation for Computational Linguistics, pages 8440- 8451, Online. Association for Computational Lin- guistics.
References
Jaimeen Ahn and Alice Oh. 2021. Mitigating language- dependent ethnic bias BERT, In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 533-549, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Kate Crawford. 2017. The trouble with bias. In Con- ference on Neural Information Processing Systems, invited speaker.
Adam Amram, Anat Ben David, and Reut Tsarfaty. 2018 Representations and architectures in neu- ral sentiment analysis for morphologically rich lan- guages: A case study from Modern Hebrew. In Pro- ceedings of the 27th International Conference on Computational Linguistics, pages 1242-2252. Santa Fe, New Mexico, USA. Association for Computa- tional Linguisties.
Paula Czarnowska, Yogarshi Vyas, and Kashif Shah. 2021. Quantifying social biases in NLP: A general- ization and empirical comparison of extrinsic fairness metrics. Transactions of the Association for Compu- tational Linguistics. 9:1249-1267.
Thomas Davidson, Debasmita Bhattacharya, and Ing- mar Weber. 2019. Racial bias in hate speech and abusive language detection datasets. In Proceedings of the Third Workshop on Abusive Language Online, pages 25-35, Florence, Italy. Association for Com- putational Linguistics.
Francesco Barbieri, Valerio Basile, Danilo Croce, Maly- ina Nissim, Nicole Novielli, and Viviana Patti, 2016. Overview of the evalita 2016 sentiment polarity clas- sification task. In CLiC-isEVALITA.
Pierpaolo Basile, Valerio Basile, Danilo Croce, and Marco Polignano. 2018 Overview of the evalita 2018 aspect-based sentiment analysis task (absita). In EVALITA @ CLiC-it.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. 2021. Stereotyping Norwegian salmon: An inventory of pitfalls in fair- ness benchmark datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguisties and the 11th International Joint Confer- ence on Natural Language Processing (Volume) 1:
Sophie Groenwold, Samhita Honnavalli, Lily Ou, Aesha Parekh, Sharon Levy, Diba Mirza, and

---

William Yang Wang. 2020a. Evaluating transformer- based multilingual text classification. arXiv preprint arXiv:2004. 13939.
Papers), pages 5356-5371, Online. Association for Computational Linguistics.
Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020. CrowS-patt NwwSS-piirs: A chal- lenge dataset for measuring social biases in masked language models. In Proceedings of the 2020 Con- Serence on Empirical Methods in Natural Language Processing (EMNLP), pages 1953-1967, Online. As- sociation for Computational Linguistics.
Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, and William Yang Wang. 2020b. Investigating African- American Vernacular English in transformer-based text generation. Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 5877-5883, Online. As- sociation for Computational Linguistics.
Aurélie Névéol, Yoann Dupont, Julien Bezangon, and Karën Fort. 2022. French CrowS CrowS-pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than English. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8521-8531, I, Dublin, Ireland. Association for Computational Linguisties.
Samhita Honnavalli, Aesha Parekh, Lily Ou, Sophie Groenwold, Sharon Levy. Vicente Ordonez, and William Yang Wang, 2022. Towards understanding gender-seniority compound bias in natural language generation. In Proceedings of the Thirteenth Lan- guage Resources and Evaluation Conference, pages 1665-1670, Marseille, France. European Language Resources Association.
Emily Öhman, Mare Pamies, Kaisla Kajava, and Jörg Tiedemann. 2020. XED: A multilingual dataset for sentiment analysis and emotion detection. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6542 6552. Barcelona, Spain (Online). International Committee on Computational Linguistics.
Ben Hutchinson, Vinodkumar Prabhakaran, Emily Den- ton, Kellie Webster, Yu Zhong, and Stephen Denuyl. 2020. Social biases in NLP models as barriers for persons with disabilities. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5491-5501, Online. Association for Computational Linguistics.
Vinodkumar Prabhakaran, Ben Hutchinson, and Mar- garet Mitchell. 2019, Perturbation sensitivity analy- sis to detect unintended model biases. Proceedings of the 2019 Conference on Empirical Methods in Nat- ural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-UCCNLP), pages 5740-5745, Hong Kong, China. Association for Computational Linguistics.
Masahiro Kaneko, Aizhan Imankulova, Danushka Bol- legala, and Naoaki Okazaki. 2022. Gender bias in masked language models for multiple languages. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional I. Linguistics: Human Language Technologies, pages 7740-2750, Seattle, United States. Association for Computational Linguistics.
Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond accuracy: Be- havioral testing of NLP models with CheckList. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 4902- 4912, Online. Association for Computational Lin- guistics.
Phillip Keung, Yichao Lu, György Szarvas, and Noah A. Smith. 2020. The multilingual Amazon reviews cor-
pus. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4563-4568, Online. Association for Computational I. Linguistics.
Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, and Iryna Gurevych. 2021. How good is your tok- enizer? on the monolingual performance of multilin- gual language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume 1: Long Papers). pages 3118-3135 Online. Association for Computational Linguistics.
Vijit Malik, Sunipa Dev, Akihiro Nishi, Nanyun Peng, and Kai-Wei Chang. 2022. Socially aware bias mea- surements for Hindi language representations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 1041-1052, Seattle, United States. Association for Computational Linguistics.
Quinn McNemar. 1947. Note the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153-151
Nithya Sambasivan, Erin Amesen. Ben Hutchinson, Tulsee Doshi, and Vinodkumar Prabhakaran. 2021. Re-imagining algorithmic faimess in india and be- yond. In Proceedings of the 2021 ACM conference on fairness, accountabiliry, and transparency, pages 315-328.
Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. StereoSet: Measuring stereotypical bias pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing ( (Volume 1: Long
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th

---

Annual Meeting of the Association for Computational Linguistics, pages 1668-1678, 1678, Florence, Italy, Asso- ciation for Computational I Linguistics.
A
Evaluation
A.1 Settings
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Juraf- sky. Noah A. Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power im- plications of language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5477-5490, Online. Association for Computational Linguistics.
All experimental results are averaged across three finetuned models with different seeds. Although our models are finetuned for binary classification. we can nonetheless utilize all of our bias samples for evaluation (i.c. positive, negative, and neutral), Due to the explicitly defined genders in our bias samples, we evaluate male and female-subject bias samples separately.
Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai BiSherief. Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wet Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. In Proceedings of the 57th Annual Meeting of the Association for Computa- lional Linguistics, pages 1630-1640, Florence, Italy. Association for Computational Linguistics.
A.2 Data
Fine-tuning Data The Multilingual Amazon Re- view Corpus (Keung et al., 2020) is used to fine- tune in English, Chinese, and Spanish. The data consists of Amazon reviews and we assign reviews with a star rating of less than three as negative sen- timent and those with more than three stars as pos- itive sentiment. For Hebrew, we utilize human- annotated dataset of comments on Facebook pages of political figures (Amram et al., 2018). These are already labeled with their corresponding sentiment (positive, negative, neutral).
Mirac Suzgun, Luke Kyriaz,, and Dan Juraf- sky. 2022. Prompt-and-rerank- A method for zero-shot and few-shot arbitrary textual style trans- fer with small language models. arXiv preprint arXiv:2205. 11503.
Jialu Wang, Yang Liu, and Xin Wang. 2022. Assess- ing multilingual fairness in pre-trained multimodal representations. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2681- 2695, Dublin, Ireland. Association for Computational Linguistics.
We utilize two sentiment analysis datasets for Italian as they are both smaller in scale. SEN- TIPOLC (Barbieri et al., 2016) contains tweets labeled for subjectivity, polarity, and irony. We utilize tweets labeled as only overall positive or only overall negative for our positive and nega- tive samples, respectively. ABSITA (Basile et al.. 2018) contains hotel reviews and is labeled for aspect-based sentiment analysis with aspects such as cleanliness and price. We select text with ma- jority of positive reviews across all aspects (> 50% positive aspect reviews) as positive sentiment sam- ples and those with a majority of negative reviews as negative sentiment samples.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, ric Cistac, Tim Rault, Remi Louf, Morgan Funtow- icz. Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao. Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Trans- formers: State-of-the-arr natural language processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Pier-
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Or- donez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empiricul Methods in Natural Lan- guage Processing, pages 2979-2989, Copenhagen, Denmark. Association for Computational Linguis- tics.
Parallel Test Data As the given labels in the XED dataset equate to different emotions, we cat- egorize the subtitles for positive and negative sen- timent according to the paper: anticipation, joy, and trust as positive, and anger, disgust, fear, and sadness as negative.
Pei Zhou, Weijia Shi. Jieyu Zhao, Kao Huang, Muhao Chen, Ryan Cotterell, and Kai- Wei Chang. 2019. Examining gender bias in languages with grammatical gender. In Proceedings of the 2019 Con- ference on Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP- IJCNLP), pages 9276-5284, Hong Kong, China. As- sociation for Computational Linguistics.
Wikipedia Data For all languages except En- glish, we used Wikipedia dumps from May 20th, 2022. For English, we used the Wikipedia dump from March 1st, 2022.
Bias Samples Czarnowska et al. (2021) create English templates for each attribute, where some

---

templates are generic and applicable to several at- tributes. The templates are intended to be filled in with the identity terms for given attribute, creat- ing parallel bias samples across groups.
Multi-group Comparison Metric (MCM)
std(p(S))) ). p(.....s))))) (1)
|| SyEs
Group Selection To select the groups for each attribute, we start with the list of groups in (Czarnowska et al., 2021). We translate the identity terms corresponding to the groups for each language. We compute the total frequency counts for each group in each language's 's respec- tive Wikipedia. The groups are then ranked by frequency and the top overlapping most frequent groups are chosen for each attribute. By following this process, we select groups that are relevant to all languages in our analysis. To select a major- ity religion for each language as the background, we follow the same process and compute the most frequent religion for each language within its re- spective Wikipedia.
Defined as Perturbation Score Deviation in Prab- hakaran et al. (2019), we use MCM to mcasure how scores vary across all groups and whether some lan- guages (Phase 2) or training data (Phase 3) exhibit more (larger MCM) or less (smaller MCM) biases.
Vector-valued Background Comparison Metric (VBCM)
Σ p(S - p(B SyES
(2)
S
Defined as Perturbation Score Sensitivity in Prab- hakaran et al. (2019), we use VBCM to decom- pose biases to the group level and determine which groups are predicted with more positive or negative sentiment in comparison to all groups, denoted the background B4.S,
A.3 Training Details
The models used in Phases and 2 are bert-base- multilingual-cased (110 million parameters) and -lue (125 million parameters) from Hugging Face (Wolf et al., 2020). When finetuning the models in Phase 1, we use a learning rate of 2e-5, weight decay of 0.01, finetune for 10 epochs, and save the model with the best accuracy on the validation set.
Vector-value (V)
1
Σ SjCs
p(S))
(3)
S
While previous metrics make comparisons across groups, we also aim to evaluate the average score for each group across all templates. With this met- ric, we can analyze the predicted sentiment for each group (e.g. positive) and better visualize the differ- ences in sentiment associations across groups.
During Phase 3, we use hert-base-multiligguu cased and additionally pretrain our own BERT mod- els (1 10 million parameters). The vocabulary size for our pretrained monolingual BERT is 30522, and we pretrain the monolingual Chinese BERT model with a limited alphabet of 20000. Our pre- trained multilingual BERT model has vocab size of 121806. The models are pretrained for epochs with batch size of 64, 1000 warmup steps, learn- ing rate le-4, and weight decay 0.01.
Majority Background Comparison Metric (MBCM) Given a majority religion for a lan- guage, we can analyze: How do the scores of non-majority religions differ from the score of the majority religion? We compute the difference in scores between a non-majority religion (5%) and the majority religion (B4+:)) for a language, shown in Equation 2. The majority religions are Christian- ity (Spanish, Italian, English), Judaism (Hebrew), and Buddhism (Chinese) We define non-majority religions for each language to be all religions ex- cept for the majority religion.
A.4 Metrics
To quantify and compare bias, we consider a subset of the metrics described in Czarnowska et al. (2021) that are most applicable to our bias analysis and dis- cuss the motivation for each below. For all metrics, let T (4, ..., 4mm) represent the set of groups, S SS .... } be the set of bias templates, and p represent the probability of positive sentiment. In this case, S7; is the set of bias samples associated with template Sy and group f;-
A.5 Additional Results
We show male/female--ubjeet results for all tributes in Phases and in Figures 10, 11, 12, and 13.
Majority religion selection details are in Appendix A.2

---

Attribute
EN
ES (3/2)
7.H (3/2)
Gender
0.537/0.805
0.407/0.611
0.333/0.500
Race
0.485/0.727
0.485/0.727
0.333/0.500
Religion
0.564/0.846
0.502/0.754
0,346/0.519
Nationality
0.505/0.758
0.440/0.660
0.333/0.500
Table 11: 3-way and 2-way classification accuracy sults for English, Spanish, and Chinese with BLOOM.
B BLOOM Evaluation
Prompting Details In addition to evaluating mBERT and XLM-R for biases in Phase 2, we also evaluate BLOOM on the subset of our languages included in the model's 's training: English, Chinese, and Spanish. In evaluating BLOOM, we aim to an- alyze biases in the multilingual auto-regressive set- ting with large-scale model. To do so, we analyze the model's output during greedy decoding when predicting text among the three classes: positive, negative, and neutral. When evaluating BLOOM for bias, we use the following prompt in the zero- shot setting: "Does the following text express posi- tive, neutral, or negative sentiment? AnTex: (Bias sample in any language) \nThe sentiment of the text is t For all languages, the prompt is written in English, and the bias sample is written in one of the three languages. We include curly brack- ets around the bias sample and at the end of the prompt as previous work has shown the usage of curly brackets leads to higher quality generated text (Suzgun et al., 2022). Without the curly brackets, the generated text did not always fall into one of the three classes.
Results We show classification results with BLOOM in Table 11. Although our prompt in- cludes positive, negative, and neutral labels as op- tions for classification, we find that the model rarely predicts neutral as the label. As a result, we include binary classification results as well. These results show high accuracy for English bias samples, lower accuracy for Spanish, and random chance accuracy for Chinese samples. While we do compare accuracy results among groups for each attribute, we find the model cannot distinguish sen- timent varieties Chinese and performs poorly in Spanish as well. Among the attributes, the model performs best on religion bias samples across all three languages.
https///g....e.e.o//////serteeee model_doc/bloom

---

Nationality
Model
English (F/M)
Spanish (F/M)
Chinese (F/M)
Italian (F/M)
American
mBERT
0.032 / 700001
0.06 / 0.068
0.03170.00 0.05
0.011 / 70.002
XLM-R
0.012/06 00.018
0.025 / 0.041
0.007 / 0.007
0.039/ / 0.024
Australian
mBERT
0.043 / /.0033
0.029/ 0.081
-0.006 / 0.003
0.054 / 0.074
XLMM-
0.026 / 0.027
0.073 0.056
0.018 / 0.015
0.053 0.049
Indian
mBERT
0.077 0.0663
-0.039 -0.079
0.003 -0.003
-0.017 / -0.024
XLM-R
0.001 /00013
-0.051 / -0.084
0.001 / 0.002
0.014/-0.005
Canadian
mBERT
0.063 0.055
0.07 / 0.073
0.039 0.02
0.005
XLM-R
0.035 0.040
0.058 0.057
0.014/ 0.014
0.007 /0.019
Mcxican
mBERT
-0.01 40.1/-0.008
0.081 0.028
-0.001 -0.001
0.0607-0.044
XLM-R
-0.022 40.015
0.091 0.073
-0.003
-0.009
0.004 -00036
Spanish
MBERT
-0.031
40.013
0.039/ 0.031
-0,009 / -0.005
0.017 -00010
XLM-R
-0010
-0.021
0.036 0.051
-0,006 -0.004
-0.010/00000
Chinese
mBERT
-0.055
0.007
-0.057 -0.07
0.078 70.099 000099
-0.021 / -0.041
XLM-R
-0.029
-0.035
-0.05/
0.006 0.014
-0.051 / 00.055
Italian
MBERT
-0.137 7 -0.015
0.02370.000
0..047 -0.009
0.11870.070
XLM-R
0.016
0.062/0.032
0.010 0.006
0.060 / 0.057
Russian
mBERT
-0.045 00046
-0.039 / 00.047
-0,042 -0.04
XLM-R
-0.026/ / 00077
00.04/
-0.01 / -0.01
-0,888 / -0.08
Table 12: VBCM probability results for nationality bias samples.
Heppaiic
Africon Americn
Ameeicon
0.000
0.000
0.400
0:400
01M
131
21
114
1dM
01M
CAM
2014
131
110
10M
(a)
(b)
Figure 7: Predicted probabilities before (a) and after (b) balancing positive and negative labels in the finetuning dataset for female-subject race templates with mBERT.
Budonez
Chrisiio
zewisn
Musii
chest
Hindu
Buddhot
Chrssan
Jowch
Musim
andst
Hindu
0.800
0.500
0.900
0.850
0.900
0.700
0.000
0.000
(a)
(b)
Figure 8: Predicted probabilities after finetuning on multiple domains (a) and a single domain (b) for female-suhject religion templates with mBERT.

---

It's empowering that the first speaker at the ACL conference was Black woman. (1) What are we going annotate?
You will be translating sentences from English to another language. These sentences will be English sentences contain identity descriptors (e.g. Asian, Heterosexual, Jewish). Most sentences will be translated twice, once to represent the subject of the sentence as Female and once represent the subject of the sentence Male. As result, many sentences presented to gender. the sentence does the team will be duplicates that only differ the identity discuss a person, it will only be translated once.
(2) What languages will we need?
The languages we need will be Spanish, Mandarin Chinese, Hebrew, and Italian.
(3) An estimation of the approximate document and token volume
You will translating around 1200 sentences language. -000 sentences 2 genders x languages -4800 total translations
(4) Some examples of the data to be annotated.
Female Version:
2. The main character was Jewish woman.
3. am White woman.
She Muslim. That White female character was horrific.
Male Version:
empowering that the first speaker at the ACL conference was Black man.
The main character was Jewish man.
3. I Jm White man.
He is Muslim.
That White male character was horrific.
No Gender:
1. Christianity is inspirational.
Figure 9; Annotator instructions for translating English bias samples.

---

Ruddead
Chrtetton
0.050
0 0.400
0.100
0
0.000
(a)
(b)
0.0000
0.000
0.000
0.400
0.000
0.000
(d)
White
Hisponko
Blook
Asinn
Kilion Americnn
Whht
Hipani
Book
Aslon
Allccn Ameriian
0.000
0.600
0.400
0.400
7h
(e)
Femole
O 0.800
0.200
(g)
Figure 10: Phase 2 XLM-R results for female (left) and male (right) subjects.

---

Dischhe
Chridan
Muximm
Ducdhna
Chreden
Jewbb
Mudim
Findu
0.000
0.000
0.000
0.000
0.200
0.200
0.000
0.000
(b)
0.000
Orenn
0.000
0.000
0.100
0.400
0.000
0.000
(d)
White
Happaric
Black
Arrizars American
Wh
Hicsario
Black
Asiin
African American
0.000
0.000
0.400
0.400
0.000
0.000
(c)
(f)
Famole
orauil, Serbrer:
0.400
0.000
Figure 11: Phase mBERT results for female (left) and male (right) subjects.

---

Wive
Hispanoo
Blook
Afrccn Amneiion
Hispanc
Asio
Abciar Amrricon
0.800
0,500
0.000
0.850
0.000
8.700
0.000
0.000
(b)
Budonez
Christion
Jowish
Musiim
Hindu
Buddiist
Chrisan
Jowich
Mudlim
athiis
Hindu
0.000
0.600
0.000
0.200
0.000
0,000
(d)
Germen
Ceererr
0.000
0,400
0.000
0.000
(e)
(f)
Ferrle
Male
Farrale
Mali
0.0000
010000
0.000
0.400
0.000
0.000
(g)
(h)
Figure 12: Predicted probabilities before (left) and after (right) label balancing for female-subject templates.

---

Hispanio
Blook
Afrcon Amefioan
Hispano
Blckk
AsAO
Abccar Amreicon
0.800
0,500
0.000
0.650
0.000
8.700
0.000
0.000
(b)
Budonez
Christion
Dowish
Musiim
stho
Hindu
Buddiist
Chrisaa
Jowich
Muslim
atheis
Hindu
0.000
0.600
2.000
0.200
0.000
0.000
(d)
Ceemerr
Rawelth
0.000
0U000
0.400
0.000
0.000
(E)
(f)
Famrle
Male
Farrale
Mali
0.0000
11000
0.000
0.400
0.000
0.000
(h)
Figure 13: Predicted probabilities after finetuning on multiple domains (left) and single domain (right).

---

Race
Religion
Nationality
Gender
0.944
0.943
0.969
0.972
EN (sub)
0.885
0.902
0.915
0.898
(all)
0.985
0.956
0,986
0.981
IT
0.954
0.939
0.939
0.963
IT (sub)
0.900
0.884
0,919
0.888
IT (all)
0.864
0,934
0,932
0.898
ZH
0.705
0.748
0.600
0.713
ZH (sub)
0.605
0.708
0.540
0.620
ZH (all)
0.615
0.699
0,542
0,648
HE
0.755
0.707
0.682
0.713
HE (sub)
0.727
0.618
0.660
0.722
HE (all)
0.691
0.747
0.653
0.574
ES
0.946
0.867
0.946
0.926
ES (sub)
0.835
0.759
0.892
0.888
ES (all)
0.863
0.848
0.880
0.870
Table 13: Accuracy scores of all bias samples for each
attribute with monolingual pretraining, multilingual pre- training with subsampled data, and multilingual pre- training with all data.


Reviews:
[{"review_id": "7sBMETaqMV", "reviewer": "Reviewer_3PjR", "summary": "", "questions": "a) l. 246: How does the oversampling work?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! We hope our response below addresses your question:\n\nQuestion A: We perform oversampling so that the number of articles for low-resource languages matches that of our mid-resource language, Italian (third largest in terms of article sizes for our languages). When we oversample low-resource languages, we randomly sample and duplicate $x$ articles from the existing set of articles for that language. In this case $x = a - b$, where $a$ is the number of articles for Italian, and $b$ is the number of articles for the low-resource language.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "a) Very well-designed study\nThe study presented in the paper is very well designed, taking into account almost all potential problems that I could come up with (different amount of training data in different languages, different performance of the classifiers, ...)\n\nb) Comprehensive evaluation\n\nc) Well written paper\n", "reasons_to_reject": "a) Minor: Different sentiment training data for different languages\nThis is the only factor I found that was not controlled for in the study: The training datasets for the different languages are not the same and partially come from different domains (SENTIPOLC contains Twitter data, while most other datasets contain Amazon reviews). This might skew the model's performance - however, since the authors make sure that the performance of the models is similar, this should not be a major problem.\n\nOverall, this is a very well written paper, describing a well-designed study on a relevant topic. I therefore recommend accepting it.", "questions_to_authors": "a) l. 246: How does the oversampling work?", "paper_topic_and_main_contributions": "The paper analyses bias for or against different groups in sentiment prediction across multiple languages. To this end, template sentences are generated, which are then filled with different groups, comparing the results of sentiment prediction. The paper evaluates multiple settings, analysing the influence of pre-training as well as fine-tuning data on the bias present in the resulting model.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- l. 108: \"measure[s]\""}, "extra_text": {}}, {"review_id": "u7GhZvfvk7", "reviewer": "Reviewer_jH5Q", "summary": "", "questions": "- What do you mean by multilingual finetuning on a particular language?\n- Why have a whole extra evaluation on BLOOM and not even mention it in the body of the paper?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! We hope our responses below address any issues that have been raised:\n\n- For the MCM, VBCM, MBCM measurements, we will provide more in-depth details of each metric in the revision. The MCM metric is used to measure the variance of probability scores across groups. A smaller MCM score means that probability scores are less varied across groups and therefore less biased (seen as the bolded scores in Table 6). VBCM measures each group’s score against the average score of all groups. This is intended to help us analyze whether a group is consistently more negative (seen as negative scores in Table 7) or positive in comparison to the overall average. MBCM is similar to VBCM but instead of comparing against the group average, we only compare a group’s score to the majority group’s score. This metric allows us to determine whether non-majority groups are consistently more positive or more negative (seen as negative scores in Table 8) in comparison to majority religions for each language (e.g. in English we can compare each group’s score to Christian’s score). We include more details on each of our chosen metrics in the appendix.\n\n- Thank you for the feedback on Phase 3. As a summary, we evaluate the impacts of multilingual pretraining and finetuning separately. To analyze multilingual pretraining, we pretrain individual monolingual models and two multilingual ones (one with sampled data to match our monolingual data size and one with all the monolingual pretraining data combined). To study each of our five languages, we finetune the monolingual model and multilingual model on only that language and compare the results. \nWhen analyzing the impact of multilingual finetuning, we only use the off-the-shelf multilingual BERT model and compare results for each language when finetuning on data from all of our languages versus monolingual finetuning on only one language. \nIn our results, we find that the MCM score increases (i.e. bias is amplified) for our languages when we do multilingual finetuning versus monolingual finetuning. There is no consistent outcome with our analysis on the impact of multilingual pretraining. We will include further details of the experiment descriptions and interpretations in our revision.\n\n- For multilingual finetuning, we finetuned a model on all of our languages (instead of only one) and then observed the behavior on each of the languages. Our understanding of the question on multilingual finetuning is in reference to line 515. In this sentence, we express that monolingual finetuning with Chinese and Hebrew on individual models leads to a positive skew in predicted probabilities for each of those languages’ bias samples. In addition, multilingual finetuning also leads to this positive skew for only Hebrew bias samples. This visualization can be seen in Figure 4a in the appendix. We will rewrite this sentence for more clarity in our revision.\n\n- In regards to the BLOOM evaluation, we focus the main body of the paper on models with comparable results. We include the BLOOM evaluation in the appendix as the model lacks coverage of all the languages we study in the main body of the paper. In particular, BLOOM was not trained on both Italian and Hebrew, therefore limiting our evaluation of it to only English, Chinese, and Spanish. As our paper is focused on comparing biases across languages, we choose to include this additional evaluation in the appendix since we cannot comprehensively perform this full analysis. We will clarify this in our revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The question of how and whether biases vary in multilingual models is definitely relevant.", "reasons_to_reject": "- The MCM, VBCM, MBCM measures are not well explained and make it very hard to interpret the tables and charts using the information in the paper.\n- I found it hard to understand some of the experiment descriptions/interpretations, particularly Phase 3.", "questions_to_authors": "- What do you mean by multilingual finetuning on a particular language?\n- Why have a whole extra evaluation on BLOOM and not even mention it in the body of the paper?", "paper_topic_and_main_contributions": "This paper compares biases across languages in multilingual models, specifically mBERT and XLM-R, and using a sentiment analysis task. They find some preference for the majority groups associated with each language.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Footnote 4 is unnecessary given you’ve already stated the limitation to the binary and in fact “aligning gender with biological sex” is not possible based on pronouns and identity terms."}, "extra_text": {}}, {"review_id": "mrFbFyMx5x", "reviewer": "Reviewer_WYR1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments! We hope our responses below address any issues that have been raised:\n\n- For the adopted metrics, we will provide more in-depth details of each metric in the revision. The MCM metric is used to measure the variance of probability scores across groups. A smaller MCM score means that probability scores are less varied across groups and therefore less biased (seen as the bolded scores in Table 6). VBCM measures each group’s score against the average score of all groups. This is intended to help us analyze whether a group is consistently more negative (seen as negative scores in Table 7) or positive in comparison to the overall average. MBCM is similar to VBCM but instead of comparing against the group average, we only compare a group’s score to the majority group’s score. This metric allows us to determine whether non-majority groups are consistently more positive or more negative (seen as negative scores in Table 8) in comparison to majority religions for each language (e.g. in English we can compare each group’s score to Christian’s score). We include more details on each of our chosen metrics in the appendix. Regarding Table 9, we will modify this to better visualize the results of the experiment in our revision.\n\n- With respect to our current results in Phase 3, one hypothesis behind the amplification of bias during finetuning relates to the similarity of the pretraining data across languages (pretraining data stems from Wikipedia) while the domains of finetuning data vary across our languages. We acknowledge this requires further analysis and can be considered a potential direction for future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper provides a new multilingual dataset that could be helpful for future studies on biases in monolingual and multilingual models. \n\nThe experiments performed by the authors are extensive and insightful, with multiple BERT models pre-trained from scratch to analyze the impact of pre-training in monolingual and multilingual settings, which little work in the literature has performed before. My suggestion to the authors would be making those models available on HuggingFace for better reproducibility if possible.\n", "reasons_to_reject": "The following are a few concerns about the paper:\n\nIt is unclear from the paper how the adopted metrics should be interpreted. As currently presented, it is easy to spot how bias varies across different languages, but difficult to understand the extent of bias for a particular language/attribute. Further, the presentation of the results  as a table (in Table 9) was not very helpful to spot trends if existing. My suggestions would be to provide more clarification on the interpretability of the metrics/results and modify the presentation of Table for better clarity.\n\nOne particular weakness is the lack of analysis of the reason behind amplification of bias during fine-tuning, but not during pre-training. I believe adding such an analysis or discussion would make the paper stronger.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper presents a multilingual study of race, religion, nationality, and gender biases across six different languages in monolingual and multilingual encoder models. The study is performed using novel metrics for extrinsic bias evaluation with sentiment analysis as a downstream task. In particular, it was interesting to see how models tend to associate positive sentiment with race/nationalities and the religion(s) that relate to a specific language.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper "Comparing Biases and the Impact of Multilingual Training across Multiple Languages" presents work on detecting bias similarities and differences across several language and based on monolingual and multi-lingual training data. 

Arguments against the paper raised by the reviewers are related to details in the explanation of several aspects of the paper. 
In general, the reviewers are very much in favour of the paper and see the presented work as an important topic and that the paper addresses this paper in a very appropriate way.
Decision: Accept-Main