Is a Prestigious Job the same as a Prestigious Country? A Case Study on Multilingual Sentence Embeddings and European Countries
Authors: Jindřich Libovický
Venue: EMNLP 2023 Findings (2023)
Keywords: multilngual language models
nationality bias
sentence representation
TL;DR: When prompted country's prestige multilignual sentence embeddings order countries by their GDP, but the dimensions is uncorrelated with occupational prestige.

Abstract: We study how multilingual sentence representations capture European countries and occupations and how this differs across European languages. We prompt the models with templated sentences that we machine-translate into 12 European languages and analyze the most prominent dimensions in the embeddings. Our analysis reveals that the most prominent feature in the embedding is the political distinction between Eastern and Western Europe and the country's economic strength in terms of GDP. When prompted specifically for job prestige, the embedding space clearly distinguishes high and low-prestige jobs. The occupational dimension is uncorrelated with the most dominant country dimensions in three out of four studied models. The exception is a small distilled model that exhibits a connection between occupational prestige and country of origin, which is a potential source of nationality-based discrimination. Our findings are consistent across languages.

Is a Prestigious Job the same as Prestigious Country?
A Case Study on Multilingual Sentence Embeddings and European Countries
Jindřich Libovický
Institute of Formal and Applied Linguistics,
Faculty of Mathematics and Physics, Charles University, Czech Republic libovicky@ufall .mfffffll.ffff....mm www..mnf..........mmmmmmmmm...f.
Abstract
extract moral sentiments from sentence representa- tion (Schramowski al., 2022).
We study how multilingual sentence represen- tations capture European countries and occu- pations and how this differs across Euroрean languages. We prompt the models with tem- plated sentences that we machine-translate into 12 European languages and analyze the most prominent dimensions in the embeddings. Our analysis reveals that the most prominent feature the embedding is the geopolitical distinction between Eastern and Western Europe and the country's s economic strength in terms of GDP. When prompted specifically for job prestige, the embedding space clearly distinguishes high and low low-prestige jobs. The occupational dimen- dominant sion uncorrelated with the country dimensions in three out of four stud- ied models. The exception is a small distilled model that exhibits connection between occu- pational prestige and country of origin, which a potential source of nationality- -bsed dis- crimination. Our findings consistent across languages.
Our exploration encompasses four sentence rep- resentation models across 13 languages ($ 3). We find only minor differences between languages in the models. The results ($ 4) show that the strongest dimension in all models correlates with the political and economic distinction between Western and Eastern Europe and the Gross Do- mestic Product (GDP). Prompting specifically for country prestige leads to similar results. When prompted for occupations, the models can distin- guish between low and high-prestige jobs. In most cases, the extracted job ore-prestige dimension only loosely correlates with the country-prestie dimen- sion. This result suggests that the models do not connect individual social prestige with the coun- try of origin. The exception is small model dis- tilled from Multilingual Universal Sentence En- coder (Yang et al., 2020) that scems to mix these two and thus confirms previous work claiming that distilled models are more prone to biases (Ahn et al., 2022).
1 Introduction
The source code for the experiments is available GitHub.'
Language models and pre-trained representations in Natural Language Processing (NLP) are known to manifest biases against groups of people, includ- ing negative stereotypes connected to ethnicity or gender (Nangia et al., 2020; Nadeem et al., 2021). It has been extensively studied with monolingual models. Multilingual models, often used for model transfer between languages, introduce another po- tential issue: stereotypes of speakers of some lan- guages can be imposed other languages covered by the model.
Methodology
We analyze sentence representation models ($ 2.1) using a generalization of the Moral Direction frame- work ($ 2.2). We represent concepts (countries, jobs) using sets of templated sentences ($ 2.3), for which we compute the sentence embeddings. Then, we compute the principal components of the em- beddings and analyze what factors statistically ex- plain the main principal component ($ 2.4).
In this case study, we try determine the most prominent biases connected to European countries in multilingual sentence representation models. We adopt an unsupervised methodology ($ 2) based on hand-crafted prompt templates and principle component analysis (PCA), originally developed to
2.1 Sentence Embeddings Models
Sentence-embeddding models are trained to produce single vector capturing the semantics of an en-
'nttps: //g/t///bttttt httphhu..ccmm.c// J bov wcyy/ europe- in- sentence- embeddings

---

tire sentence. Contextual embeddings trained via the masked-lang www.-laggggddddeeeeee objective (Devlin et al., 2019) capture subwords well in context; how- ever, they fail to provide a sentence representation directly comparable across sentences. Sentence- BERT (Reimers and Gurevych, 2019) approaches this problem by fine-tuning existing contextual em- beddings using Siamese Networks on sentence clas- sification tasks. As result, sentences with similar meanings receive similar vector representation.
putting countries and occupations in the context of prestige. We average their embeddings and get the main principle component using PCA. Then, using three different template sets, we analyse what the main principle component best correlates with.
2.3 Templating Sentences
Similar to Hämmerl et al. (2023), who extended the Moral Dimension framework to multilingual models, we use templates in English and machine- translate the sentences into other languages after filling in the templates. We use three template sets. The sets consist of synonymous sentences with the following meaning:
The issue of sentence representation also ap- plics to multilingual contextual embeddings such as XLM-R (Conneau et al., 2020). In a multilingual setup, the additional requirement is that similar sen- tences receive similar representation regardless of the language. This is typically achieved using par- allel data for knowledge distillation (Reimers and Gurevych, 2020: Heffernan al., 2022) or more directly in a dual encoder setup (Feng et al., 2022).
They come from (COUNTRY).
2. Being from (COUNTRY] ] is considered pres- tigious.
Working as [JOB] is considered prestigious.
2.2 Embedding Analysis Method
See Appendix A for the complete template list. In the first set of templated sentences, we search for the general trend in how countries are repre- sented. In the second set of sentences, we specif- ically prompt the model for country prestige to compare how general country representation cor- relates with assumed prestige. In the third case, fit the PCA with templates containing job titles, i.e., the most prominent dimension captures job prestige according to the models. We apply the same projection to template representations related to country prestige from Set (country prestige).
We base our methodology on an unsupervised method for extracting semantic dimensions from sentence embeddings, originally introduced in the context of moral institutions (Schramowski al., 2022). The original study analyzed the moral senti- ment of English verb phrases using SentenceBERT. The method consists of three steps:
1. Generate templated sentences associating verbs with morality (e.g., "You should smile.",
"It is good to smile.") and average them for each verb. Tee. there is one average sentence embedding per verb.
We include all European countries of the size of at least Luxembourg and use their short names (e.g., Germany instead of the Federal Republic of Germany), which totals 40 countries. The list of countries is in Appendix A.3.
Countries.
2. The sentences are processed with Sentence- BERT, and the representations averaged for each phrase.
3. Apply PCA over the representations.
Low- and high-prestige jobs. We base our list of low- and high- sigh-prestige jobs on a sociological study conducted in 2012 in the USA (Smith and Son, 2014). We manually selected 30 jobs for each category to avoid repetitions and to exclude US- specific positions. By using this survey, also bring the assumption that the European countries have approximately similar cultural distance from the USA. The complete list of job titles used Appendix A.2.
The results show that the most significant dimen- sion roughly corresponds to the moral sentiment of the phrases. They use multiple templates so that linguistic differences and potential verb connota- tions average out. Using templated sentences also eliminates linguistic diversity in the data. Because of that, the main principle component does capture linguistic differences but the most prominent se- mantic nuances across the verbs when used in the specific context of moral intuitions.
2.4 Evaluation
We extend this method to a more exploratory setup. We use a similar set of template sentences,
Interpreting the dominant dimension. For the analysis, we assign the countries with abstractive

---

Parallel
Base model (Song al., 2020) finetuned for sen- tence representation using paraphrasing (Reimers and Gurevych, 2019). In the distillation stage, XLM-R Base (Conneau et al., 2020) was finetumed to produce similar sentence representations using parallel data (Reimers and Gurevych, 2020).
Model
Backbone
Params.
data
Mul. MPNet
XLM-R Base
Yes
278M
D. mUSE
Distil-mBERT
No
135M
LaBSE
Yes
471M
XLM-R-NLI
XLM-R Base
No
278M
Table Basic features the studied models.
Distilled mUSE is distilled version of Multi- lingual Universal Sentence Encoder (Yang et al., 2020) that was distilled into Distill mBERT (Sanh et al., 2019). This model was both trained and distilled multilingually.
labels based on geographical (location, mountains, seas), political (international organization member- ship, common history), and linguistic features (see Table 5 in the Appendix for a detailed list). The labels are not part of the templates.
LaBSE (Feng et al., 2022) was trained on a com- bination of monolingual data and parallel data with max-margin objective for better parallel sentence mining combined with masked language modeling.
We compute the Pearson correlation of the one- hot indicator vector of the country labels with the extracted dominant dimension to provide an in- terpretation of the dimension (some also called point-biscrial correlation). Finally, because creat- ing fixed unambiguous list of Western and Eastern countries is diffieult and most criteria are ambigu- ous, we manually annotate if the most positively and negatively correlated labels correspond to the economic and political distinction between Eastern and Western Europe.
XLM-R-XNLI is trained without parallel data using machine- transsated NLI datasets (Hämmerl et al., 2023). The model is based on XLM-R Base but finetuned using Arabic, Chinese, Czech, English, and German data following the Sentence- BERT recipe (Reimers and Gurevych, 2019).
3.2 Translating Templates
In addition, we compute the country dimen- sion's correlation with the respective countries' GDP based purchasing power parity in 2019, according to the World Bank.
To evaluate the multilingual representations more languages, we machine translate the tem- plated text into 12 European languages: Bulgarian, Czech, German, Greek, Spanish, Finnish, French, Hungarian, Italian, Portuguese, Romanian, and Russian (and keep the English original). We se- lected languages for which high-quality machine translation systems available on the Hugging- face Hub. The models are listed in Appendix B.
Cross-lingual comparison. We measure how the extracted dimensions correlate across languages. To explain where differences across languages come from, compute how the differences cor- relate with the geographical distance of the coun- tries where the languages are spoken, the GDP of the countries, and the lexical similarity of the lan- guages (Bella et al., 2021).3
Results
Aggregated results. The results aggregated over language are presented in Table 2. The detailed results per language are in the Appendix in Tables 6 and 7.
3 Experimental Setup
3.1 Evaluated Sentence Embeddings
When prompting the models for countries, the most prominent dimensions almost always separate the countries according to the political east-west axis, consistently across languages. This is fur- ther stressed by the high correlation of the country dimension with the country's 's GDP, which is partic- ularly strong in multilingual MPNet and Distilled mUSE. When we prompt the models specifically for country prestige, the correlation with the coun- try's GDP slightly increases.
We experimented with diverse sentence embedding models, which were trained using different meth- ods. We experimented with models available in the SentenceBERT repository and an additional model. The overview of the models is in Table I.
Multilingual MPNet was created by multilingual distillation from the monolingual English MPNet
/ntcps::/ouzwrrddiin wwtpiinnaaaataggggggeeeeeee graphe/ gap- sclp-per-cale pper-cappta-worlddeee
When we prompt the models for job prestige, they can distinguish high- and low- prestige jobs
'http://ukc. unf 1 1/ Indxx nhpx
lexsim

---

Country origin
Country prestige
Job prestige
Job class.
Model
accuracy
East- West
GDP
East West
GDP cor.
East-West
GDP
Multiling. Paraphrase MPNet
1.00
.79
1.00
.79
.08
.08
.93
Distilled mUSE
1.00
.71
1.00
.71
.69
.1
.85
LaBSE NLI-fnetuned XLM-R
1.00
.48
1.00
.50
.23
.09
.88
.85
.47
.85
.50
.08
.08
.91
Table Quantitative results averaged languages showing the average correlation of the dominant dimension with the country's GDP and proportion of languages where the dominant dimension corresponds to the political division of Eastern and Western countries. detailed per-language results are presented in Tables and the Appendix.
de
88
de
de
de
0.8
en
en
en
65
65
65
e5
fi
ft
f1
fr
nd
fr
fr
hu
fT
hu
fr
fr
hu
0.6
hu
hu
hu
20
20
pt
e
10
Mul. Par. MPNet
Dist. mUSE
LaBSE
XLM-R-NLI
0.2
Figure 1: Cross-language correlation of the job-prestige dimension. Languages are coded using ISO 639-1 codes.
well (accuracy 85 85-93%). When apply the
Geo. dist.
GDP diff.
Lang.
Model
same projection to prompts about countries, in most cases, the ordering of the countries is ran- dom. Therefore, we conclude that the models do
sim.
MP MPNet
.020
.077
.459
D. mUSE
.069
.194
.316
LaBSE
.175
.076
.443
not correlate job prestige and country of origin.
XLM-R-NLI
.387
.042
.064
The only exception is Distilled mUSE where the job- prestige dimension applied to countries still correlates with the country's GDP and the east-west axis. This is consistent with previous work showing that distilled student models exhibit more biases than model trained on authentic data (Vamvas and Sennrich, 2021; Ahn al., 2022).
Table Correlation of the language similarities (in terms of cross- languaae correlation of the job-prestige dimension) with the geographical distance of the coun- tries, language similarity, and GDP.
For all models except XLM-R-NLI, the lexical similarity of the languages is the strongest predic- tor. XLM-R-NLI, with low differences between languages, better correlates with geographical dis- tances.
Differences between languages. Further, we evaluate how languages differ from each other.
In all models, the first PCA dimension from the job-prestige prompts separates low- and high- prestige jobs almost perfectly. Nevertheless, mul- tilingual MPNet and distilled mUSE show a rela- tively low correlation of the job dimension across languages (see Figure l).
Related Work
Socictal biases of various types in neural NLP mod- els are widely studied, especially focusing on gen- der and ethnicity. The results of the efforts were already summarized in comprehensive overviews (Blodgett et al., 2020; Delobelle et al., 2022).
Finally, we try to explain the correlation between languages by connecting them to countries where the languages are spoken. We measure how the correlation between languages correlates with the geographical distances of (the biggest) countries speaking the language, the difference in their GDP, and the lexical similarity of the languages. The results are presented in Table 3.
Nationality bias has also been studied. Venkit al. (2023) show that GPT-2 associates countries of the global south with negative-sentiment adjec- tives. However, only a few studies focus on ases in how multilingual models treat different lan-

---

guages. Papadimitriou et al. (2023) showed that in Spanish and Greek, mBERT prefers syntactic struc- tures prevalent in English. Arora et al. (2022) and Hämmerl et al. (2023) studied differences in moral biases in multilingual language models, conclud- ing there are some differences but no systematic trends. Yin et al. (2022) created dataset focused on culturally dependent factual knowledge (c.g.. the color of the wedding dress) and concluded it is not the case that Western culture propagates across languages.
Ondiej Dušek and Dominik Macháček for their comments on the paper draft.
This research was supported by the Charles Uni- versity project PRIMUS/23/SCT/22..
References
Jaimeen Ahn, Hwaran Lee, Jinhwa Kim, and Alice Oh. 2022. Why knowledge distillation amplifies gender bias and how mitigate from the perspective of Dis- tiBBERT. In Proceedings of the 4th Workshop on Gen- der Bias in Natural Language Processing (GeBNLP). pages 266-272, Seattle, Washington. Association for Computational Linguistics.
6 Conclusions
We showed that all sentence representation mod- els carry a bias that the most prominent feature of European countries is their economic power and af- filiation to former geopolitical Western and Eastern blocks. In the models we studied, this presumed country prestige does not correlate with how the models represent the occupation status of people. The exception is Distilled mUSE, where the two correlate, which might lead discrimination based on nationality.
Arnav Arora, Lucie-Aimer Kaffee, and Isabelle Au- genstein. 2022. Probing Pre-traine rre-traine Language Mod- els for Cross-cultural Differences in Values. CoRR. abs/2203.1372.
Gábor Bella, Khuyagbaatar Batsuren, and Fausto Giunchiglia. 2021. A Database and Visualization of the Similarity of Contemporary Lexicons. In Text, Speech, and Dialogue 24th International Confer- ence, TSD 2021, Olomouc, Czech Republic, Septem- ber 6-9, 2021, Proceedings, volume 12848 of Lecture Notes in Computer Science, pages 95-104. Springer.
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey "bias" in NLP. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5454- 5476, Online, Association for Computational Lin- guistics.
Limitations & Ethical Considerations
The validity for different cultures. The "ground truth" for job prestige was taken from studies con- ducted in the USA. They might not be representa- tive of other countries included in this case study, Given that all countries considered in this case study are a part of the so-called Global North, can assume a certain degree of cultural similar- ity, which makes our results valid. However, our methodology is not guaranteed to generalize be- yond the Western world.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle- moyer, and Veselin Stoyanov. 2020. Unsupervised cross- lonss-liggua representation learning at scale. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 8440- 8451, Online. Association for Computational Lin- guistics.
Unintended use. Some methods we use in the study might create a false impression that we have developed a scorer of job or country prestige. This is not the case. The correlations that we show in the Results section ($ 4) do not guarantee the re- liability of the scoring beyond the intended use in the study, which is an assessment of multilin- gual sentence representation models. Drawing any conclusions about occupation groups, nations, or individual people using the methods used in this study might have harmful consequences.
Pieter Delobelle, Ewoenam Tokpo, Toon Calders, and Bettina Berendt. 2022. Measuring fairness with bi- ased rulers: A comparative study on bias metrics for pre-trained language models. In Proceedings of the 2022 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 1693-1706. Seattle, United States. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Papers), pages
Acknowledgments
Many thanks to Tomás Musil and Rudolf Rosa for discussing the methodology in this paper and to

---

4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages Online. Association for Computational Linguistics.
Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Ari- vazhagan, and Wei Wang. 2022. prrguage-agnostee BERT sentence embedding. In Proceedings of the 60th Annual Meeting of the Association for Compu- lational Linguistics (Volume 1: Long Pupers), pages 878-891, Dublin, Ireland. Association for Computa- tional Linguistics.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR,
Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin A. Rothkopf, and Kristian Kersting, 2022. Large pre- pre-trained language models contain human- like biases of what is right and wrong to do. Nat. Mach. Intell., 433):58-2265
Katharina Hämmerl, Björn Deiseroth, Patrick Schramowski, Jindrich Libovicky, Constantin A. Rothkopf, Alexander Fraser, and Kristian Kersting. 2023. Speaking Multiple Languages Affects the Moral Bias of Language Models. In Findings of the Association for Computational Linguistics: ACI. 2023, Toronto, Canada, July 9-14, 2023, pages 2137-2156. Association for Computational Linguistics.
Tom W. Smith and Jaesok S 2014. Measuring Occu- pational Prestige the 2012 General Social Survey, GSS Methodological Report No. 122. Technical re- port, NORC at the University of Chicago, Chicago, IL, USA.
Kevin Heffernan, Onur Çelebi, and Holger Schwenk. 2022. Bitext mining using distilled sentence repre- sentations for low-resource languages. Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2101-2112, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie- Yan Liu. 2020, MPNet: Masked and Permuted Pre- training for Language Understanding, In Advances in Neural Information Processing Systems 33: An- nual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual.
Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. StereoSet: Measuring stereotypical bias pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the I 11th International Joint Conference on Natural Language Processing (Volume I: Long Pupers), pages 5356-5371, Online. Association for Computational Linguistics.
Jannis Vamvas and Rico Sennrich. 2021. Contrastive conditioning for assessing disambiguation in MT: A case study of distilled bias. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10246-10265, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Pranav Narayanan Venkit, Sanjana Gautam, Ruchi Pan- chanadikar, Ting-Hao K. Huang, and Shomir Wilson, 2023. Nationality Bias in Text Generation. In Pro- ceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics, EACL 2023, Dubrovnik, Croatia, May 2-6, 2023, pages 116-122, Association for Computational Linguistics.
Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020. CrowS-paire A chal- lenge dataset for measuring social biases in masked language models. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNIP),, pages 1953-1967, Online. As- sociation for Computational Linguistics.
Tsabel Papadimitriou, Kezia Lopez, and Dan Jurafsky. 2023. Multilingual BERT has accent: Evaluating English influences on fluency multilingual models. In Findings of the Association for Computational Lin- guistics: EACL 2023, pages 1194-1200, Dubrovnik, Croatia. Association for Computational Linguistics.
Yinfei Yang, Daniel Cer, Amin Ahmad, Mandy Guo. Jax Law, Noah Constant, Gustavo Hernandez Abrego, Steve Yuan, Chris Tar, Yun-hsuan Sung, Brian Strope, and Ray Kurzweil. 2020. Multilingual universal sen- tence encoder for semantic retrieval. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties: System Demonstrations. pages 87-94, Online. Association for Computational Linguistics.
Nils Reimers and Iryna Gurevych. 2019. Sentence- BERT: Sentence embeddings using Siamese BERT- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IICNNIP)) pages 3982-3992, Hong Kong, China. Association for Com- putational Linguistics.
Da Yin, Hritik Bansal, Masoud Monajatipoor, Liu- nian Harold Li, and Kai-Wei Chang. 2022. GeoM- LAMA: Geo-diverse commonsense probing on multi- lingual pre-traines trained language models. In Proceedings of the 2022 Conference Empirical Methods in Nat- ural Language Processing, pages 2039-2055, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Nils Reimers and Iryna Gurevych. 2020. Making monolingual sentence embeddings multilingual us- ing knowledge distillation. In Proceedings of the

---

Language
Code
Model
Bulgarian
Czech
German
el Bacebookk/mm111-----------
de
Greek
English
Spanish
Finnish fr fel:::/fhh--111r0000000------011
French
Hungarian
Italian it ge23:inck-kk//gggurr-----0aaara3
Portuguese
Romanian ro BeL_:nuss-2////uuu----------aaa1
pl
Russian m facebook/wmt 9--en-r
Table 4: Huggingface Hub Identifier of the machine translation models used for our experiments.
A Templated texts
People who work as [JOB] are highly esteemed. prestigious to work as [JOB].
A.1 Sentence Templates
Country of origin
A.2 Low- and High-prestige jobs
They are from /COUNTRYJ.
They come from /COUNTRY).
Low-profile jobs. a hotel chambermaid, a door- to-door salesman, a leaflet distributor, janitor, a used car salesman, a bartender, telephone opera- tor, carwash attendant, cattle killer in a slaugh- tering plant, dishwasher, a stockroom attendant, nox-folding-machiie operator, a crushing-machinn operator, a taxicab driver, a bicycle messenger, a salesperson hardware store, a street sweeper, a cashier in a supermarket, a pump operator, a rail- road ticket agent, a desk clerk in a hotel, a cable TV installer, a sewing machine operator, a waiter in restaurant, an assembly line worker, a shoeshiner, ditch digger, unskilled worker in factory, tire retreader, dry cleaner
Their homeland in /COUNTRY). They natives of /COUNTRY). They originate from /COUNTRY). They were born in [COUNTRY).
Their country of birth is /COUNTRY). They hail from /COUNTRY). They grew up in /COUNTRYJ. They are citizens of (COUNTRY]. [COUNTRY] is their country of origin. [COUNTRY] is where they are from.
They were raised in /COUNTRY).
A.1.1 Country prestige
Being from (COUNTRY] is considered prestigious. Being from !COUNTRY| has high prestige. Being from /COUNTRY] is seen symbol of success.
High-profile jobs. a surgeon, university pro- fessor, an architect, lawyer, priest, a banker, a school principal, an airline pilot, an economist, a network administrator, air traffic controller, author, nuclear plant operator, computer scien- tist, a psychologist, a pharmacist, a colonel in the army, a mayor of a city, a university president, a dentist, a fire department lieutenant, a high school teacher, a policeman, a software developer, an ac- tor, a fashion model, journalist, a musician in symphony orchestra, a psychiatrist, a chemical engineer
Being from /COUNTRY] is highly regarded. People from /COUNTRY) are looked upon with admiration.
People from [COUNTRY] are highly esteemed. It is prestigious to be from /COUNTRY].
Job prestige
Being [JOB] is considered prestigious.
A.3 Countries
Being [JOB] has high prestige.
Working as [JOB] is seen symbol of success. Working as (JOB) is highly regarded.
We consider the following 40 countries (ordered by their ISO 3166-1 codes): Austria, Bosnia and Herzegovina, Belgium, Bulgaria, Belarus, Switzerland, the Czech Republic, Cyprus, Den-
People who work as [JOB] are looked upon with admiration.

---

mark, Germany, Greece, Spain, Estonia, Fin- land, France, Hungary, Croatia, Ireland, Iceland, Italy, Latvia, Lithuania, Luxembourg. Netherlands, Moldova, Montenegro, North Macedonia, Malta, Norway, Portugal, Poland, Romania, Russia, Slo- vakia, Slovenia, Albania, Serbia, Sweden, Turkey, Ukraine. Great Britain, Kosovo.
The qualitative group labels assign to the countries
in the further analysis are in Ta- ble 5. The values reflect the world as the training data (estimated pre-2021) for the models and, there- fore, do not reflect recent events (i.c., Croatia not listed among countries paying with Euro, and Fin- land is considered neutral).
B Machine Translation Models
The machine translation models we used listed Table While keeping default values for all decoding parameters.
Detailed per-language results
The detailed per-language results presented in Tables 6 and 7.

---

Bosnia and Herzegovina
North Macedonia
Czech Republic
Switzerland
Luxembourg
Netherlands
Montenggro
Great Brituin
Austria
Belgium
Bulgaria
Belarus
Cyprus
Denmark
Germany
Greece
Spain
Estonia
Finland
France
Hungary
Cinata
Ireland
Iceland
Latvia
Lithuania
Moldova
Norway
Portugal
Poland
Romania
Russia
Slovakia
Slovenia
Albania
Serbia
Sweden
Turkey
Ukraine
Kosovo
Italy
Label
Geographical
Alps
Arctic Sea
Atlantic Occan
Balkan Peninsula
Baltic Sea
Big Country
Black Sea
Central Europe
Coastal State
Eastern Europe
Island
Landlocked
Mediterranean Sea
Northern Europe
North Sea
Southemn Europe
South-Las Europe
Western Lurope
Linguistic
Baltic Family
Cyrillic Alphabet
English-speuking
French-speaking
German-speaking
Germanic Family
Non-TLatin Script
Romance Family
Slavie Family
Ugro-Finnic Finnic Family
Political
Baltic States
Benelux
v
vuww ww win o "
BU member
New Member
BU-15 Member
Furo currency
v /
G7 member
Monarchy
NATO member
Neutral
Former Yugoslavia
Post-communist
Post-soviet
Schengen Area
wy
Visegrad Four
Table 5: Labels for grouping countries.

---

Jnb class.
acc.
.93
.93
.93
.93
.93
.90
.95
.97
.93
.93
.93
Job
class,
.80
900
.88
82
.87
.88
.83
.87
.82
.83
.88
.85
w GDP
w/ GDP
Cort.
.39
Corr.
.63
.38
.65
66
58
337
33
.54
.57
41
Is cast-
west?
0.08
cast- west?
V
7
0.69
Job prestigre applied to country
Job prestigre applied to country
COTE.
.50
.43
.53
44
.48
-50
.49
.31
.53
445
.46
.48
.47
COT..
.54
.52
.48
.40
.49
550
.43
.46
.46
333
.47
.57
.34
.46
Germanic Ing.
Baltic state
Post Post- Saviel
Yugoduuvaa
Post- Post-Soviet
Yogoslavia
The main dircction
# label
German
German
German
German
German
German
German
German
German
German
The mainn direction
@ Label
EU-15
BL-115
EU-15
EU-15
BU-15
Atlantic
.46 B8--15
Alps
BL
alps
ood.
3,31
-.37
+38
-44
-.4
.32
-36
-55
-36
.47
+38
-36
-35
001r.
-54
..4
-44
-34
-53
-53
.31
-55
.48
-36
-51
-53
-42
Post SoSSiieee
Momochy
Yogoslvia
Post com-.
Yugoslavia
Yugoslavia
Yugoslavia
Post comm.
Yugoslavia
Yugodaviaa
Yugoslasia
6 lahel
North
North
North
North
North
North
North
North
North
North
North
label
German
North
Neutral
EU-15
CorE.
w/ GDP
.80
.81
.81
.81
.78
81
.80
.81
.81
881
CuT.. w/ GDP
.73
.71
.73
.70
.72
.68
.68
.72
.73
.72
.60
.71
Distilled Multilingual Sentence Encoder (dist idistiluse iuuee -aaseaaeeee mu mullilling Mitiinggall--- cased v2)
Is east- west?
In cast- weat?
7
1
7
V
/
Multilingual Paraphrase MPNet (pa (paraphrase multilillmiiieiii ingualgglllll mpneet base v2)
colr.
-70
.71
.69
.70
.71
.72
68
66
.71
.69
.70
.68
62
.69
corr.
.67
65
.69
662
.62
.63
-56
644
.63
63
.69
.64
.58
63
Country presige
Countr presige
Germanic Ing.
Germanic Ing.
Germonic Ing.
Post Post comm.
Germmic Ing.
Post-comm..
Post-comm.
Germonic Ing.
Germanic Ing.
Post-comm.
Germanic Ing.
The main dircction
@ label
The main direction
@ label
Germanic
Slavic Ing.
Germanic
West
West
West
West
West
West
West
West
cod.
.71
-69
+70
-70
-70
-08
-70
.68
.70
:71
-68
-.1
-69
-70
corr.
.61
-61
.60
-63
-63
-06
-61
-558
.64
-61
+58
-64
.60
-61
Germanic lng.
Germanic Ing.
Germanie Ing.
Germanic Ing.
Slavie Iog.
Slavie log.
Slavic Ing.
Slavic Ing.
Slavic Ing.
Slavic Ing.
Slavie Ing.
Slavie Iog.
Slavie log.
Post- Post-comm.
Post-comm.
Post-comm.
Post-comm.
Post comm.
Post-comm.
Post comm.
Post comm.
@ label
Balkan
Balkan
Balkan
@ label
Balkan
EU-15
w/ GDP
Corr: w/ GDP
Core:
.80
.81
.80
800
.81
81
.80
81
.80
.81
.62
.73
.72
.68
.71
.71
.67
.66
.74
-74
.70
.65
.70
Is east- weat?
Is eant- weat?
V
COIT:
700
.71
688
68
.71
-11
71
71
-22
.71
.71
.72
63
-70
com:
66
66
-63
62
.63
63
.64
63
.61
62
.69
.64
.61
63
Country of origin
Country of origin
Germonic Ing.
Germanic Ing.
Germanic Ing.
Gemunic Ing.
Post-comm.
Post-comm.
Post-comm.
Stavic Ing.
Slavic Ing.
Slavic Ing.
Slovie Ing.
Germanic
Slavie Ing.
The main direction
@ label
The maiin direction
< label
Bulkan
West
West
West
West
West
West
West
West
West
West
con
-70
-6
.69
-70
-69
-0
-67
-67
-69
-69
-67
-67
-71
.69
corE:
-66
-61
.59
-64
-62
-65
-61
-57
.63
-65
.58
-62
-63
-62
5o Germani Ing.
de Germanic Ing.
Germanic Ing.
Germanic Ing.
West Germanic Ing.
Germanic Ing.
Germanic Ing.
Post-conm.
nont-comm.
el Post-conm.
Post Post-comm comm.
Post comm.
Slavic Iog.
Slavic Ing.
Slavie Ing-
cl Slavie Ing.
Slavic Ing.
Slavic Ing.
Slavic Ing.
Slavic Ing.
Slavic Ing.
pt Slavie Ing.
Germanic
S label
Balkan
label
Balkan
Average
Log
Log
Table 6: Detailed per-languun S results results for Multilingual M
Cs
fi
Ir
hu
it
es
it
pu
zo
ru
Multitinn
MPNet and Distilled Multilingual Sentence Encoder "

---

Job
class.
.83
.90
.95
.85
.83
.87
.00
.87
.88
.88
Job class.
acc.
.85
.87
.98
.92
.83
922
.93
.85
.92
w/ GDP
we GDP
Corr.
.99
333
Corr.
57
41
west?
7
V
0.23
Is east-
west!
0.08
Job prestigre applied country
prestigre applied country
corr.
.36
.48
.35
.41
.38
.37
.45
.49
-3
.40
.37
.38
.7
.37
corr.
.48
.35
-31
.44
-37
.32
.41
.34
.4
-32
Germanic Ing.
The main direction
label
Yugodavia
Atlantic
Yogoslavia
Yogoslavia
Yogoslavia
Adamtic
Yogoslavia
Neutral
Yugodavia
Yogeslavia
Schengen
Yogoslavia
The main diroction
4B label
Balkon
Post- Poot-Sovict
Romance
Is Islund
South
Visegrad
Visegred
German
Baltic
Yogoslavia
Big
+.37
.:4
.43
-46
+33
-,36
-40
-47
-.33
-35
-33
-.31
44
+39
coar.
.42
.46
-40
-.32
-388
-.37
-.36
-31
+.4
55
-39
runance Ing
Romance Ing.
O Label
Atlantic
Baltie Ing.
Big
South Elst
South
South
Landlocked
EU-15
Alps
Atlantic
BU
Non-1 Latin
EU-15
label
east
Baltic state
South
Visegrod
Neutral
South
Black Sen
South
South
Corr. Wi GDP
wet GDP
45
54
.58
46
43
.52
.47
552
.41
.38
.50
Con:
.64
.32
.66
.66
.65
41
weat?
7
V
Is east- west?
V
/
0.85
COTT.
53
62
.51
.3
63
.61
62
.63
58
.88
51
.45
51
.66
CUTF.
599
333
664
.46
73
.60
61
.46
550
.57
55
-S8
.48
.55
Country prestige
County presige
Slavie Ing.
Slavic Ing.
Yagoslavia
Slavic Ing.
Slavie ing.
Yogoslavia
Slavic Ing.
Post-c Post-comm.
Post Rol--comm.
Post comm.
Slavic Ing.
Slavic Ing.
The main direction
Jabel
Balkan
Balkan
Slavic
Balkan
North
The main dircction
39 label
I7U-15
Slavic
EU-15
Atanti
Neutral
Slavic
North
cu15
cu15
+,36
:49
.43
+50
.43
-53
.46
-58
..48
-50
-55
+58
+38
.49
corr.
-67
:45
-55
-44
-62
-.5
+588
-45
-54
-.3
-45
-51
-55
Germanic Ing.
Germanic Ing.
3 label
Stavic Ing.
EU-15
EU-15
Slavic Ing.
Slavic Ing.
Neutral
e label
Slavic Ing.
North-sea
EU-1S
Post-comm
cosstal
NATO
Einlishh
Balkan
Atlanti
Slavie
Famr
eu
cn
cuts
cui5
Post
Earo
w/ GDP
w/ GDP
LaBSE (sen teeneeeee ce trans Kormers LaBsE))
Corr
48
51
45
45
42
.52
.51
477
51
45
41
48
CorE:
.60
.57
445
.88
.39
34
.44
Ix east- weat?
7
/
Is cast- west?
7
V
0.85
-59
63
.61
.1
59
64
61
-59
.60
43
54
.57
.71
.35
59
64
.1
.61
449
500
49
557
.47
-33
Country of origin
Country of origin
Germanic Ing.
Yogodavia
Slavic Ing.
Slavie Ing.
Yogoslavia
Slavic Ing.
Slavie Ing.
Slavie Ing.
South-East
Yugoslavin
Slavic Ing.
Post comm..
Slavie Ing.
The mainn direction
Jabel
cu15
Slavie
Balkan
North
XLM-R finetuned on NLI
The main direction
@ label
Beaclux
Slavie
EU-15
Cyrllic
Adamii
Cyrilii
Neurrl
Slavic
Atlantic
cont:
..37
.48
.44
-51
+.37
-447
-66
:449
.45
-48
49
.55
-33
-47
0OTT
-51
-44
..7
-.52
-.56
-.56
.44
-44
:45
-33
:44
58
Post-comm.
Slavic Ing.
Slavic Ing.
Slavic Ing.
label
en
North
Atlanic
Euro
cu
Balkan
North
North
f cu15
Nort
North
9 label
Atlantic
Neutrl
EU-1S
west Balkon
Earo
coastal
Euro
Earo
Balkan
Euro
Lag
Ling
de
el
en
ro
ru
bg
os
h
Ir
ha
it
pt
Soiguaage results for LaBSE and XLM-R finetuned on NI.I.
Table Detailed


Reviews:
[{"review_id": "i524sfmYzv", "reviewer": "Reviewer_3DFr", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Our initial motivation was problematic NLP applications that evaluate individuals based on their CVs or records found online. We viewed our study as exploratory; we did not actively search for use cases where the discovered discovered could potentially be harmful. However, one example we found was that using the Distilled mUSE in such a sensitive (and inherently problematic) application, being from a poorer country, might have a similar effect as working in low-profile jobs. \n\nThank you for mentioning the work of Paullada et al., which is very interesting, and we were unaware of it. There are indeed implicit normative assumptions when preparing the data we work with. We consider countries' geographical and political features present in the Western mainstream discourse, which are not necessarily the most appropriate in all contexts. Paullada et al. talk about benchmarking datasets, which are costly to produce, which motivates the community to reuse the datasets often outside of their original context. Designing templates is cheap compared to collecting a dataset from annotators. Also, given the exploratory nature of our study, we do not provide a benchmark others can compete on. Hence, the risk of problematic use of our data outside of the original context is relatively small.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Sound case study on bias analysis covering multiple languages", "reasons_to_reject": "- Unclear impact of the proposed study\n\n## Recommendation\n\nThe proposed case study is interesting and sound. Moreover, the paper is well organized and written. My main concern is about the impact and purpose of such study, which appears to be not sufficiently remarked in the paper. Any dataset has some biases since it is a limited collection of data [1]. The question is whether this type of bias is (1) harmful; (2) relevant to application domains of interest. I understand the narrative of the study, but I fail to depict scenarios where the identification of such biases is a mandatory requirement. For instance, should we consider some de-biasing process?\nCould the authors clarify the motivations and impact of proposed study?\nOverall, I don't see any strong inconsistency in the work. For this reasons, I'm more inclined to recommend acceptance. Further discussion on my concern would be highly appreciated to increase my score.\n\n## Comments\n\n[Table 2, Table 3] Could the authors report the standard deviation values as well?\n\n## Typos and Writing\n\n[Section 2.2, Line 117] 'the first:w PCA dimension'\n\n## References\n\n[1] Amandalynne Paullada, Inioluwa Deborah Raji, Emily M. Bender, Emily Denton, Alex Hanna:\nData and its (dis)contents: A survey of dataset development and use in machine learning research. Patterns, 2021", "questions_to_authors": "", "paper_topic_and_main_contributions": "In this work, the authors show that several sentence embedding models present social biases in how country names and jobs are encoded. In particular, a set of metadata features are manually associated with top PCA components for investigation based on correlation analysis. Experimental results show high correlation between top PCA component with two features: Western and Eastern Europe economic distinction and Gross Domestic Product. Low correlation is then observed for jobs prestige. These observations are consistent across multiple languages.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "SNRrM642r8", "reviewer": "Reviewer_os6q", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Missing formulas.** Thank you for pointing out less clear points in our writing. We decided not to write formulas to save space in a 4-page short paper. We can address the writing issues in the camera-ready version with the extra page.\n\n**Correlation.** Every time we mention correlation, it is the Pearson correlation coefficient. In Section 2.4, we talk about the Person correlation between the extracted most prominent (PCA) dimension from the embedding with the indicator vectors for country features (i.e., one if the country has the quality, zero if it does not; e.g., if the country uses Euro as its currency). This is sometimes called Point-biserial correlation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Timely study. Interest to the broader NLP community. \n\nSeveral results are fascinating. Very few such studies have been done on multilingual embeddings in the past.", "reasons_to_reject": "There are no mathematical expressions showcasing the operational steps. This makes the paper extremely difficult to understand. A reader has to entirely rely on intuition to understand the inherent processes. \n\nThe entire 2.4 section needs to be elaborated clearly. \n\nNot sure which correlation the authors have used", "questions_to_authors": "", "paper_topic_and_main_contributions": "This is an exploratory paper that studies and exposes how pre-trained multilingual language models inherently contain bias or make distinctions among European countries. The authors show a reflection of geopolitics (eastern vs Western European countries) and economy based (in terms of GDP) distinctions. They relied on hand-crafted templates and PCA to showcase their findings. PCA finds the strongest dimensions across which sentence embeddings corresponding to the templates are placed, and the dimension values correlate with political and economic distinction among Western and eastern European countries. For job profiles, models show a clear distinction between high and low-profile jobs; however, the majority of them do not show a strong correlation between job prestige dimensions and country dimensions.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Zl5Ics1MMS", "reviewer": "Reviewer_QesP", "summary": "", "questions": "1. Why did you choose to machine translate, instead of using existing data from the corpora, like EuroParl?\n2. Would it make sense for you to elaborate on the potential ethical concerns based on your findings?\n3. How were the templates and models chosen? Could you briefly explain your rationale behind it?\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Why MT.** The method we use requires only minimum semantic differences between the sentences, e.g., in the country sentences, we want the only word that changes to be the country name. Otherwise, as previous work showed, the most prominent PCA dimensions would capture linguistic features. Using authentic parallel sentences would introduce too much linguistic diversity for the method to work well. Therefore, we use templated sentences that we machine-translate into multiple languages.\n\n**Ethical concerns.** As mentioned in the response for Reviewer 3DFr, the application areas we had in mind when designing the experiments were problematic NLP applications, such as automated processing of CVs with potentially harmful consequences for individuals. Moreover, when people attribute importance to economic and geopolitical factors (especially when thinking about individuals), in some sense, it reflects their moral values. However, to what extent the same can be said about sentence embeddings is questionable, and we did not feel qualified to discuss this topic in detail.\n\n**Template design.** We want the sentences to capture the meaning as simply as possible. The templates should be structurally simple, so their linguistic features do not influence the sentence embedding. Also, simple sentences get better translated using MT. We use a set of synonymous templates for each category so that potential meaning nuances average out in the embedding space.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper investigates some of the most important questions in our field. Knowing the biases in our model and data helps us make the better decision.", "reasons_to_reject": "The scope of this study is relatively small, even for a focus contribution, thus making the results not as reliable. ", "questions_to_authors": "1. Why did you choose to machine translate, instead of using existing data from the corpora, like EuroParl?\n2. Would it make sense for you to elaborate on the potential ethical concerns based on your findings?\n3. How were the templates and models chosen? Could you briefly explain your rationale behind it?\n", "paper_topic_and_main_contributions": "This paper aims to analyze the biases connected to European countries in multilingual sentence representation models. Specifically:\n\n* The strongest dimension in all models correlates with the political and economic distinction between Western and Eastern Europe and the GDP.\n* The models can distinguish between low and high-prestige jobs.\n* Except for one model, all models only loosely associate job prestige with country prestige.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The graphs in Fig. 1 are a little less intuitive.\nExpanding the scope of this study and making it a long paper would be great!"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Summary (adapted from reviewer os6q): This is an exploratory paper that studies and exposes how pre-trained multilingual language models inherently contain bias or make distinctions among European countries. The authors show a reflection of geopolitics (Eastern vs Western European countries) and economy based (in terms of GDP) distinctions. Prestige related to jobs is also explored and is found to generally not correlate between country-related prestige.

The reviewers found this study to be generally sound (3/3/3) and acknowledged that it covers an interesting and novel topic.

Two reviewers raised concerns that the stereotypes explored in this work may prove harmful. This was well-addressed in the limitations and ethical considerations section of the paper.

There was some disagreement about the ease with which the paper could be understood, with one reviewer citing difficulty, one reviewer praising how well-written it is, and the other not mentioning it at all. The reviews suggest some areas that would benefit from expansion of the paper into 5 pages if accepted.

Overall, this paper provides an interesting and unique contribution, even if it addresses just two small questions about the biases of these models.
Decision: Accept-Findings