SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypes (2024)

Abstract: While generative multilingual models are rapidly being deployed, their safety and fairness evaluations are largely limited to resources collected in English. This is especially problematic for evaluations targeting inherently socio-cultural phenomena such as stereotyping, where it is important to build multilingual resources that reflect the stereotypes prevalent in respective language communities. However, gathering these resources, at scale, in varied languages and regions pose a significant challenge as it requires broad socio-cultural knowledge and can also be prohibitively expensive. To overcome this critical gap, we employ a recently introduced approach that couples LLM generations for scale with culturally situated validations for reliability, and build SeeGULL Multilingual, a global-scale multilingual dataset of social stereotypes, containing over 25K stereotypes, spanning 23 pairs of languages and regions they are common in,1 with human annotations, and demonstrate its utility in identifying gaps in model evaluations. Content warning: Stereotypes shared in this paper can be offensive.

heading: 1 Introduction
text: Generative multilingual models (Brown et al., 2020; Chowdhery et al., 2022; Anil et al., 2023) have gained popular usage in the recent years due to their gradually increased functionalities across languages, and applications. However, there has been a severe lack in cross cultural considerations in these models, specifically when it comes to evaluations of their safety and fairness (Sambasivan et al., 2021). These evaluations have been known to be largely restricted to Western viewpoints (Prabhakaran et al., 2022), and typically only the English language (Gallegos et al., 2023).
1Languages (in ISO codes): ar, bn, de, es, fr, hi, id, it, ja, ko, mr, ms, nl, pt, sw, ta, te, th, tr, vi; Details in Table 5.
This is inherently problematic as it promotes a unilateral narrative about fair and safe models that is decoupled from cross cultural perspectives (Arora et al., 2023; Zhou et al., 2023). It also creates harmful, unchecked effects including model safeguards breaking down when encountered by simple multilingual adversarial attacks (Yong et al., 2024).
As language and culture are inherently intertwined, it is imperative that model safety evaluations are both multilingual and multicultural (Hovy and Yang, 2021). In particular, preventing the propagation of stereotypes – that can lead to potential downstream harms (Dev et al., 2022; Shelby et al., 2023) – is crucially tied to geo-cultural factors (Hinton, 2017). Yet, most sizeable stereotype evaluation resources are limited to the English language (Nadeem et al., 2021; Nangia et al., 2020). While some efforts have created resources in languages other than English (Névéol et al., 2022), they are limited to specific contexts. On the other hand, some approaches such as by Jha et al. (2023) have global coverage of stereotype resources but are restricted to the English language alone. Conse-
842
quently, they fail to capture uniquely salient stereotypes prevalent in different languages of the world, as simply translating them to other languages will lose out on cultural relevance (Malik et al., 2022).
In this work, we address this critical gap by employing the SeeGULL (Stereotypes Generated Using LLMs in the Loop) approach (Jha et al., 2023) to build a broad-coverage multilingual stereotype resource: SeeGULL Multilingual. It covers 20 languages across 23 regions of 19 countries they are commonly used in. It contains a total of 25,861 stereotypes about 1,190 identity groups, and captures nuances of differing offensiveness in different global regions. We also demonstrate the utility of this dataset in testing model safeguards.

heading: 2 Dataset Creation Methodology
text: Stereotypes are generalizations made about the identity (id) of a person, such as their race, gender, or nationality, typically through an association with some attribute (attr) that indicates competence, behaviour, profession, etc. (Quinn et al., 2007; Koch et al., 2016). In this work we create a multilingual and multicultural dataset of stereotypes associated with nationality and region based identities of people. We use the methodology established by Jha et al. (2023), which is constituted primarily of three steps: (i) identifying relevant identity terms, (ii) prompting a generative model in a few-shot setting to produce similar candidate associations for identity terms from (i), and finally (iii) procuring socially situated human validations for those candidate associations.
We chose 20 languages that diversify coverage across global regions (A.1) as well as prevalence in documented LLM training datasets (Anil et al., 2023). Some languages are used as a primary language in multiple countries with distinct geocultures and social nuances (e.g., Spanish in Spain and Mexico). We consider each language-country pair individually and conduct the following steps separately for each pair.

heading: 2.1 Identifying Salient Identity Terms
text: Salient identities and stereotypes can vary greatly across languages and countries of the world, and a multilingual stereotype dataset needs to reflect this diversity. To reliably create the dataset at scale, we scope and collect stereotypes only about national, and local regional identities.
Nationality based demonyms: We use a list of 179 nationality based demonyms in English,2 and translate them to target languages.3 In languages such as Spanish, Italian, and Portuguese, where demonyms are gendered (e.g., Bolivian in English can be Boliviano (masculine) or Boliviana (feminine) in Italian), we use all gendered versions.
Regional demonyms We source regional demonyms (such as Californians, Parisians, etc.) within each country from established online sources in respective languages (see A.8 for details). A lot of these demonyms are present only in the respective target language without any English translation, such as the Dutch demonym Drenten for a person from region of Drenthe in Netherlands), and the Turkish demonym Hakkârili for a person from Hakkâri province in Turkey. Additionally, for languages with gendered demonyms, we include all gendered forms for all the regional identities. Finally, for the languages for which we collect stereotypes in multiple countries (for e.g., Spanish in Mexico and Spain) we gather regional identity terms for both locations separately.

heading: 2.2 Generating Associations
text: To generate associations in different languages, we use PaLM-2 (Anil et al., 2023), which is a generative language model trained on large multilingual text across hundreds of languages. Using few shot examples of stereotypes from existing datasets (Nadeem et al., 2021; Klineberg, 1951), we instruct the model to produce candidate tuples in the format (id, attr) (Jha et al., 2023). The model’s demonstrated abilities for cross lingual functionalities (Anil et al., 2023; Muller et al., 2023; Fernandes et al., 2023) support its effective usage for our task of multilingual generation. The template Complete the pairs: (id1, attr1)(id2, attr2)(id3, translated in different languages is used to prompt the model. The generated text gives us a large volume of salient candidate associations.

heading: 2.3 Culturally Situated Human Annotations
text: Associations generated in steps so far need to be grounded in social context of whether they are indeed stereotypical. We obtain globally situated annotations for tuples in each of the 20 languages
2https://w.wiki/9ApA 3https://translate.google.com/
in the country or region of country they are commonly used in (e.g., tuples in French are annotated by French users in France, tuples in Tamil are annotated by Tamil users in Tamil Nadu, India). For languages Bengali, Portuguese, and Spanish that are common in two countries each, we obtain human annotations from both countries. Annotators were diverse in gender, and compensated above prevalent market rates (more details and annotation instructions in A.3).
Stereotype Annotations. Three annotations are collected for each candidate tuple in their respective language. The tuples are also annotated in country specific manner, i.e., French tuples are annotated by French users in France specifically. We adopt this approach since region of annotator residence impacts socially subjective tasks like stereotype annotations (Davani et al., 2022). In addition, for languages that are common in multiple countries, we get separate annotations in each country (e.g., Spanish in Spain and Spanish in Mexico). We obtain annotations for a total of 35,131 tuples in this step.
Offensiveness Annotations. After obtaining annotations on whether a tuple is a stereotype, we follow up to estimate how offensive it is. For each tuple that gets annotated as a stereotype by at least one annotator, we obtain human annotations on how offensive it is. We do so by obtaining three in-language, globally situated annotations for each attribute term in the dataset on its degree of offensiveness on a Likert scale of ‘Not offensive’ to ‘Extremely Offensive’. Any tuple in our dataset is estimated to be as offensive as the average offensiveness rating of the attribute term in the tuple. A total of 7159 unique attribute terms are annotated for their degree of offensiveness in this step.

heading: 3 Dataset: SeeGULL Multilingual
text: We introduce the dataset SeeGULL Multilingual (SGM), a large scale dataset of stereotypes with broad global coverage. The stereotypes are in the form of (identity term, attribute), and include information such as how frequently they were identified as stereotypes, and their mean offensiveness rating. A snapshot of the data is in Table 1, and the data, and data card are available online 4 and detailed in Appendix A.1.
4https://github.com/ google-research-datasets/SeeGULL-Multilingual
Coverage: SGM covers stereotypes in a total of 20 languages, as collected from 23 regions across 19 countries of the world. The dataset has a total of 25,861 stereotypes about 1,190 unique identities - including gendered demonyms where applicable - and spread across 7,159 unique attributes.
Overlap with English SeeGULL: The English SeeGULL (SGE) resource from (Jha et al., 2023) has approximately 7,000 stereotypes about nationalities. SGM has 9,251 unique nationality based stereotypes, of which, only 949 stereotypes are in common with SGE. These 949 unique stereotype occur as a total of 2370 tuples in SGM, present in various languages in different ways, such as (Afghans, terrorists) appearing as (afghani, terroristi) in Italian, and (Afghanen, terroristen) in Dutch. The maximum overlap is seen in the Spanish dataset collected in Spain (13.2%), and Portuguese in Portugal (13%), while the least overlap was for Tamil (4.8%), and Hindi (5.37%).5 Additionally, 10,292 regional demonym based stereotypes are all newly introduced in SGM, making the overall dataset overlap with SGE about 5%.
Country-level Differences: Languages contain socio-cultural information which can differ at places of use. Among the languages covered in our dataset SGM, the languages Bengali, Spanish, and Portuguese are commonly used across two countries each. We observe this difference in stereotypes for each of these three languages by obtaining human annotations across the two countries. Some examples of the same are in Table 2. For e.g., as gathered by annotations, the stereotype Crimeanos, ladroes (or Crimeans, thieves) in Portuguese is prevalent in Portugal but not in Brazil. At an aggregate level, of the 1138 common tuples annotated in Portuguese in Portugal and Brazil, 45.4% of the tuples were marked as stereotypical by at least 2 annotators in Portugal compared to 74.6% tuples marked as such in Brazil. This trend is consistently noted for each of the 3 languages (A.6). It highlights the geo-cultural subjectivity of stereotypes, and how perspectives differ despite sharing the same language.

heading: Stereotypes about Gendered Demonyms.
text: Some languages have gendered nouns and specifically, gendered demonyms. These gendered demonyms result in gendered, and sometimes intersectional stereotypes about people in different
5Based on exact match of translated stereotypes.
countries. SGM records these for languages Spanish, Portuguese, Italian, and Dutch. For e.g., in Portugal, the identities Bragantinos (male) and Bragantinas (female) associated with the region of Braganca are associated with attributes party-goers and conservative respectively. We see most notable differences between attributes associated with gendered demonyms in the languages Portuguese and Spanish, with attributes about beauty such as pretty, or brunette being associated with feminine identities, while warrior, or brave with masculine identities.
4 SGM for Analysis and Evaluations 4.1 Offensive Stereotypes in SGM While all stereotypes can have negative downstream impacts, some associations are especially offensive, such as associations that imply degeneracy and criminality. Aggregating over stereotypes about nationalities across all languages in SGM, we note how Albania and Rwanda have some of the most offensive stereotypes associated with them, while Singapore, Canada, and Maldives have the least offensive stereotypes associated (A.4). Figure 1 shows the aggregated offensiveness associated with different countries of the world.
Table 3 showcases some examples of highly offensive stereotypes associated with different national and regional identities (also A.4).
The perception of an attribute or stereotype as offensive or not can vary by language, and geoculture (Zhou et al., 2023). So we also aggregate over the individual languages, and observe that Italian and Swahili have the most number of offensive stereotypes with about 22% of all stereotypes for
these languages being marked as “Somewhat Offensive” or more. On the other extreme, Hindi (1.83%) and Korean (2.66 %) are the languages having the least fraction of offensive stereotypes (full list in A.4 Table 7) .
4.2 Foundation Model Evaluations with SGM Evaluating stereotyping by multilingual language models is challenging due to paucity of resources in languages apart from English. SGM enables us to create an evaluation set measuring whether a system endorses stereotypical associations across a socially situated, globally localized, and more comprehensive set of stereotypes, and whether the extent of endorsing stereotypes differs by language.
We adapt evaluation methods for measuring bias in inference capabilities (Dev et al., 2020; Parrish et al., 2022) to create the evaluation of foundation
models depicted in Figure 2. Each question in the task contains only one stereotypical answer, with other identity terms randomly sampled. We create an evaluation set from stereotypes in SGM to create 4,600 questions, drawing 100 samples across each language, country, and demonym type. These stereotypes are almost entirely unique to SGM, with only 7% of also present in SGE. The task is generative, as generative models and systems are increasingly common in downstream applications, and they can produce unexpected answers to questions (Anil et al., 2023), or reflect more nuanced safety policies related to stereotypes (Glaese et al., 2022; Thoppilan et al., 2022).
We evaluate four different models: PaLM 2, GPT-4 Turbo, Gemini Pro, and Mixtral 8X7B. We observe that all models endorse stereotypes present in SGM, and at different rates when the same queries are asked in English (Table 4). We note that PaLM 2 has the highest rate of endorsement, while Mixtral demonstrate the lowest. Our results also show that English-translated queries would have missed a significant fraction of stereotype endorsements in three out of four models, further demonstrating the need for multilingual evaluation resources. Figure 3 also notes that models tend to endorse stereotypes present in different languages at different rates. These findings underline the critical gap filled by SGM and the forms of multilingual evaluation it enables. We also encourage future work to explore other ways to create evaluation sets from SGM that can measure expressions of representational harms and stereotypes.

heading: 5 Conclusion
text: For holistic safety evaluations of multilingual models, English-only resources or their translations are not sufficient. This work introduces a large scale, multilingual, and multicultural stereotype re-
source covering a wide range of global identities. It also exposes how these stereotypes may percolate unchecked into system output, due to the prevalent lack of coverage. In considerations of model safety, cross cultural perspectives on stereotypes, their offensiveness, and potential harms must be included. We encourage future work to explore other methods to utilize SGM to measure expressions of representational harms and stereotypes within application-specific contexts for global users.

heading: Acknowledgements
text: We thank Kathy Meier-Hellstern, Dasha Valter, and Jamila Smith-Loud for their helpful discussions and feedback, and Clément Crepy, Alex Castro-Ros, Sumanth Doddapaneni, Nithish Kannen, Ahmed Chowdhury, Deniz Kose, Shantanu Thakar, Mindy Khanijau, Martin Borgt, and Zu Kim for spot validation of multilingual seed data. We also thank the anonymous reviewers for their feedback during the review process.

heading: Limitations
text: The dataset created in this work is constrained by the resources needed to create large scale, quality data. The dataset covers 20 languages and not the full range of many thousands of languages and dialects used across the world. Unfortunately, generation quality of most models is limited to few languages currently which guide our methodology. Further, we obtain annotations from 23 regions, whereas it could be from a much larger set given the spread of the 20 languages. This is constrained both by the availability of annotators and the cost of data annotations. Next, we limit the identity terms of recorded stereotypes to be demonyms associated with nationalities and regions within each nation. We also limit the granularity with which regions are considered, and also don’t include regions within all countries at a global scale. These are design choices for reliably collecting stereotypes at scale, guided by how stereotypes are socio-culturally situated (Jha et al., 2023; Hovy and Yang, 2021). While this helps create a dataset that is grounded in local knowledge, there are other stereotypes at other levels of granularities, and about other identities that are not covered by this work. We hope that this work acts as a foundation, based on which larger, multilingual safety datasets can be built.

heading: Ethical Considerations
text: We emphasize that this dataset does not capture all possible stereotypes about any identity, or stereotypes about all geocultural identities. Thus, this dataset should not be used alone to categorize any model or its output as completely devoid of stereotypes. Instead careful considerations should be made by dataset users depending on the intended application. Further, we explicitly call out the intended usage of this dataset for evaluation purposes in the attached Data Card (A.1). This dataset contains a large number of stereotypes which can help build model safeguards. We caution users against unintentional, or malicious misuse.

heading: A Appendix

heading: A.1 Dataset
text: The dataset contains 25,861 annotated stereotypes across 23 pairs of languages and regions they are common in (Table 5), and is available online 6.
The first two columns of Table 10 describes the languages, countries (of annotations), and the total annotations that are being released as part of this dataset. Since data disagreements are features of subjective data (Davani et al., 2022), we consider any associations with at least 1 annotation (of 3 annotators) as stereotype to be sufficient for the tuple to be included in the published dataset. The filtering of the data for usage is left to the user. The data card detailing intended usage, data collection and annotation, costs, etc. is also made available online 7.
Table 6 shows the distribution of tuples across the nationality and regional axis. Of the 25,861 annotated tuples, 19,543 stereotypes have unique English translations (via Google Translate API). The differences arises due to the fact that we, by design, get a few tuples annotated in two different
6https://github.com/ google-research-datasets/SeeGULL-Multilingual
7https://github.com/ google-research-datasets/SeeGULL-Multilingual/ blob/main/SeeGULL_Multilingual_Data_Card.pdf
countries speaking the same language (section 3 and A.6). Finally, stereotypes having different gender based identity terms but with same attributes (e.g (mauritana, árabe) and (mauritanos, árabe)) are back-translated to English in exact same way and are thus counted as such.

heading: A.2 Related Stereotype Resources
text: Stereotype resources are essential for generative model evaluations, and a large body of work pushes to increase the overall coverage of these resources (Nadeem et al., 2021; Nangia et al., 2020; Jha et al., 2023). These resources help significantly bolster model safeguards (Nagireddy et al., 2023; Bai et al., 2023; Jha et al., 2024). Thus, it is imperative that the resources cover global identities, to enable models across modalities and languages to be safe and beneficial for all. There have been attempts to increase these resources across languages (Névéol et al., 2022; Sólmundsdóttir et al., 2022; Vashishtha et al., 2023), and cultures (Bhatt et al., 2022; Dev et al., 2023). However, due to the cost of curating, these resources are often limited in both size, and global coverage. In this work, we address these challenges by leveraging social information captured and generated by multilingual models and globally situated annotations.

heading: A.3 Annotation Details
text: We get annotations from humans for two different task. The first task, called Stereotype Annotation is used to determine if an (identity, attribute) tuple is considered as stereotypical or not. The second task, Offensive Annotation is for determining the offensiveness of a given attribute in a particular country. All annotations were procured through a partner vendor who handled the recruitment, obtained informed consent, and provided clean, anonymous ratings within each task.
Annotators were recruited such that they used the target language and resided in the country of
popular usage. They were also selected such that every data point was annotated by at least one nonmale identifying person. Annotators were paid above prevalent market rates and respecting local minimum wage laws in respective countries.

heading: A.3.1 Task: Stereotype Annotation
text: Annotators: Since we are documenting stereotypes in a multilingual setting, our annotation tasks are in different target languages depending on the country the task is being conducted in. Annotators are proficient in English reading and writing, as well as in the target language that the data is in. Moreover, since we have regional identities corresponding to each country, we ensure that annotators are people living in that country. Finally, as described in section 3, for the languages of Spanish, Portuguese, and Bengali, we perform human annotations separately in two different countries each. Labels: The annotators had the options to select between three labels.
• Non-Stereotype (NS) • Stereotype (S) • Unsure (U) (with justification)
The label Unsure captures the case where either the annotator does not have full confidence, or if the tuple is factual in nature.

heading: A.3.2 Task: Offensiveness Annotation
text: Annotators: The offensiveness annotation task is also in different target languages per dataset. Annotators are proficient in English reading and writing, as well as in the target language that the data is in. The countries that the raters are from is also different per data segment (for example, Spanish users in Mexico, Spanish users in Spain, Bengali users in Bangladesh, etc.). Each data point (attribute term) is rated by 3 annotators of diverse gender identities. Labels: The annotators need to label each attribute as one of the following on a Likert scale.
• Not Offensive (NO) (-1): if the attribute is not perceived as offensive at all, or is positive or neutral such as ’good’, ’okay’, etc in English. Consider similar terms in the language the data is in for this task. • Slightly Offensive (O1) (+1) • Somewhat Offensive (O2) (+2) • Moderately Offensive (O3) (+3) • Extremely Offensive (O4) (+4): if the at-
tribute is perceived as extremely offensive when associated with the identity (like fe-
male, Russian, Asian, Jew, etc.) of a person such as attributes implying criminal behavior and activities, such as ’criminal’, ’terrorist’, etc in English. Consider similar terms in the language the data is in for this task. • Unsure (with justification) (U) (0): if the annotator is not sure about if the attribute is offensive.
The answers can vary from Extremely offensive to Not offensive. The integers from (-1) to (+4) are used for calculating the mean offensiveness of an attribute and are not visible to the annotators.

heading: A.4 Offensiveness
text: For all the stereotypes in SeeGULL Multilingual, we also get the offensive annotations of the corresponding attributes on Likert scale (A.3.2). For all the attributes, we average out the offensiveness annotations by the three annotators and call it the ”mean offensiveness” score.
Table 7 shows the percentage of stereotypes that are annotated as ”Somewhat offensive (O2)” or higher, per language and country.
Finally, stereotypes in SeeGULL Multilingual can be thought of either belonging having a nationality based demonym or a regional (within a country) based demonym. For all the nationality based demonyms in SGE, we group them based on their
corresponding countries and get an average of offensiveness scores associated with them. Table 8 shows the top 20 countries/regions which have the most offensive stereotypes associated with them. Similarly, the table 9 lists out the countries having the least offensive stereotypes associated with them.

heading: A.5 Overlap with English SeeGULL
text: SeeGULL Multilingual dataset contain a total of 25,861 stereotypes out of which a total of 2370 stereotypes (949 unique stereotypes) were overlapping with SGE. Thus, about 5% of unique stereotypes in SeeGULL Multilingual overlap with SGE. The Table 10 shows the overlap of SGE with SeeGULL Multilingual corresponding to each of the 23 language and country combinations.

heading: A.6 Stereotypes in a Language across Countries
text: A few languages are spoken across different countries in the world. These countries, that may share the same language, due to different socio-cultural backgrounds, can have a different notions of what is considered a stereotype. Table 11 quantitatively demonstrates how much annotations vary across countries

heading: A.7 Foundation Model Evaluations

heading: A.7.1 Creating the Evaluation set
text: To create the evaluation set, we create a balanced sample across country, language, and regional or international demonyms. Within each bucket, we take all attributes (e.g., orderly) where we could also create three distractor demonyms that do not also share an association with that same attribute. From there, we first sample attributes, then sample from potential distractor demonyms for that attribute. We randomize the demonyms to form a question item. To encode each question item into a prompt, we first substitute the attribute (in the target language) into the English instruction prefix. Then, we separately translate the prefix into the target language, as well as a suffix instruction. Finally, we take those translations and merge them with the SeeGULL Multilingual demonyms (which are already in the target language) into the prompt for the evaluation set. We create parallel English-language prompts using the same sample of question items. To encode questions into English prompts, we use the same instructions and process but without translation, using the English demonyms and attributes from the SeeGULL Multilingual dataset.

heading: A.7.2 Multilingual capabilities of Models
text: Foundation models have varying multilingual capabilities across languages. For example, the underlying PaLM 2 language model was trained on hundreds of languages (Anil et al., 2023) and Gemini was trained to support a range of multilingual capabilities (Gemini Team Google, 2023). Mixtral supports English, French, German, Italian, and Spanish (Mistral AI, 2024), while GPT systems are primarily built using English data only (OpenAI et al., 2023). We evaluate all foundation models on all languages included in SeeGULL Multilingual.

heading: A.7.3 Evaluation protocol
text: In order to demonstrate that SeeGULL Multilingual can be used for improving foundation models, we run inference without additional safety guardrails or mitigation layers that are typically used by downstream application developers. Mistral (Mistral AI, 2024) and Gemini (Google, 2024a) provide configurable safety guardrails which we disable, and PaLM 2 includes metadata about safety with responses (Google, 2024b) which we do not consider. GPT models do not support configurable safety through the API.
We run inference for evaluations through public APIs for four families of foundation models. We draw one sample from each model with
temperature=0. All system versions were fixed, and inference was run during January and February 2024. Each system was queried with temperature=0.0. Model version are show in Table 12.
Model response styles varied by foundation model, even with unambiguous and consistent instructions. To score responses, we use a heuristic to parse decoded text, and considered the model to endorse the stereotype if it produced text a) used the format as instructed and produced the letter of the stereotypical association, b) instead generated the exact word of the stereotypical association, c) produced text containing only the letter of the stereotypical association formatted as instructed, but with other additional text, and d) all formatted letter choices, repeating one letter choice twice.

heading: A.8 Regional Demonyms
text: There is no single place containing regional demonyms for all the countries of the world. We source the regional demonyms online from the following sources followed by manual validation.
France: • https://en.wikipedia.org/wiki/Regions_of_
France • https://en.wiktionary.org/wiki/Category:
fr:Demonyms • https://en.wiktionary.org/wiki/Appendix:
French_demonyms Germany: • https://en.wikipedia.org/wiki/List_
of_adjectival_and_demonymic_forms_of_ place_names#Federated_states_and_other_ territories_of_Germany
Japan: • https://en.wikipedia.org/wiki/List_of_
regions_of_Japan • Since no particular demonym are found, we default to
”People from [name of the region]”. South Korea: • https://en.wikipedia.org/wiki/Provinces_
of_South_Korea • Since no particular demonym are found, we default to
”People from [name of the region]”. Bangladesh: • https://en.wikipedia.org/wiki/List_of_
adjectival_and_demonymic_forms_of_place_ names#Bangladeshi_divisions Turkey: • https://en.wikipedia.org/wiki/Provinces_
of_Turkey • https://en.wiktionary.org/wiki/Category:
tr:Demonyms Portugal: • https://pt.wikipedia.org/wiki/Lista_de_
gent%C3%ADlicos_de_Portugal • http://www.portaldalinguaportuguesa.org/
index.php?action=gentilicos Brazil:
• https://en.wikipedia.org/wiki/List_of_ adjectival_and_demonymic_forms_of_place_ names#Indian_states_and_territories
Kenya: • No particular demonym, defaulted to ”People from
[name of the region]”. Netherlands: • https://nl.wiktionary.org/w/index.
php?title=Categorie:Demoniem_in_het_ Nederlands&from=F
A.9 Licenses of models and data used The data (SGE) was released with CC-BY-4.0 licence 8 which permits its usage for research purposes. The intended usage guidelines of the different models were adhered to 9 10 11. We abide by the terms of use of any models used in this paper.
8https://github.com/google-research-datasets/seegull/ tree/main?tab=CC-BY-4.0-1-ov-filereadme
9https://mistral.ai/terms-of-service/ 10https://ai.google.dev/terms 11https://openai.com/policies/business-terms

heading: Future Work
text: We encourage future work to explore other methods to utilize SGM to measure expressions of representational harms and stereotypes within application-specific contexts for global users