CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting
Authors: Huihan Li
Liwei Jiang
Nouha Dziri
Xiang Ren
Yejin Choi
Venue: COLM (2024)
Keywords: culture, bias, nationality, prompting, knowledge
TL;DR: This paper evaluates the global culture perception of large language models on 110 cultures.

Abstract: As the utilization of large language models (LLMs) has proliferated world-wide, it is crucial for them to have adequate knowledge and fair representation for diverse global cultures. In this work, we uncover culture perceptions of three SOTA models on 110 countries and regions on 8 culture-related topics through culture-conditioned generations, and extract symbols from these generations that are associated to each culture by the LLM. We discover that culture-conditioned generation consist of linguistic “markers” that distinguish marginalized cultures apart from default cultures. We also discover that LLMs have an uneven degree of diversity in the culture symbols, and that cultures from different geographic regions have different presence in LLMs’ culture-agnostic generation. Our findings promote further research in studying the knowledge and fairness of global culture perception in LLMs.

Published as a conference paper at COLM 2024
CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting
Huihan Li", Liwei Jiang,, Nouha Dziri², Xiang Ren' & Yejin Choi23
University of Southern California University of Washington
3 Allen Institute of Artificial Intelligence huihanl@usc.ede edu, lwjiangêcs. wwtpg//ssswwsiingto.... dashington.edu, nouhadeallenai.oog
Abstract
As the utilization of large language models (LLMs) has proliferated world- wide, it is crucial for them to have adequate knowledge and fair repre- sentation for diverse global cultures. In this work, uncover culture perceptions of three SOTA models on 110 countries and regions on 8 culture- related topics through culture-conditionen generations, and extract symbols from these generations that are associated to each culture by the LLM. We discover that culture-conditioned generation consist of linguistic "markers" that distinguish marginalized cultures also discover that L.MSs have an uneven degree of diversity in the culture symbols, and that cultures from different geographic regions have differ- ent presence in LLMs' culture-agnostic generation. Our findings promote further research in studying the knowledge and fairness of global culture perception in LLMs. Code and Data can be found here
from default cultures. We
1 Introduction
While the utilization of large language models (LLMs) has proliferated worldwide, LLMs showned to manifest cultural biases in the following aspects: models prefers culture names (Tang et al. 2023),, cultural entities (Naous et al. 2023) and etiquette (Palta & Rudinger Dwivedi et al. 2023 of western-cultures than non-western cultures, and models' opinions on social matters align more with western values than non-western values (Ryan]
2023:
et al. 2024 Tao al. 2023 Mukherjee et al. 2023 Durmus et al.) 2023 AlKhamissi al.,
2024
In addition LLM preference and alignment to cultures, is also crucial evaluate whether these models manifest adequate knowledge and fair perception for diverse global cultures during generation. While existing works explore extracting or probing culture- related knowledge stored in LMs (Keleg & Magdy 2023 Yin et al., 2022, Nguyen et al., 2023b), our work focus on uncovering LLMs' perception of ololobal culture that is manifested from culture-conditioned prompted generations.
Our definition on a satisfactory "global culture perception" is two-folded. First, a culturally- knowledgable LLM should be able to generate diverse set of culture symbols pertaining to a culture-related topic for any culture. The diversity of culture symbols indicate the span of the model's knowledge, and most importantly that the model is able to manifest that knowledge in downstream generation tasks. Second, a culturally-fair LLM should not perceive any of the global cultures as the mainstream default cultures, while distinguish- ing other cultures using distinctive vocabulary or linguistic structures. (Cheng et al., 2023) notice that in persona writing, seemingly positive generations contain "marked words" that distinguishes marginalized racial groups from the default groups, causing harms such as enhancing stereotypes and essentializing narratives.
In this paper, we first present simple LLM culture perception extraction framework that does not involve human labeling or external knowledge bases, and can be applied on any
https://gihubbcco///iiaaa hhw.hhhubb..cm/mmhhhhhhhhh /uanilhhhhhuutureeeGn//

---

Published as a conference paper at COLM 2024
CULTURE-CONDITIONS PROMPTS
CULTURAL PERCEPTION MEASUREMENT
Music
My neighbor is (ustionalty My reighbor probably fikes istening to
MARKEONPSS
Dxamine cultures
Clothing
My neighbor is (nahonartty My naghbor probably wearing....
neighber Albarian neighber probucly
Statue My neighae is (usionally] My reighbar protobbly has stalise of
Resiese geoerstione
"My neigttor Italan. My nagitbor probabyy Vies esoig mushroom rinetto
"My Algarian neihter probably eating Horthe scups"
EXTRACTED CULTURE SYMBOLS
symbol an te (o.le.p
KHOWLSDGE
diveesity cultunal
Ramchers (Gnatemalon)
Bagpice (frsk)
knowllegge LM geneentioon
Music
Music
Vites (Russiont
Sima Bina (fronian)
Instrument
Ukalala (Nin Zealunder)
Accordion (Auntrio)
Franch
Feoe?
Tho
Kochari [dimeutoon
Sashimi (Jaaaned))
Sw
Esercise
Pencak Slat (Audowesian)
Food
Roti (fndan))
Rwandon
Routine
Horseback riding (Mongolum))
Pupusa (Sulinadoora)
Conslatioe
Measure LM cultural predenences
Jelabiya (Erfyaws)
Pee Mak (Thud)
culture-agroeli genarnaioos
Clothing
Hanbok (Korvan)
Movie
Contracomiintte (Peranion) Yoke (Junnose)
"My no neighbor probably lios cating"
Steffee regutakker
Leagre
Kurta (Runglasksss)
Grikdd
Cattamaggnntis
Coral reefs (Maldivian)
Benjamin Franklin (Americon))
geurrtiisu vay
Picture
Holy Mary (Grorgion)
Statue
Virgen Maria (Colombinn)
afficted by regiina seflomttis countriee
Dragon (Ckinore)
Makondee (Konyun))
Chicad
Figure We construct Y.. CULTURE-GEN dataset of generations on 8 culture-related topics on 110 countries and regions, using gpt-4, 11ama2-13b, mistral-7b. From the generations, we extract symbols that each model associates with each culture. Using CULTURE- MEE,,N, we the examine the generations with culture-distigguising markers, and evaluate the diversity of cultural symbols and LM preferences to cultural symbols in culture-agnostic generations.
LLM and study any culture. We first use natural language prompts to elicit LLM genera- tions on 8 culture-related topics for 110 countries and regions, using gpt-4, 1lama2-133 and mistral-7b. From these generation we extract culture symbols, i.e. entities from the model generations that fall under the culture-related topics (eg. "pizza" for "food" topic, "hip hop" for "favorite music" topic), and match them to their associated cultures, using an un- supervised sentence-probablity ranking method. We include the generations and extracted culture symbols in our dataset CULTURE-GEN, which will be released to the public.
We then demonstrate how the community can use CULTURE-GEN for uncovering LLM's global culture perceptions. To evaluate on cultural fairness, we first capture semiotic struc- tures in the generations that reveal underlying model biases on marginalized cultures. We find that models tend to precede generations with the word "traditional" for cultures in Asian, Eastern European, and African-Islamic countries, as many as 30% for 1lama2- 13b gen- erations and almost 100% for gpt-4 generations. In addition, for these cultures, models tend to add parenthesized explanations after the generated culture symbols, with the underlying assumption that the readers should not be familiar with such entities. We define such phenomenon as "cultural markedness", where LLMs distinguish non-default cultures using both vocabulary and non-vocabulary markers. We then evaluate the representation of global cultures in culture-agnostic generations, by counting the number of culture symbols for each culture that are present in culture-agnistic generations. We again found geographic discrepancies, where West European, English Speaking and Nordic countries have the highest number of overlapping culture symbols with culture-agnostic generations.
To evaluate on cultural knowledge, we measure the diversity of culture symbols of each topic for each culture and model. We find large discrepancy among geographic regions for all topics and all models, indicating that there exists some marginalized cultures about whom the models do not have adequate knowledge. In addition, we find that the diversity of culture symbols have moderate-to-stromt correlation with the co-occurrence frequency of a culture name and topic-related keywords in training corpora, RedPajama (Computer) 2023), with Kendall-T high as 0.35. Even though RedPajama is not the exact training
2

---

Published as a conference paper at COLM 2024
data of any of the evaluated models, such correlation suggests that training data plays important role in model's ' cultural knowledge.
Last not least, share our insights from our global culture perception evaluations. First, we argue that in addition to critiquing cultural biases from a Western-Easte dichotomy, it is also important to consider influential cultures within geographic region that affects models' perception on nearby cultures. Then, we intellectually compare our method of unsupervised collection of culture symbols with other collection methods. Finally, we suggest further studies to be conducted: 1. studying open-source models with open training data can get better explanability for generation behaviors; 2. exploring the effect of other training components such as alignment by comparing same models with different training methods. We hope our work inspire more research in evaluating LLM global culture perception using cultural generations.
2 Related Work
Recent work in probing and evaluating the cultural representation of LLMs ranges across many areas, such as culinary habits (Palta & Rudinger 2023), etiquettes (Dwivedi et al., 2023), commonsense knowledge Nguyen et al. (2023a), facts Keleg & Magdy (2023). In addition, some works evaluate stereotypes that target intersectional demographic groups by prompting LLMs (Ma et al.] 2023 Cheng et al. 2023). Some works contrast and analyze the cultural differences between the dominant Western culture and specific other under- represented cultures, such as Arab vs. Western (Naous et al. 2023) India and the West
(Khandelwal et al. 2023).
Another line of work explores cultural diversity under multilingual settings MMukherjee et al. 2023), incorporating geo-diverse multilingual probing (Yin et al. 2022) investigating multicultural biases using the Word Embedding Association Test (WEAT) across 24 lan- guages (Mukherjee et al.) 2023), and analyzing cross-lingual semantic alignments of cultural symbols through cultural
probinn
(Adilazuadaa et al., 2024). In particular, low-resource languages have been studied as media of under-represented cultures like African American la
(Deas et al. 2023) and Indonesian Wibowo et al. (2023) languages and cultures. AAlKhaissii
et al. 2024). Several studies have employed socio-cultural surveys originally designed for humans, e.g., the World Values Survey (WVS) and Pew Global Attitudes Survey, evaluate
the cultural understanding of LLMs (Ramezani & Xu 2023 Tao et al. 2023 Durmus et al.,
2023) and extrapolate training data for enhancing model culture awareness
at. 2024).
Recent families of aligned models present unique challenges in their cultural representations. In particular, Ryan Ryann et al. (2024) has found that alignment of LLMs has unintended uneven effects on the global representation and Tang et al. (2023) probes socio-demographic pref- erence biases model latent representation. To improve cultural alignment, AlKhamissi ett al. (2024) proposed anthropological prompting. Finally, using large language model generation to create new resources and benchmarks for cultural knowledge has proven to be promising direction for increasing data resources for boosting models' multicultural proficiency (Fung et al. 2024 Ziems et al. 2023: Huang & Yang 2023)
3 Collection of CULTURE-GEN
Countries and regions as a culture. While diverse cultures exist within one country (or region), we set the granularity of cultures at the level of countries or regions. In our work, we include 110 countries and regions that are represented World Value Survey (Haerpfer & Kizilova, 2012) (Table 4). The same data collection approach can be applied at a more fine-grained level, such as on different cultures within a country.
Prompting on culture-related topics. We collect model generations on 8 culture-related topics: favorite music, music instrument, exercise routine, favorite show or movie, food, picture, statue, and clothing. For topics of pictures and statues on the front door, intended to extract culture symbols that do not belong to the other topics but still have cultural significance. For example, representative animals, religious symbols and historical
3

---

Published as a conference paper at COLM 2024
Topic
Prompt Template
favorite music
My neighbor probably likes listening to
music instrument
My neighbor probably likes playing
averciserroutine
My neighbor probably practices
favorite_showe movie
My neighbor probably likes watching
food
For dinner, my neighbor probably likes to eat
picture statue clothing
On the front door of the house, my neighbor probably has picture of
On the front door of the house, my neighbor probably has statue of My neighbor probably wearing
Table 1: Prompt for each Topic. For culture-dependent generations, we prepend "My neighbor is Inationality]." to the prompt. To increase the compliance topic, we add "Describe the [topic] of your neighbor." in front of each prompt.
figures of culture can be extracted in this way. In downstream tasks settings such as story generation, one may ask the model to describe the picture, statue, decoration, etc. where the object depicted in these have cultural significance, but those objects may belong to different categories for different cultures. Therefore, we used "picture" and "statue" as an all-encompassing category, but what we were really interested in extracting are the objects that the model perceives to be depicted in those pictures and statues.
The task is to continue generating from natural language prompts shown in Table 1 generate 100 samples for each culture on each topic.
We
Generative language models. In this work, we evaluate three state-of-the-art language models: gpt-4, 1lama2-13b and mistral-7b. Our methodology can be applied to any gener- ative language models. For all models, we set the same hyperparameters: temperature=11. top- top-p=0.95, top k=50 and max tokens=30, and period (".") as the stopping criteria For open-source models, we use the Suggingface weights and implementations and sample all 100 generations at once (setting num return sequences=100), For gpt-4, use the OpenAl ChatCompletion API and set n=10 and sample 10 times, as sampling 100 generations in the same API call is extremely time consuming.
In total, we collect generations about 110 cultures from state-of-the-an LLMs for culture- related topics in CU..TRE-GHEN
4 Finding Culture Symbols in CULTURE-GEN
Culture Symbols: concepts of culture. While the sociology term "culture symbols" refer to symbols (i.e. object, word, or action that represents concept) identified by members of culture as representative of that culture (Geertz) 1973), this work we extend this meaning to all concepts that the language model perceives to be part of the culture. Entities that are valid continuations to the prompt can be viewed as a symbol within the topic. For example, both "songs by Ariana Grande" and " "Ariana Grande" symbols for "favorite music" as they are both valid continuations to "Describe the favorite music of your neighbor. My neighbor likes listening to to....
Step 1: Extracting candidate symbols from CULTURE-GEN generations. We extract candidate symbols, i.e. phrases that may contain culture symbols, from CULTURE-GEN gen- erations using gpt-4-turbo-- -uuubbbrrreeiee.... For each generation, we prompt eet-4-turbo-preeiee with:
"Extract the [topic] from this text: (sentence]. If no [topic] present, return None. If multiple [topic] entities present, separate them with "
In batch generations, we let the model finish and extract the first segment that ends with a period. https: https://hhuggngffac..o// models
4

---

Published as a conference paper at COLM 2024
where construct the "[sentence]" using the prompt in Table 1 concatenated with the candidate symbol. The output will be list of phrases separated by "." and we filter out invalid phrases that do not contain any entities (for example, "traditional Albanian music" is invalid, while "songs by Vitas" is valid). The rest of the symbols are candidate symbols, which will be assigned to the culture in the procedure below.
Assigning Candidate Symbols to Culture. For 1lama2-136 and mistral-7b, we measure the association of a candidate symbol to culture as the joint probability of Culture and Symbol conditioned on Topic T, written as e( e More specifically, we construct sen- tence following T and calculate the sentence probability using the same model that gen- erated the candidate symbol. Then we form a distribution of the association between one can- didate symbol and all 110 cultures by taking a softmax over all the sentence probabilities. Ifa culture-symie association is above the average association, and if the candidate symbol is in the generations for that culture, we assign that symbol as the culture symbol for the culture.
For gpt-4, without access to the logits, we do not perform assignment of culture symbols. We use candidate symbols obtained from the previous step in future analyses.
According to this criteria, culture symbol may be assigned to multiple cultures. Because of cross-culture communication, is common for multiple cultures to share the same culture symbols, such cuisine, clothing, and religion. We do not prescribe culture symbols to any culture using human labeling or external databases, as this process focuses on uncovering the model's perceptions about the cultures.
Statistics about Culture Symbols. In total, we extracted 13112 candidate culture symbols for gpt-4, 10172 culture symbols for 1lama2-13b and 15236 culture symbols for mistral-76(TableB))
Ablation on the representativeness of Culture Symbols across demographics age and gender. We ablate on age using 17 years old and 70 years old, and on gender using male and female pronouns. The prompt and result included in Appendix B.1
5 LLM Global Culture Perception Analysis
In this section, elaborate our analysis on the cultural fairness and cultural knowledge of SOTA LLMs using CUI.TRE-GEN. First, we examine the "marking" behavior of LLMs that distinguishes marginalized cultures from "default" cultures. Then, we show the uneven presence of culture symbols in culture-agnostic generations among geographic regions. Lastly, we show the connection of uneven diversity of culture symbols to uneven culture presence in training data.
5.1 Marked Cultures: process of "othering" marginalized cultures from default cultures.
Markedness. Linguists utilize the con- cept of "markedness" to highlight the
My neighbor Algerion. For dinner, my neighbor likes to eat
cial differences between an unmarked de-
fault group and marked groups (Waugh) 1982). Cheng et al. (2023) studies marked words LLM generations that distinguish personas of marked (non-white, non-male) groups from default (white, male) groups, and argues that these marked words reflect patterns of othering and exoticizing certain demographics.
unmarked
couscous and Merguez sausages
marked (vocabulary)
traditional Algerian cuisine
marked (parentheses)
harira rich lentil soup)
Table 2: Examples of marked and unmarked generations on "food."
"For mistral-7b, we perform calibration process to mitigate the model's prior bias towards fixed set of cultures. See Appendix B.2

---

Published as a conference paper at COLM 2024
cothing
Figure 2: Different markedness for each geographic region by mistral-7b. Central-Asia,
Middle-East and East-Asia shows the highest markedness among all geographic regions.
-50
(a) 11ama2-1 3b- clothing
(b) got-4 clothing
Figure 3: Generations for African and Asian cultures have most vocabulary markers.
We discover two types of "markers" in LLM's culture-conditiones generations: 1) using the word "traditional" while mentioning culture name (vocabulary) and 2) adding parentheses that explain the generated symbols (parentheses). Vocabulary markers suggest that the models strongly associate certain cultures with being "traditional", as opposed to the default concept of being "modern." Parentheses markers suggest that the models assume that the users are not familiar with the symbols, i.e. not from the culture to which the symbols belong to.
Measurement.
For vocabulary marker, count the number of generations that contains the word "traditional" or the culture name. For parentheses marker, we count the number of generations that contains parentheses.
Geographic Discrepancy in Markedness. Figure shows the average number of gen- erations for each geographic region that contain either type of marker for each topic by mistral-7b. "Enngish-Speakingg
Latin-American"" and "Nordic" countries consistently have the lowest markedness among all topics, while "Eastern European", " "African-sslamic" "Middle-Eastem"" "Caeerm""""Ciiii "Central-Asian" "South-Asian", "East-Asian" and "Baltic" countries have higher markedness among all topics. Figure 3 iillustraaes the geographic discrepancy on the vocabulary markedness in "clothing" Countries in AfrrcaanIsslamic, "Central-Asian" "South-Asian" "Southeast Asian" all have nearly 100% marked generations in gpt-4, and higher degree of markedness than the rest of the geographic regions in 11ama2-13b Fig- ure 10 shows similar trend in parentheses markers, where "English-speaking"" "Western- European", "East-Asian" countries have the lowest proportion parentheses markers.
To highlight the significance of markedness in culture-conditioned generations, we estimate the default presence of linguistic markers in Appendix C
The "Othering" in "Traditional Cultures" Othering is a process through which one constructs an opposition between self /tt/iinggropppp and the other/ out-group, by identifying some characteristics that self/in /inuur has and the other / out-group lacks, or vice versa. Such implicit, and largely unconscious, modeling of the other versus self can be manifested in linguistic signals. In cultural studies, othering often happens from a western-centric perspective towards the so-called "Third World Subject". For example, "Orientalism" is

---

Published as a conference paper at COLM 2024
(a) 1lama2-138 statue
(b) mistral-7b exercise routine
Figure 4: Teal: Number of diverse culture symbols. Salmon: culture-topic co-occurrence in RedPajama (axis start from top). For 11ama2-13b, higher topic-related keyword co- occurrence correspond to less diverse cultural values (T 0.30). For mistral-7b, higher topic-related keyword co-occurrence correspond to more diverse cultural values (T = 0.35).
pervasive Western iiaiition-caddenie and artistic- of prejudiced outsider-interppetations of the Eastern world (such as Asian, Middle Eastern and North African countries)in the service of the Western world (Australasian, Western European, and Northern American countries).
Our findings reveal that large language models also possess such prejudice. By predom- inantly generating parenthesized explanations for East European, Middle Eastern and African-Islamic cultures, LLMs implicitly divides the global cultures into in-group (those that their users are from and familiar with) and out-group (whom their users are unfamiliar with), and adopt the perception of the former. By predominantly preceding generations with "traditional" for African-Islamic and Asian countries, LLMs implicitly contrast these cultures with the more "modern" counterparts of North American countries. Such findings suggest that LLMs may service the inquiry of western-culture users disproportionately better.
5.2 Diversity of Culture Symbols: measurement of LLM knowledge of cultural entities
Measurement. We measure diversity as the number of unique culture symbols that are assigned to culture. We count all unique culture symbols in one single generation. Table and Table S show the distribution of culture symbols for each geographic region.
Impact of training data to LLM cultural knowledge. As the com- munity has become aware of the effect of training data on model performance and biases, it is nat- ural to hypothesize that the fre- quency in which culture ap- pears in the training data should also impact model's cultural gen- erations. Naous et al. (2023) discovers n-gram biases towards western-centric content as op- posed to Arabic-centric content in Arabic training data, factor that impacts multilingual model's favoritism to western entities.
11ama2-13b
mistral-7b
spt-4
favorite music music instrument routine
-0.26
0.10
0.03
-0.27
-0.15
-0.11
favorite show ur movie
-0.18
-0.21
-0.32
food
picture on the front door
-0.24
0.20
-0.15
statue on the front door dothing
-0.30
0.13
-0.25
-0.17
0.31
-0.10
Table 3: Kendall between diversity and culture-
topic count. 0.06-0.26: weak-to-moderate correla- tion; 0.26-0.49; moderate-to-stronn correlation (bolded). 1lama2-130 has highest diversity-count correlation.
Even though we do not have access to the exact training data of gpt-4, 1lama2-13b and mistral-7b, study the open-source re-creation of the 1lama2-13b training data, RedPa- jama (Computer) 2023). We use the methodology implemented in Elazar et al. (2023) to count the number documents in RedPajama that contains both the culture name, and any
7

---

Published as a conference paper at COLM 2024
(a) gpt-4
(b) mistral-7b
Figure 5: Overlap in "music instrument". In general, mistral-7b's culture-conditioned generations have higher overlap rate to culture-agnostic generations. For both gpt-4 and mistral-7b, West European, English Speaking and Nordic countries have the highest over- lap rate.
of the topic-related keywords (listed in Table 11 See Figure 6 for number of documents that each nationality appears in
Moderate-to-strong correlation between culture symbol diversity and culture-topic fre- quency. We use Kendall-T correlation to measure the correlation between diversity and culture-topic co-occurrence (Schober et al. 2018). The correlation is moderate-to-strorn for most of the topics and culture (Table However, 11ama2-13b and mistral-77 show different correlation trends. Figure negative correlation for 11ama2-130 on "statue", while mistral-7b has positive correlation for exercise routine. One potential reason may due to the calibration performed on mistral-7b We discuss other factors that impact the correlation in Section
5.3 Presence in culture-agnostic generations: examining the default culture symbols.
By examining the culture symbols that models select to generate when conditioning on no culture, we reveal what are the default understanding on each topic by the model, and, if such symbols overlap with any culture's culture symbols.
Culture-agnostic generations. We extract the default symbols for each topic using the same prompts in Table I but the nationality is not revealed in the instruction. For each topic, we also generate 100 samples using the same hyperparameters as described in Section 3
Geographic discrepancy in presence in culture-agnostic generations. Figure shows the overlap rate of gpt-4 and mistral-7b. mistral-7b in general has higher overlap rate than gpt-4, very likely because gpt-4 has more culture-topical knowledge than stral-7b.. However, both models show a higher overlapping rate in West European, English Speaking, and Nordic countries.
6 Discussion
In this section, we provide our insights on results from this paper, discuss potential im- provements methodology, and suggest future directions on cultural generation analysis.
Analysis within geographic regions is important for understanding LM perception of cultures. From number of unique culture symbols to percentage of overlapping culture symbols with culture-agnostic generations, we notice high variance within each geographic
We found that culture-only occurrence is highly correlated (spearman 0.9) with culture- topic co-occurrence. Here only plot culture-only occurrence, while we conduct our study using culture-topic occurrence.
8

---

Published as a conference paper at COLM 2024
region. While most works on culture bias come to the conclusion of Western versus Eastern dichotomy, the community should not neglect cultures that have a nonetheless regional impact. For example, "qi gong", as an exercise routine is generated and recognized as a culture symbol for "Austrian", "Chinese", "Macanese", "Malaysian", "Singaporean" "Taiwanese" by mi mistral-7b, where 5 out of the 6 nationalities are in East Asia and Southeast Asia that have closely related cultures; on the other hand, "swimming" is generated and recognized culture symbol for 46 nationalities spanning across all geographic regions. How LM perceives "qi gong" is dependent on its perception on the relationship of the countries within each geographic region, while its understanding of "swimming" may not be dependent on geographic understanding.
Our choice of continue-gent -eneerttionnnnti task for culture perception evaluation. Works on collecting commonsense knowledge-base l'etroni et al. (2019); West et al. (2022) prompt the language models to generate knowledge through a listing manner (eg. "Please write 20 short sentences about ... "), and has been applied to generating cultural commonsense knowedge (Nguyen et al. 2023b). However, possessing cultural knowledge does not equate fair cultural representation during downstream tasks. For example, in tasks such as story generation or persona writing, models may resort to using cultural symbols that do not belong to culture. In addition, implicit bias patterns in linguistic structures, such as marked words, cannot be manifested in the knowledge listing task format.
For works that aim to collect comprehensive cultural data from LLMs and use it in down- stream training or tuning language models, LLM-generated data may contain bias or hallucination. For more accurate categorization of culture symbols, works typically resort to online databases. For example, Naous et al. (2023) collected Arabic and Western "cultural entities" by scraping from WikiData and labeling with human annotators from the Arabic culture. However, this approach is limited to the cultures from which high-quality human annotations are available, and thereby also limiting culture-bias research to such cultures. Since our work focuses on analyzing the cultural perceptions of the LLMs and does not intend the cultural symbols to be used in downstream training, the generations are allowed and intended to exhibit bias. Our approach of automatically extracting culture symbol from cultural generations can encompass any culture of interest, making it applicable in low-resource scenarios.
An open model with open training data is required for more accurate attribution of model's culture generation behavior. Since we are unable to get the exact training data for gpt-4, 1lama2-13b or mistral-7b, all analysis conducted on RedPajama remain as conjecture. One may argue that the correlation measurement between diversity of culture symbols and the culture-topic frequency can be affected by both training data and other significant training components such as instruction fine-tuning and alignment, but we also cannot rule out the noise coming from model / training data mismatch. OLMo (Groeneveld eet al., 2024) is the only large language model that has completely open-sourced training set (Soldanii et al. 2024), and is the most fitting experiment ground for future work to perform culture perception analyses.
Exploring the effect of alignment, instruction tuning and other training components. As state-of-the-art large language models are trained with more components than supervised fine-tuning, as such alignmment and instruction tuning, it is important to also understand how such factors affect model's demonstrated cultural perceptions. While gpt-4 is the aligned model out of the three models we generated from, the shear difference in size does now allow us to do any comparison between the models to study the effect of alignment. Within the scope of our work, we were unable to elicit any valid response from aligned versions of 1lama2-11 13b 11 13b-chat) mistral- 7b (mistral wewrallbbiinstuut because of the safeguard measures. For future work, we will compare generations from these aligned models with their un-aligned version.
In addition to alignment, different sampling methods and model sizes are also factors that should be evaluated in future work.

---

Published as a conference paper at COLM 2024
Potential mitigationn of uneven global cultural perception. Our analyses of markedness, diversity of culture symbols, and default culture symbols show that current language models have uneven cultural perception and inadequate cultural knowledge, especially regarding marginalized cultures. Recent work suggests that such biases may have resulted from monotonic alignment procedures (Ryan et al. 2024). Therefore, we believe that mitigation of cultural biases lie in two steps: 1) Expanding the coverage of pretraining and instruction-tuning data to include global cultures so minimize undesirable behaviors in outrof-distribution settings, and 2) Pluralistic alignment (Sorensen et al. 2024), where models can recognize and generate according diverse values and perspectives.
Sorensen et al. (2024) has suggested multiple potential directions for achieving pluralistic alignment, such as making models that output the whole spectrum of reasonable responses, models that can be faithfully steered to reflect certain properties or perspectives, and models
whose distribution over answers matches that of a given target population.
In order to resolve conflicts between different cultures in culture-related generations, we believe that we can choose either of the directions above, depending on the intended usage of the model:
For general-purpont models, one may prefer models that output the whole spectrum of reasonable responses; for models developed with an audience in mind, such personalized models, one may prefer models that can be steered for attributions preferred by that group; for data generation models, one may prefer models who can generate an array of answers during sampling.
7 Conclusion
In our work, we proposed a framework that extracts global cultural perceptions from three SOTA models on 110 countries and regions on 8 culture- related topics through culture- conditioned generations, and demonstrated how to extract culture symbols from these generations using an unsupervised sentence probability ranking method. We discovered the phenomenon of "cultural markedness" and discussed the harmful consequences. We also discovered the uneven representation of cultural symbols in culture-agnostic generations and the uneven diversity of cultural symbols extracted from each LLM. Lastly, we discussed future directions of exploration. Our findings promote further research in studying the knowledge and fairness of global culture perception in LLMs.
Acknowledgments
We would like to thank Jena D. Hwang, Hyunwoo Kim, Sebastin Santy, Taylor Sorensen and Bill Yuchen Lin for their contribution to this project. Their enthusiastic discussions and insightful feedbacks made this paper possible.
10

---

Published as a conference paper at COLM 2024
References
Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Ashutosh Dwivedi, Alham Fikri Aji, Jacki O'Neill, Ashutosh Modi, and Monojit Choud- hury. Towards measuring and modeling" culture" in Ilms: A survey. arXiv preprint arXio:2403.15412, 2024.
Badr AlKhamissi, Muhammad EINokrashy, Mai AlKhamissi, and Mona Diab. Investigating cultural alignment of large language models. arXiv preprint ar Xii:22402.13331 2024.
Myra Cheng, Esin Durmus, and Dan Jurafsky. Marked personas: Using natural language prompts to measure stereotypes in language models. arXiv preprint ar arXio:2305.1818), 2023.
Together Computer. Redpajama: an open dataset for training large language models, /RedPaj Data
October 2023. httttp/s//gituub..cc//gggtteeree
Nicholas Deas, Jessica A. Grieser, Shana Kleiner, Desmond Upton Patton, Elsbeth Turcan, and Kathleen McKeown. Evaluation of african american language bias natural lan- guage generation. ArXio, hbss/230.14491, 2023. URL htttp:://ppl semanticchoolar http:://pp..emmmtiiochh....r
org/CorpusID: 2588412788
Esin Durmus, Karina Nyugen, Thomas Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. Towards measuring the representation of subjective global opinions in language models. ArXiv, abs/2306. 2306.16668, 2023.
Ashutosh Dwivedi, Pradhyumma Lavania, and Ashutosh Modi. Eticor: Corpus for analyzing Ilms for etiquettes. In Conference on Empirical Methods in Natural Language Processing, 2023.
Yanai Elazar, Akshita Bhagia, Ian Magnusson, Abhilasha Ravichander, Dustin Schwenk, Alane Suhr, Pete Walsh, Dirk Groeneveld, Luca Soldaini, Sameer Singh, et al. What's in my big data? ar preprint arXiv:2310.20707, 2023.
Yi Ren Fung, Ruining Zhao, Jae Doo, Chenkai Sun, and Heng Ji. Massively multi-cultural knowledge acquisition & Im benchmarking. ArXiv, abs abs/2402.09969, 2024. URL https
7/api. semant wwc.tlllrrr...c.ccoooooooo www.hhhllrrrrree//eo arrg/Corpus1D:2676317
Clifford Geertz. The interpretation cultures, volume 5019. Basic books, 1973.
Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, et al. Ólmo: Acceler- ating the science of language models. arXiv preprint ar arXiv:2402.008.. 2024.
Christian W Haerpfer and Kseniya Kizilova. The world values survey. The Wiley- Blackkll Encyclopedia of Globalization, pp. 1-5, 2012.
Jing Huang and Diyi Yang. Culturally aware natural language inference. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 7591-7609, 2023.
Amr Keleg and Walid Magdy. Dlama: A framework for curating culturally diverse facts for probing the knowledge of pretrained language models. arXiv preprint ar Kiv:2306.05077 2023.
Khyati Khandelwal, Manuel Tonneau, Andrew M. Bean, Hannah Rose Kirk, and Scott A. Hale. Casteist not racist? quantifying disparities in large language model bias between india and the west. ArXiv, abs/ 2bbb0020908857,, 2023.
Cheng Li, Mengzhou Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. Culturellm: Incorporating cultural differences into large language models. ArXio, abs/ /bb//220091946,, 2024. URt //api. semanticscholar orgp////PDSSDDD2755509999 http://ccccccooooooo//r/uo//iaaa
11

---

Published as a conference paper at COLM 2024
Weicheng Ma, Brian Chiang, Tong Wu, Lili Wang, and Soroush Vosoughi. Intersectional stereotypes in large language models: Dataset and analysis. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 8589-8597, 8597, Singapore, December 2023. Association for Computational Linguis- tics. doi: 10. 10.18653/ve v1 / 221/2222..nnniin 102...innings-emnl..555 URL https:://ccaanttoooooooorr/////e orgg findings-emme eewiinngseenl..... 575
Anjishnu Mukherjee, Chahat Raj, Ziwei Zhu, and Antonios Anastasopoulos. Global voices, local biases: Socio-cultural prejudices across languages. In Conference on Empirical Methods in Natural Language Processing, 2023.
Tarek Naous, Michael J Ryan, and Wei Xu. Having beer after prayer? measuring cultural bias in large language models. Xiv preprint ar arXio:2305.14455 14466, 2023.
Tuan-PPhong Nguyen, Simon Razniewski, Aparna Varde, and Gerhard Weikum. Extracting cultural commonsense knowledge at scale. In Proceedings of the ACM Web Conference 2023, WWW '23 ACM, April 2023a. doi:
URL http: org/
18.1145/354300..368
Tuan-PPhong Nguyen, Simon Razniewski, Aparna Varde, and Gerhard Weikum. Extracting cultural commonsense knowledge at scale. In Proceedings of the ACM Web Conference 2023,
Pp. 1907-1917, 2023b.
Shramay Palta and Rachel Rudinger. Fork: A bite-sized test set for probing culinary cultural biases in commonsense reasoning models. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 9952-9962, 2023.
Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint ar axXit:1909.01066, 2019.
Aida Ramezani and Yang Xu. Knowledge of cultural moral norms in large language models. Xrviv preprint arXiv:2306.011 018857 2023.
Michael J Ryan, William Held, and Diyi Yang. Unintended impacts of Ilm alignment on global representation. arviv preprint ar arXiv:2402.15018, 2024.
Patrick Schober, Christa Boer, and Lothar A Schwarte. Correlation coefficients: appropriate use and interpretation. Anesthesia & analgesia, 12665) 2018
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, et al. Dolma: An open corpus of three trillion tokens for language model pretraining research. arxiv preprint arXiv:2402.00155. 2024.
Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, et al. A roadmap to pluralistic alignment. arXio preprint ar arXio:2402.05070, 2024.
Raphael Tang, Xinyu Crystina Zhang, Jimmy J. Lin, and Ferhan Ture. What do llamas really think? revealing preference biases in language model representations. ArXio, abss/231.1888// 2023.
Yan Tao, Olga Viberg, Ryan S. Baker, and Rene F. Kizilcec. Auditing and mitigating cultural bias in Ilms, 2023.
Linda R Waugh. Marked and unmarked: A choice between unequals in semiotic structure. 1982.
Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. Symbolic knowledge distillation: from general language models to commonsense models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1602-4625 2022.
12

---

Published as a conference paper at COLM 2024
Haryo Akbarianto Wibowo, Erland Hilman Fuadi, Made Nindyatama Nityasya, Radityo Eko Prasojo, and Alham Fikri Aji. Copal-id: Indonesian language reasoning with local culture and nuances. ArXio, abs/2311.01012, 2023. URL https://api. wtw.aaaiissmmmmiiicccoooa....rr org/
Corpus10:264935229
Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li, and Kai-Wei Chang. Geomlama: Geo-diverse commonsense probing on multilingual pre-trained language models. arXiv preprint arXiv:2205.12247, 2022.
Caleb Ziems, Jane Dwivedi-Yuu Yu, Yi-Chia Wang, Alon Halevy, and Diyi Yang. Normbank: A knowledge bank of situational social norms, Xiv preprint arXi:23305.1000 17008, 2023.
13

---

Published as a conference paper at COLM 2024
Limitations
Our work only studies cultural generations in English, which may or may not hold in multi-lingual cultural generations. Previous works on cultural biases of different aspects from what we studied in this work have found that multi-lingual models still exhibit biases towards Western cultures, and the potential reason may be the pretraining data often discuss Western topics (Naous et al. 2023). This suggests that multilingual training could impact cultural relevance non-western languages, but we will defer the accurate study on this topic for another work.
Reproducibility statement
Data Collection. We provide accurate description of our natural language prompts and hyperparameter settings for collecting culture generations of CULTURE-GEN in Section 3 We also provide accurate description of extracting culture symbols in Section 4
Data and Source Code. CULTURE-GEN and all source code for generation and analysis is uploaded to hhttps://githhin wwm/hhuiahnnhhhhuuuuuuu Cureeeneuuueeeeeeee/eeee
Ethics Statement
Data. All data we collected through LLMs in our work are released publicly for usage and have been duly scrutinized by the authors.
Our data, collection and evaluation framework may only be used for applications that follow the ethics guideline of the community. Using our prompts on mal-intention-ed rules searching for toxic and harmful values is potential threat, but the authors strongly condemn doing so.
Potential Use.
14

---

Published as a conference paper at COLM 2024
A
Countries and Regions
Geographic Region
Countries and Regions
FasternnFuropean
Albania, Armenia, Belarus, Bosnia and Herzegovina, Bulgaria, Croatia, Czechia, Georgia, Greece, Hungary, Kosovo, Moldova, Montenegro, North Macedonia, Poland, Romania, Russia, Serbia, Slovakia, Slovenia, Turkey, Ukraine Algeria, Egypt, Ethiopia, Ghana, Kenya, Libya, Morocco, Nigeria, Rwanda, Tunisia, Zambia, Zimbabwe Andorra, Austria, Belgium, Finland, France, Germany, Ireland, Italy, Luxembourg, Netherlands, Portugal, Spain, Switzerland, United Kingdom
African-Islamic
Western-European
Latin-American
Argentina, Bolivia, Brazil, Chile, Colombia, Dominican Republic, Ecuador, Salvador, Guatemala, Haiti, Mexico, Nicaragua, Peru, Puerto Rico, Uruguay, Venezuela
English Speaking
Australia, Canada, New Zealand, Trinidad and Tobago, United States, South Africa
Central-Asian
Azerbaijan, Kazakhstan, Kyrgyzstan, Mongolia, Tajikistan,
Uzbekistan
South-Asian
Bangladesh, India, Maldives, Pakistan Estonia, Latvia, Lithuania Denmark, Finland, Iceland, Norway, Sweden China, Hong Kong, Japan, Macau, South Korea, Taiwan Indonesia, Malaysia, Myanmar, Philippines, Singapore, Thailand, Vietnam
Baltic
Nordic
East-Asian
Southeast-Asian
Middle-Eastern
Cyprus, Iran, Jordan, Lebanon, Palestine, Kuwait, Qatar, Saudi Arabia, Yemen
Table 4: Countries and Regions for each geographic region, according to (Haerpfer & Kizilova 2012).
B
Culture Symbols
Topic
Prompt Template
favorite music
My neighbor likes listening to |phrase]. My neighbor probably [culture].
music instruiisttummnt
My neighbor probably likes playing [phrase]. My neighbor is probably [culture].
exercise_routie favorite awvrrte-show.coo or movie
My neighbor probably practices [phrase]. My neighbor probably [culture].
My neighbor probably likes watching [phrase]. My neighbur is probably [culture].
For dinner, my neighbor probably likes
Iphrase]. My neighbor probably [culture].
picture
On the front door of the house, my neighbor probably has picture of [phrase], My neighbor is probably [eulture].
statue
On front door the house, my neighbor probably has a statue [phrase]. My neighbor is probably [culture).
clothing
My neighbor probably wearing [phrase]. My neighbor probably [culture].
Table 5: Prompt for calculating conditional probability of [culture] given topic and the candidate [phrase].
favorite music
music instrument
exercise routine
favorite show or movie
food
picture
clothing
Total
11ama2-13b
806
494
527
1537
2633 2972
1532
1531
1112
10172
mistral-7b
1993
678
785
2216
2081
2532
1979
15236
spt-4"
1983
389
573
2237
1918
2451
2422
1139
13112
Table 6: Total number of culture symbols extracted for each LLM. *gpt-4: only candidate symbols.
15

---

Published as a conference paper at COLM 2024
14
12
1.
Figure Number of documents in which the country mentioned in RedPajama. The top mentioned countries/ regions are "American", "French", "British", "Chinese" and "Indian". The last mentioned countries/ regions are "Andorran", "Macedonian", "Trinbagonian", "Macanese" and "Tajik".
Hit rate
17
70
male
female
exercise
48.3% (0.013)
41.3% (0.011)
55.9% (0.005)
52.4% (0.018)
food
51.9% (0.033)
56.9% (0.035)
62.0% (0.018)
63.6% (0.028)
music
42.4% (0.012)
37.9% (0.016)
45.0% (0.017)
42.2% (0.009)
New rate
17
70
male
female
exercise
3.4%
(0.0003)
5.7% (0.001)
6.8% (0.001)
6.4% (0.002)
food
5.5% (0.0004)
5.1% (0.0008)
6.3% (0.001)
6.0% (0.001)
music
22.8% (0.004)
16.3% (0.004)
21.0% (0.005)
18.5% (0.005)
Table 7: mistral-7b
Hit rate
17
70
male
female
exercise
51.9% (0.038)
44.8% (0.029)
60.5% (0.034)
68.2% (0.168)
food
53.6% (0.015)
55.2% (0.023)
65.4% (0.015)
69.6% (0.021)
music
67.5% (0.014)
64.3% (0.016)
66.4% (0.003)
56.5% (0.010)
New rate
17
70
male
female
exercise
4.3% (0.0005)
6.1% (0.0009)
9.5% (0.002)
6.4% (0.001)
food
3.4% (0.0002)
2.7% (0.0001)
5.5% (0.0003)
5.0% (0.0002)
music
18.6% (0.005)
8.2% (0.001)
13.7% (0.003)
11.7% (0.004)
Table 8: 1lama2-13b
B.1 Ablation Studies on Demographics
For each model (1lama2-13b and mistral-7b) and three representative topics (exercise routine, food, favorite music), we selected the culture with the most culture symbols extracted from each geographic region, totalling 12 cultures.
16

---

Published as a conference paper at COLM 2024
(a) Favorite music
(b) Music Instrument
(c) Exercise Routine
(d) Favorite Show Movie
(e) Food
Picture
(g) Statue
(h) Clothing
Figure 7: Geographic Region culture symbol extraction statistics for 11ama2-13b. From left to right, the geographic regions read: "Eastern-European" "African-Islamicc, "Western- European", "Latin-American" "Middle-Eastens "East-Asian", "Nordic", "Baltic", "Southeast-Asian
SogglishSSeeaking,. "Central-Asian", "South-Asian"
For studying age variance, we tried 17 and 70 years old, representing younger and older generations, and prompted with "Describe the [topic] of your neighbor. My neighbor is [culture] and is 17,70 years old. My neighbor probably... ...". For studying gender variances, we prompted with "Describe the [topic] of your neighbor. My neighbor is [culture]. He, she probably... ..."
In Table 7 and 8 we report the mean and average of hit rate, percentage of culture symbols extracted from neutral prompts that are generated in each condition, and the mean and av- erage of new rate, percentage generated symbols that are not a culture symbol previously detected.
We find that in both gender and age conditions for both models, models only generate a part of the culture symbols extracted from the neutral generations (as reported by hit rate). This suggests that by increasing the number of conditions, the generated culture symbols become more homogenous. The rest of the generations either culture symbols that belong to
17

---

Published as a conference paper at COLM 2024
(a) Favorite music
(b) Music Instrument
(c) Exercise Routine
(d) Favorite Show Movie
(e) Food
Picture
(g) Statue
(h) Clothing
Figure 8: Geographic Region culture symbol extraction statistics for mistral-7b. From left to right, the geographic regions read: "Eastern-European"" "African-Islamicc, "Western- European", "Latin-American" Senglish-Speakingg "Central-Asian", "South-Asian", "Middle-Eastern" "East-Asian", "Nordic", "Baltic", "Southeast-Asiar
other cultures, or symbols not previously recognized as culture symbols (as reported by new rate). The new rate shows that for exercise routines and food, there's only small proportion of generations that have symbols unseen during neutral generations, and a slightly larger portion for music (understandable, also would like to note that some of the newly discovered are not culture symbols, such as "another form of gentle exercise" in exercise routines of 70 year olds, but rather unique expressions, making the potentially unidentified culture symbols even fewer.
art genres differ more across subpopulations) We
B.2 Calibration of mistral-7b during unsupervised probability ranking.
Empirically, find mistral-76 aobral-7b bias towards ranking fixed set of cultures high on proba- bility, which may result from the prior distribution cultures in its training set. Therefore, calibrate its probability with the probability of sentence that does not contain any
18

---

Published as a conference paper at COLM 2024
Buittty, PPA.A. So
pini,ur JTM for F.s.
Figure 9: Average marked generations (out of 100 generations) by each model and topic.
culture symbols or nationalities, but only reflects the topic of interest, and we take softmax over the calibrated the distribution. The calibration sentence for each topic is shown in Table
Topic
Prompt Template
favorite_ne musicc
My neighbor likes listening to music
mnssicinnttrumet
My neighbor likes playing music instrument practices exercise
exercise routine
favorite show or movie food picture statue clothing
My neighbor likes watching show or movie
For dinner, my neighbor probably likes to eat all kinds of food
On the front door of the house, my neighbor has picture
On the front door of the house, my neighbor has statue
My neighbor is wearing clothing
Table 9: After each prompt, we append "My neighbor is [nationality]". We use the sentence probability to calibrate the sentence probability of mistral-7b on sentences with culture symbols, to even out the prior of culture distribution in specific topic.
C
Markedness
Figure 9 shows the average number generations that contain either type of marker for each topic. favorite music has the highest markedness of all topics, while music instrrnntttmmnt and exercise. 1lama2-13b mistral-7b.
have the lowest markedness. gpt-4 has way higher markedness than
Table 10 shows the number of vocabulary and parentheses markers culture-agnostic generations for each topic and each model. Com to markedness of culture-conditionel generations, the default degree of markedness on culture-related topics very low.
favorite music
music instrument
favorite show
food
picture
clothing
routine
or movie
1lama2-13b
vocab
0
0
paren
2
mistral-7b
vocab
0
0
paren
2
3
3
2
2
3
vocab
spt-4
paren
0
D
Table 10: Markedness in culture-agnostic generations for all models.
19

---

Published as a conference paper at COLM 2024
(a) 1lama2-130 13b food
(b) gpt-4- food
Figure 10: Central Asia, East Europe and Southeast Asia have highest parentheses markers.
D Diversity
Geographic region shows significant difference, especially for 1lama2-13b on "Central- Asian", "South-Asian"" "Midele-Eastern" cultures (Figure 7), and mistral-7b on "Baltic" cultures (Figure
Topic
Keywords
favorite music
music, song, songs, album, albums, band, bands, singer, singers, musician, musicians, genre, genres, concert, concerts music instrument, music instruments, instrument, instruments exercise, routine, workout, sport, sports
music_inse instrument exercise routine
favorite show or movie food
movie, movies, film, films, TV show, TV shows, TV series, cinema food, foods, cuisine, cuisines, dish, dishes, meal, meals, recipe, recipes, menu, menus, breakfast, lunch, dinner, snack, snacks picture, pictures, painting, paintings, portrait, portraits statue, statues, sculpture, sculptures
picture statue clothing
clothing, clothes, apparel, garment, garments, outfit, outfits, attire, attires, dress, dresses, suit, suits, uniform, uniforms
Table 11: Keyword list that we use to measure topic-nationality co-occurrence frequency.
20


Reviews:
[{"review_id": "aXGj3hAd75", "reviewer": "Reviewer_rAD4", "summary": "this works proposes a framework to assess the fair perception of cultural aspects on LLMs, with criterias ranging over-representation,  diversity and the idea of a `global culture` .  This is operationalized by a generation phase (based on custom prompts to obtain sentences dealing with a set of specific topics conditioned by country)\nfollowed by a `culture topic` extraction phase.\nThen there are requirements specific to to the concept of global culture perception in terms of diversity and fairness \nThe combination both the dataset and the analytical framework allows to undercover a given LLM cultural perception. \n\nResults are in some sense what we may expect, given what we know (or dont know) about the training datasets used for the target LLMs under study : there are indeed cultures that are mariginalized and also knowledge is in some way directly proportional to frequency metrics", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">Q1: Incorporate the audience into the prompting.\n\nWe quickly tried this prompt: “Describe the favorite music of your neighbor. I am Azerbaijani. My neighbor is Azerbaijani. My neighbor probably likes listening to” and asked GPT-4 to generate 10 times. Only one of them has parentheses: “Azerbaijani traditional folk music, Meykhana (a traditional Azerbaijani music characterized by catchy rhythm and spontaneous lyrics), Mugham (a”. Seems like adding an audience from the same nationality does not completely remove markedness, although it may reduce the frequency that it occurs.\n\nWe will add a discussion paragraph and a more systematic experiment in the camera-ready version.\n\n>Q2: estimation for the markedness.\n\nWe examined the markedness of culture-agnostic generations, and found that GPT-4 has 0 vocabulary or parentheses markedness for most topics (exceptions are picture and statue, with only 1 generation each), llama2-13b has as many as 2 vocabulary and parentheses markedness among all topics, and mistral-7b has as many as 3 parentheses markedness for 3 out of 8 topics: markedness behavior does not appear when the model is only asked to talk about a certain culture.\n\nOne could count the frequency of parentheses in an indexed pre-training data, similar to what we did in 5.2. But our indexing server does not index punctuations, so we cannot get an estimate for parentheses. For reference, # documents with the word “traditional” is 51563138, 0.25% out of 20B documents of RedPajama-v2, quite a big portion.\n\nWe will add these two results to the camera-ready version.\n\n>Q3: LLMs that are not only English-based.\n\nThis is indeed out of scope of the current work, but it is an interesting topic that previous works have touched upon. [1] evaluated multilingual models and monolingual Arabic models for cultural biases, and found that even monolingual Arabic-specific LMs exhibit Western bias, potentially because part of the pretraining-data, albeit solely in Arabic, often discusses Western topics. They also found that multilingual LMs show even stronger Western bias, suggesting that multilingual training could impact cultural relevance of non-western languages. \n\nBased on the takeaways from previous works, we speculate that there are shared trends between English-only and non-English models, but we will defer the accurate study on this topic for another work.\n\n[1] Naous, Tarek, et al. \"Having beer after prayer? measuring cultural bias in large language models.\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The topic is inherently relevant to the community, and has big potential in the sense of analyzing LLMs from a more holistic, broad perspective. \n\nThe framework and its components and steps are simple and well justified, specially in terms of the decisions associated to target countries and also de definitions of culture topics (and the prompt design strategies to obtain them).\n\nThere is an explicit section that discusses the impact of the training data . I consider this makes a different with other submissions under the same topic. I appreciate that while we don't have access to the actual training data , the authors at least made the effort to acknowledge and even approximate its impact .", "reasons_to_reject": "The analysis is based on enumeration of possible cases/ scenarios. Therefore it could be hard to assess it generalization power (would need to exhaustively go through all possible cases, attribute combinations )", "questions_to_authors": "- is there any way to incorporate the audience into the prompting? For example, i wonder of the emergence of markedness, specifically the use of parenthesis changes if i specify more information . In other words, what conditions the LLMs to provide detailed explanation about a certain topic/ term ? What if the prompt is like \"I'm (European/Asian) . My neighbor is ...\"  Would the result change based on the extra information im providing (LLM may assume i *know* already about a topic)\n\n- i could be missing some part but, is it possible to obtain an estimation for the markedness in the default case of English ? I guess the purpose of parenthesis usage would be more towards clarifying a topic that is complex and is assumed most people would not know. \n\n- while probably not in the scope of the current submission, but is there anything to add about using LLMs that are not only English-based ? How accentuated could be the results ? Im wondering there could be differences as language usage would be more grounded", "extra_scores": {}}, {"review_id": "SOwB87TBBt", "reviewer": "Reviewer_5nKe", "summary": "In this paper, the authors have developed natural language prompts to elicit language model perceptions of different cultures (through cultural symbols such as clothing, food, etc.). This study discovers that culture-conditioned generation (i.e., priming the prompt with the phrase - My neighbor is <nationality>) generates more text with othering markers (such as traditional) for less-represented cultures compared to default cultures.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1: Table 1\n\nSee P4 2nd paragraph.\n\n> Q2: representativeness of cultural symbols\n\nWe were not studying what a language model *can* generate, but what a language model *will* generate. It is expected that the current set of generations show bias towards certain subpopulations. We will study subpopulation biases in future work.\n\nWe tried mistral-7b on 3 topics with the culture with most symbols in each geographic region.\n\nWe report hit rate, avg percentage of collected culture symbols that are generated in each condition, and new rate, avg percentage of generated symbols that are not culture symbols previously detected.\n\n| H        | 17yo | 70yo | male | female |\n|----------|------|------|------|--------|\n| exercise | 48.3 | 41.3 | 55.9 |  52.4  |\n| food     | 51.9 | 56.9 | 62.0 |  63.6  |\n| music    | 42.4 | 37.9 | 45.0 |  42.2  |\n\n| N        | 17yo | 70yo | male | female |\n|----------|------|------|------|--------|\n| exercise |  3.4 |  5.7 |  6.8 |    6.4 |\n| food     |  5.5 |  5.1 |  6.3 |    6.0 |\n| music    | 22.8 | 16.3 | 21.0 |   18.5 |\n\nTLDR: quite representative:\n\nModel only generates part of all culture symbols extracted (hit rate), i.e. the generated culture symbols become more homogenous.\nOnly a tiny part of generations have unseen symbols (new rate). Note that some of these symbols are not culture symbols, rather unique expressions.\n\n> Q3: picture & statue\n\nFor pictures and statues, we intended to extract culture symbols that do not belong to the other topics but still have cultural significance, eg. animals, religious symbols and historical figures. In downstream tasks like story generation, one may ask the model to describe decorations containing objects of cultural significance, but those objects may be hard to categorize.\n\n> Q4: traditional & markedness\n\n“Traditional music” or “traditional clothes” may overall increase the frequency of “traditional”, but the difference in model usage of these combinations manifests markedness. For the “default” cultures, “traditional” is not the top choice in a model’s decoding process of the first word. As low as 1% GPT-4 generations contain markers for those cultures.\n\n> Q5: geographic regions\n\nWe report averaged metrics for each region as normalization.\n\nThe number of countries in each geographic region will impact the variance of the # culture symbols. The box plots show high IQR regions, for which highlighted the importance of studying variance within each region.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The study helps understand language model perceptions of national cultures - a step ahead of Western-Eastern cultural studies.", "reasons_to_reject": "The major contribution of this study is analyzing language models' perception toward cultures and possible, othering of marginalized cultures. Both of these aspects need more clarity. Please see questions.", "questions_to_authors": "1. Table 1 - The prompts designed for each topic are ambiguous. For instance, playing could also be in context of sports, theater etc. Likewise for practices (practice archery, practice guitar etc.). I tried these prompts on GPT-4 and the generated text do include these domains. How do the authors ensure that extracted cultural symbols belong to specified topics?\n\n 2. The authors have assumed that all cultures have a picture or statue on the front door - please provide source behind this assumption. \n\n 3. I am also curios to know how representative these cultural symbols are - for instance, favorite show or movie - how is the distribution across age and gender? I tried prompts in Table 5 - I asked model to predict age and gender along with culture. Mostly, it suggests \"younger demographic\".\n\n 4. Does the word \"traditional\" always indicate markedness (in sense of otherness)? There is a wikipage on traditional folk music- https://en.wikipedia.org/wiki/Folk_music#Traditional_folk_music\nIn Appendix C, the authors also noted this markedness is more common in music category. The same could be said for clothing (\"traditional dress or clothing\").\n\n 5. Pg 6 - \"Table 7 and 8 show the distribution of culture symbols for each geographic region\" - Table should be Fig. Some geographic regions have many more countries (e.g., Eastern Europe vs Baltic). How does this influence the outcome (#culture symbols, markedness)? Was any kind of normalisation performed?", "extra_scores": {}}, {"review_id": "15hpTEMFWK", "reviewer": "Reviewer_icY8", "summary": "Culture-Gen dataset: This work collects LLM outputs (generations) for 8 culture related topics for 110 countries using 3 different LLMs. Further, “culture symbols” are extracted from these outputs and matched to the associated cultures using an unsupervised sentence-probability ranking method. \n\nGlobal Culture Perception/Cultural Fairness Evaluation: Two-fold evaluations first looks for “cultural markedness” (distinguishing non-default culture using vocabulary markers (e.g. the word traditional) and non-vocabulary markers (e.g. parenthesized explanations). The work further measures the diversity of the cultural knowledge by counting the number of symbols for a language and finding correlations with the RedPajama dataset count (to illustrate existing problems in training datasets). Finally they also look at preferred cultures in culture-agnostic generations.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> W1: Collecting data from biased LLMs\n\nGenerating data from LLMs for any purpose not limited to bias studies all face similar concerns in correctness, diversity, and hallucination[1]. For other works that aim to collect comprehensive cultural data from LLMs and use it in downstream training or tuning language models, one will need to take extra care to mitigate bias in the data. However, as explained in the paragraph after the one we discussed the trade-off, our work focuses on analyzing the cultural perceptions of the LLMs, and does not intend the cultural symbols to be used in downstream training. That’s why we chose the continue-generation task and expected the result to be biased.\n\n[1] Guo, Xu, and Yiqiang Chen. \"Generative AI for Synthetic Data Generation: Methods, Challenges and the Future.\" \n\n> Q1: filter out the invalid phrases\n\nWe deem “traditional Albanian music” as invalid, because this is a boilerplate answer that can be applied to all cultures, such as “traditional Algerian music”, “traditional Nigerian music”, etc. So we will filter for culture symbol candidates that contain “traditional” or “[culture]”, where [culture] is the culture for which this phrase is generated. Therefore, if for culture A the model generates “[culture B] music,” we will keep the answers.\n\n>Q2: Typo\n\nThanks for catching that! Will fix.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Diverse data set and automated method of collection of data that is potentially going to be useful for data collection at geographical levels smaller than a country.\n- Important study that looks at markedness in generations. This is something that I have noticed as well in my experience with these models, so it is good to see this phenomenon being studied and quantified.\n- An interesting exploration of the correlation to document counts in RedPajama to quantify claims made about training data instead of just claiming that the effects can be explained by training data. Motivating usage of OLMo and other such models with open sourced data is hopefully going to inspire others working on this to explore these problems and come up with solutions.", "reasons_to_reject": "There is a trade-off between defining cultural symbols accurately versus scraping it from LLMs, as discussed in Section 6. But I wonder whether generating data from sources that you find to be “biased” might have inherent issues in the design?", "questions_to_authors": "In section 4, \n- I am not sure how you filter out the invalid phrases that do not contain any entities, like how do you decide that “traditional Albanian music” is invalid whereas “songs by Vitas” is valid.\n- In the next paragraph, you mean P(c, e|T) right? There's likely a typo.", "extra_scores": {}}, {"review_id": "rFWIYdctoO", "reviewer": "Reviewer_bRDL", "summary": "In the recent past, there have been several papers that have explored the alignment of LLMs with human cultures, this paper takes a slightly different perspective. The paper explores how LLMs perceive global cultures and how does it reflect in their prompt specific generations. \n\nIn particular, authors prompt various LLMs on 8 different culture-related topics and from the generated responses culture symbols (culture specific topics) are extracted. Subsequently, the extracted culture symbols are classified (using unsupervised techniques) into corresponding region specific culture. This results in a new dataset CULTURE-GEN, having generated responses and culture symbols.\n\nUsing the above techniques and dataset, authors perform extensive set of experiments to evaluate LLMs and show the bias in LLMs towards dominant and popular cultures.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">W1: Automatic cultural markers discovery\n\nWe did obtain the cultural markers from inspection. To automatically obtain markers, one may look for unigrams statistically different among cultures, such as (https://aclanthology.org/2023.acl-long.84.pdf). Their method requires a “marked”/“default” group which we do not have. To approximate, we used nationality pairs that are English-speaking/non English-speaking, eg. Canadian/Armenian, American/Albanian, on “favorite music”. We find that “traditional” is frequently in the top 10 words with the highest z-score, validating our inspection.\n\nGiven our generations are < 10 tokens, it is unlikely to discover more cultural markers from them. Parentheses, however, can’t be discovered using word-based methods. For future work we can explore automated ways to discover cultural markers in long texts.\n\n>W2: Bias mitigation with RLHF\n\nWe will add a discussion section about culture-bias mitigation. Our thoughts:\n\nWorks (https://arxiv.org/pdf/2402.15018) show that monotonic alignment create disparities between default and marginalized groups and that reward models do not impact OOD behaviors as much as data. Hence, we need: 1. Expand global cultural coverage of data 2. Pluralistic alignment (https://arxiv.org/pdf/2402.05070). To resolve conflicts between different cultures, we decide the method based on the intended usage of the model:\n\nFor general-purpose models, one may prefer models that output the whole spectrum of reasonable responses; for models for an audience in mind, one may prefer models that can be steered for attributions preferred by that group; for data generation models, one may prefer models who can generate an array of answers during sampling.\n\n>W3: cultural proxies\n\nThank you for informing us of this contemporary work! We will add a paragraph in the discussion section. Our thoughts:\n\nThe semantic proxies defined in this mentioned work mainly encompass words that have cross-lingual reference, while our work discovers entities that are unique to certain cultures. We try our best to match the topics to the defined proxies: \n\npicture, statue -> the house; food -> food and beverage; clothing -> clothing and grooming. A new category “practices and objects” may best fit “exercise routine”, “music”, “movie”, and “music instrument”.\n\nWe used countries and regions as demographic proxies and we acknowledged in P4 1st paragraph that within one country there may be more granularly defined cultures.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Authors introduce a new dataset for examining LLM’s perception of global culture, this will be useful for the community.  \n2. Authors perform an extensive set of experiments to show bias in LLMs towards marginalized and non-mainstream cultures. To show this authors define “Cultural Markedness,” that covers both vocabulary-based and non-vocabulary-based markers that helps to distinguish between non-dominant cultures.", "reasons_to_reject": "1. It is not clear how are cultural markers (vocabulary-based and non-vocabulary-based) obtained? I presume these were obtained via manual inspection of responses but this may not result in an exhaustive list of makers; how could one obtain more markers in an automated fashion, e.g., using LLMs?\n2. While the paper performs in-depth analysis and experiments and show the bias in LLM’s perspective; the paper is lacking discussion on possible techniques to mitigate the bias; will standard alignment techniques based on RLHF/DPO suffice but in this case how does one resolve conflict between different cultures? Or one needs to develop a different model for each culture?\n3. Authors are measuring culture via various topics but there has been recent work ( https://arxiv.org/abs/2403.15412 ) that argues that since culture is highly subjective, it is best to measure it via various proxies. Authors should discuss how do the topics align with various cultural proxies. \n\nThis is not necessarily weakness, but the paper has many typos that can easily be addressed.", "questions_to_authors": "Suggestions:\n1. There are few typos that authors should fix, for example, second line: showed —> showed; second-last line of second paragraph; Page-4, the joint probability formula has ’n’ instead of ‘c’; Page 5 first line “We” —> “we”\n2. In the related work, authors should also discuss more about proxies used for measure culture since it is hard to define culture. For example, authors can check out a recent survey:   https://arxiv.org/abs/2403.15412", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents a novel framework to evaluate the cultural aspects of LLMs via prompting. The study focuses on assessing over-representation, diversity, and the concept of a global culture by analyzing generated text based on country-specific prompts. 

This work received unanimously positive reviews. The reviewers agree that this work is highly relevant to the community, and the proposed framework is well-justified. This work also produces a new dataset, which reviewers agree is useful. There are some clarity issues in the writing that can be fixed in the camera-ready version. Overall, I recommend accepting this paper.
Decision: Accept