Having Beer after Prayer? Measuring Cultural Bias in Large Language Models (2024)

Abstract: As the reach of large language models (LMs) expands globally, their ability to cater to diverse cultural contexts becomes crucial. Despite advancements in multilingual capabilities, models are not designed with appropriate cultural nuances. In this paper, we show that multilingual and Arabic monolingual LMs exhibit bias towards entities associated with Western culture. We introduce CAMeL, a novel resource of 628 naturally-occurring prompts and 20,368 entities spanning eight types that contrast Arab and Western cultures. CAMeL provides a foundation for measuring cultural biases in LMs through both extrinsic and intrinsic evaluations. Using CAMeL, we examine the cross-cultural performance in Arabic of 16 different LMs on tasks such as story generation, NER, and sentiment analysis, where we find concerning cases of stereotyping and cultural unfairness. We further test their text-infilling performance, revealing the incapability of appropriate adaptation to Arab cultural contexts. Finally, we analyze 6 Arabic pre-training corpora and find that commonly used sources such as Wikipedia may not be best suited to build culturally aware LMs, if used as they are without adjustment. We will make CAMeL publicly available at: https://github.com/tareknaous/camel

## introduction we live in a multicultural world, where the diversity of cultures enriches our global community. in light of the global deployment of lms, it is crucial to ensure these models grasp the cultural distinctions of diverse communities. despite progress to bridge the language barrier gap @xcite , lms still struggle at capturing cultural nuances and adapting to specific cultural contexts @xcite . truly multicultural lms should not only communicate across languages but do so with an awareness of cultural sensitivities, fostering a deeper global connection. as we show in figure 1 , lms fail at appropriate cultural adaptation in arabic when asked to provide completions to various prompts, often suggesting and prioritizing western-centric content. for example, lms refer to alcoholic beverages even when the prompt in arabic explicitly mentions islamic prayer. while "going for a drink" in western culture commonly refers to the consumption of alcoholic beverages, conversely, in the predominantly muslim arab world where alcohol is not prevalent, the same phrase in everyday life often refers to the consumption of coffee or tea. western-centric entities are also generated by lms when suggesting people's names and food dishes, despite being inappropriate to the cultural context of the prompts. such observations raise concerns, as users may find it upsetting to see inadequate cultural representa-tion by lms in their own languages. this leads to the question: do lms exhibit bias towards western entities in non-english, non-western languages ? while considerable effort has gone into exploring biases in lms with regards to groups of different demographic or social dimensions @xcite such as religion @xcite , race @xcite , and nationalities @xcite , much less work ( §2) has examined the cultural appropriateness of lms in the non-western and non-english environments. in order to address this gap, we center our study on culturally relevant entities, as they are important aspects of cultural heritage @xcite and can symbolize regional identities @xcite . to the best of our knowledge, there is no resource readily available for doing so, especially one that can contrast arab vs. western cultural differences. we thus construct a new benchmark, camel (cultural appropriateness measure set for lms), which consists of an extensive list of 20,368 arab and western entities extracted from wikidata and commoncrawl, covering eight entity types (i.e., person names, food dishes, beverages, clothing items, locations, authors, religious places of worship, and sports clubs), and an associated set of 628 naturally occurring prompts as contexts for those entities ( §3). we show that camel entities and prompts enable cross-cultural testing of lms in versatile experimental setups, including story generation, ner, sentiment analysis, and text infilling ( §4). we benchmark 16 lms pre-trained with arabic data ( §4.1). our results reveal concerning cases of cultural stereotypes in lm-generated stories, such as the association of arab names with poverty/traditionalism ( §4.2), and cultural unfairness, such as better ner tagging performance of western entities and higher association of arab entities with negative sentiment ( §4.3). we further show that lms exhibit high levels of preference towards western-associated entities even when prompted by contexts uniquely suited for arab culture-associated entities ( §4.4). lastly, we discuss that the prevalence of western content in arabic corpora may be a key contributor to the observed biases in lms. we analyze the cultural relevance of 6 arabic pre-training corpora by training n-gram lms on each corpus and comparing their text-infilling performance on camel. we find that sources such as wikipedia may not be ideal for building culturally-aware lms ( §5). ## related work there have been several recent efforts on examining the cultural alignment of lms. one line of work explored the moral knowledge (e.g., judgment of right and wrong actions) encoded in lms @xcite @xcite , probing their ability to infer moral variation on topics with cultural divergence of opinions @xcite . it has been found that lms can be biased towards the moral values of certain societies (e.g., american @xcite ) and political ideologies (e.g., liberalism @xcite ). similar works studied lms' understanding of cross-cultural differences in values and beliefs (e.g., attitude towards individualism) @xcite , and what opinions they hold on political @xcite or other global topics @xcite . these past studies have quantified the alignment of lms through their responses to cultural surveys @xcite @xcite , where lms were probed using survey type of questions in a qa setting (e.g., 'is sex before marriage acceptable in china?'), or cloze-style questions reformulated from these surveys (e.g., 'in china, sex before marriage is [acceptable/unacceptable]'). @xcite and @xcite have shown that lms reflect values and opinions aligned with western culture when probed with such surveys, which persists across multiple languages. another line of work explored how well lms store culture-related commonsense knowledge by probing for their ability to answer geo-diverse facts (e.g., 'the color of the bridal dress in china is [red/white]') @xcite @xcite . other studies probe lms for cultural norms such as culinary customs @xcite and time expressions @xcite . @xcite studied social norm reasoning as an entailment classification task. different from existing work, we study how lms behave with entities that exhibit cultural variation (e.g., people names, food dishes, etc.). we extract and annotate an extensive list of cultural entities from wikidata and commoncrawl, which in turn enables the evaluation of lms using naturallyoccurring prompts that we collect from social media, instead of the artificial prompts used in surveybased studies. our dataset provides a foundation ## construction of camel we describe the construction process of camel, starting by collecting entities that exhibit cultural variation. we then obtain prompts from twitter/x data as natural contexts for these entities, which enable various testing setups for measuring cultural biases in lms (see examples in figure 2 ). ## measuring cultural bias in lms using camel, we measure cultural biases of several monolingual and multilingual lms ( §4.1). first, we analyze stereotypes in lm-generated stories ( §4.2). we then examine cross-cultural fairness of lms on the ner and sentiment analysis tasks ( §4.3). finally, we benchmark the capability of lms on culturally appropriate text-infilling ( §4.4). ## conclusion we introduced camel, a novel dataset of naturally occurring prompts and culturally-relevant entities as prompt completions across eight entity types. we showed that when operating in arabic, lms exhibit bias towards western entities, failing in appropriate cultural adaptation. lms also show cultural unfairness on tasks such as ner and sentiment analysis, and stereotypes in generated stories. by releasing camel, we hope to enable the evaluation and development of culturally-aware lms.