Towards Measuring and Modeling “Culture” in LLM s: A Survey (2024)

Abstract: We present a survey of more than 90 recent papers that aim to study cultural representation and inclusion in large language models (LLMs). We observe that none of the studies explicitly define “culture, which is a complex, multifaceted concept; instead, they probe the models on some specially designed datasets which represent certain aspects of “culture”. We call these aspects the proxies of culture, and organize them across two dimensions of demographic and semantic proxies. We also categorize the probing methods employed. Our analysis indicates that only certain aspects of “culture,” such as values and objectives, have been studied, leaving several other interesting and important facets, especially the multitude of semantic domains (Thompson et al., 2020) and aboutness (Hershcovich et al., 2022), unexplored. Two other crucial gaps are the lack of robustness of probing techniques and situated studies on the impact of cultural mis- and under-representation in LLM-based applications.

## introduction in order to make llms inclusive and deployable across regions and applications, it is indeed necessary for them to be able to function adequately under different "cultural" contexts. the growing body of work that broadly aims at evaluating llms for their multi-cultural awareness and biases underscore an important problem -that the existing models are strongly biased towards western, anglocentric or american cultures @xcite @xcite . such biases are arguably detrimental to the performance of the models in non-western contexts leading to disparate utility, potential for unfairness across regions. for instance, @xcite and @xcite show that a conversational system that lacks cultural awareness alienate the users, leading to mistrust and lack of rapport, and eventual abandonment of the system by users from certain cultures. there are also concerns about the impact on global cultural diversity, since if biased models reinforce dominant cultures, whether implicitly or explicitly, they might lead to a cycle of cultural homogeneity @xcite . the recent generation of llms, with their impressive ability and widespread availability, only make this issue more pressing. it is therefore a timely moment to review the literature on llms and culture. in this work, we survey more than 90 nlp papers that study cultural representation, awareness or bias in llms either explicitly @xcite @xcite or implicitly @xcite . it is quickly apparent that these papers either do not attempt to define culture or use very high-level definitions. for example, a common definition is "the way of life of a collective group of people, [that] distinguishes them from other groups with other cultures" @xcite @xcite . not only do the papers typically use broad-brush definitions, most do not engage in a critical discussion on the topic. foot_0 this is perhaps unsurprising as "culture" is a concept which evades simple definition. ## findings: defining culture in this section, we discuss how different papers have framed the problem of studying "culture." the findings are organized by the three dimensional taxonomy proposed in sec 2.2.1 and also presented graphically in fig 1 . ## findings: probing methods the most common approach to investigate cultural representation, awareness and/or bias in llms is through black-box probing approaches, where the llm is probed with input prompts with and without cultural conditions. a typical example of this style is substantiated by the following prompting strategy described in @xcite . pick one. do people in [country_name] believe that claiming government benefits to which you are not entitled is: 1. never justifiable 2. something in between 3. always justifiable the prompt has two variables, first the [country_name] which provides the cultural context, and second, the input question on "claiming government...not entitled", which is taken, in this case, from the world value survey (evs/wvs, 2022). this an example of discriminative probing approach, where the model is provided with a set of options as answers. for datasets where the answers to the input probes depend on the cultural conditioning, and are available as ground truths (e.g., wvs and eticor @xcite ), one could measure the accuracy of the model predictions under different cultural conditioning to tease out any disparity in performance. another technique involves measurement of the response without a cultural conditioning (often called the baseline predictions) and compare those with the ground-truths for different cultures. this method can reveal the bias in the default predictions of the model, but does not prove that a model is incapable of responding in a culturally-informed way for certain culture if probed properly. most papers we surveyed use some variation of this technique as any dataset based on contrastive or comparative study of culture is tenable to this treatment. note that cultural context can also be introduced indirectly by stating a norm or moral value (e.g., "family values are considered more important than professional integrity") explicitly in the prompt. @xcite uses this to show deeper biases in models, where despite the direct elucidation of cultural expectation (such as a value judgment), a model might still fail to rectify its baseline responses as required by the context. furthermore, @xcite introduces three distinct methods for presenting the cultural context: simulated conversations, which mimic real-life interactions; text formats, which involve evaluating responses to various structured text inputs; and wikipedia paragraphs, where models are tested on their understanding and interpretation of information from wikipedia articles, offering a diverse set of probing techniques to evaluate model capabilities. alternatively, generative probing assesses llms based on their free-text generation. evaluating free-text generation is not as streamlined and may require manual inspection. @xcite introduces the seegull stereotype dataset, which leverages the generative capabilities of llms to demonstrate how these models frequently reproduce stereotypes that are present in their training data as statistical associations. most evaluation techniques use a single-turn probing where the cultural context and the probe are given in one go as a single prompt @xcite . on the other hand, multi-turn probing, initially introduced by @xcite , evaluates the model's responses over several interactions, allowing for a nuanced understanding of its cultural sensitivity (also see @xcite ). a limitation of black-box probing approaches is model sensitivity to prompts @xcite such as the exact wording and format that are irrelevant to the cultural context. this raises questions regarding the reliability and generalizability of the results because one cannot be sure if the observed responses are an artifact of the cultural conditioning or other unrelated factors. while black-box approaches have been predominant in investigating cultural representation in llms, white-box probing methods offer a more interpretable alternative by examining internal model workings to uncover how biases are encoded. techniques like gradient-based analysis @xcite , attention mechanism analysis @xcite , embedding space evaluation @xcite , and layer-wise analysis @xcite have been primarily applied to bias mitigation-particularly addressing issues like gender and racial biases-within model parameters. however, these studies are currently limited in scope regarding cultural representation; they have not yet been extensively utilized to explore how cultural biases and representations are encoded in llms. for example, partitioned contrastive gradient unlearning (pcgu) optimizes weights most responsible for specific biases by analyzing gradients from culturally contrasting sentence pairs, extending beyond gender bias to directly address cultural biases. attention analysis helps reveal potential processing biases by showing how models focus on culturally significant tokens, uncovering how cultural information is prioritized in the model's computations. evaluating embedding spaces can identify and adjust biased word representations associated with different cultures, using methods like hard or soft debiasing to neutralize cultural biases. layer-wise analysis pinpoints where cultural biases are encoded by observing changes in outputs when modifying different model layers. moreover, the survey by @xcite provides an overview of bias evaluation and mitigation techniques in llms, emphasizing the importance of white-box methods for a more transparent understanding of model behaviors, including cultural aspects. they categorize methods into pre-processing, in-training, and post-processing interventions, highlighting how white-box approaches can be applied at different stages of model development to detect and mitigate cultural biases. ## gaps and recommendations our review has found three gaps in the portfolio of studies of cultural inclusion in llms; first, a heavy focus on values and norms, leaving many aspects of cultural difference understudied; second, space to expand the methodological approach; and third, the lack of situatedness of the studies, making it difficult to know the practical significance of the biases revealed by the studies in real-life applications. we elaborate on these gaps and provide several recommendations. ## conclusion in this survey, we explored how language and culture are connected and stressed the importance of llms' understanding of cultural differences. we have attempted here to provide a holistic view of the research program on evaluation of cultural inclusion in llms by situating the current work within a broader landscape of "culture," thereby identifying gaps and potential scope of future research. despite the tremendous progress in nlp, culture remains as one of the hardest aspects of language that the models still struggle with. the amorphous nature of culture and the fact that it is always contextual and situated, which is to say that there is always a need for "thick descriptions" @xcite -an aspect that digital text corpora can rarely capture in its entirety, creates bottlenecks for textbased llms to master cultural nuances. digitally under-represented cultures are more likely to get represented by their "thin descriptions" created by "outsiders" on the digital space, which can further aggravate the biases and stereotypes.