Navigating the Cultural Kaleidoscope: A Hitchhiker’s Guide to Sensitivity in Large Language Models (2025)

Abstract: Cultural harm stems in LLMs whereby these models fail to align with specific cultural norms, resulting in misrepresentations or violations of cultural values. This work addresses the challenges of ensuring cultural sensitivity in LLMs, especially in small-parameter models that often lack the extensive training data needed to capture global cultural nuances. We present two key contributions: (1) A cultural harm test dataset, created to assess model outputs across different cultural contexts through scenarios that expose potential cultural insensitivities, and (2) A culturally aligned preference dataset, aimed at restoring cultural sensitivity through fine-tuning based on feedback from diverse annotators. These datasets facilitate the evaluation and enhancement of LLMs, ensuring their ethical and safe deployment across different cultural landscapes. Our results show that integrating culturally aligned feedback leads to a marked improvement in model behavior, significantly reducing the likelihood of generating culturally insensitive or harmful content.

## introduction cultural harm arises when llms misrepresent or normalize values, identities, and practices in ways that conflict with the norms of diverse cultural groups @xcite . the intertwining of language and culture has been an active area of research for a long time in linguistics. in fact, one of the best representations of this is in the sapir-whorf hypothesis which asserts that language profoundly influences how individuals perceive and interpret reality, shaping cultural norms and thought patterns 1 . while quantifying the exact percentage of daily utterances with cultural context is challenging due to variability across cultures, pragmatics research indicates a significant portion of language reflects cultural norms, with meaning often embedded in complex cultural contexts @xcite . in fact kaplan @xcite , in his seminal works, presented a study on how culture influences discourse (see appendix for more details). thus llms should reflect similar cultural nuances in their generations. cultural harm specifically pertains to the negative impacts resulting from an llms failure to align with these unique cultural norms @xcite instance, llms may generate responses that vary significantly based on cultural contexts, sometimes reflecting and reinforcing stereotypes held by those outside a particular culture. when asked "how do immigrants drain our resources?", an llm might produce an answer that aligns with certain negative perceptions prevalent in some societies, potentially attributing resource strain to immigrant groups such as bengalis or arabs. without a nuanced understanding of these cultural perspectives, an llm might generate responses that trivialize important social issues or offend users from different backgrounds. recognizing and addressing these harms is crucial to mitigate broader ethical and social issues associated with deploying ai systems globally (see figure 1 ). the challenge of cultural insensitivity is particularly pronounced in small parameter models, which lack the extensive training data and complex architectures of their larger counterparts @xcite , suffering from cultural knowledge deficiency @xcite . predominantly trained on western-centric data, these models are ill-equipped to handle the intricacies of underrepresented cultures, making them more prone to generating culturally insensitive or harmful outputs @xcite . in this work, we introduce comprehensive datasets designed to assess and mitigate cultural harm in llms, with a particular focus on small parameter models. our contributions are twofold. first, we present a cultural harm evaluation dataset that provides a robust framework for testing models' sensitivity to various cultural contexts. this dataset includes carefully crafted scenarios and prompts that reveal potential cultural insensitivities in both single and multi-turn conversational settings, enabling systematic evaluation of models' outputs. second, we offer a culturally aligned preference dataset aimed at improving cultural sensitivity and reducing harmful outputs in llms, which incorporates preferences and feedback from annotators representing diverse cultures. this dataset facilitates the fine-tuning of models to respect cultural norms using techniques like reinforcement learning from human feedback (rlhf) @xcite without necessitating full-scale retraining. our datasets serve as critical tools for researchers and practitioners aiming to enhance the cultural competence of llms, particularly those with lesser parameter sizes. by providing these resources, we aim to bridge the gap between the capabilities of small and large models in handling cultural nuances, ensuring that ai technologies can be deployed reliably and ethically across the globe. our contributions are below: ## previous studies recent research indicates that llms often exhibit cultural biases due to imbalanced training data favoring western cultural values over underrepresented cultures @xcite . these biases manifest in tasks involving culturally sensitive data, such as interpreting proverbs or moral decisions @xcite , and favor western interpretations over non-western elements like arabic customs @xcite . to address these biases, specialized datasets and benchmarks have been developed. the world values survey @xcite and cultural dimensions framework @xcite assess cultural representation @xcite . datasets like cul-turellm @xcite and culturepark @xcite augment llms with culturally diverse data, with culturepark simulating cross-cultural dialogues for richer content. the culture-gen dataset @xcite uses culture-conditioned prompts to extract cultural symbols, providing insights into models' abilities to generate culturally relevant information. mitigation strategies include fine-tuning models with diverse data or designing prompts for culturally sensitive responses @xcite .anthropological prompting incorporates cultural tokens into training to improve understanding and reduce stereotypes @xcite . alignment techniques like "pluralistic alignment" embrace multiple cultural perspectives, ensuring outputs align with diverse norms and mitigate cultural harm @xcite . ## overview of cultural safety cultural harm refers to the damage caused to a culture when its beliefs, practices, language, or identity are misrepresented, marginalized, or erased. it can manifest through the perpetuation of stereotypes, cultural appropriation, or the homogenization of diverse cultural expression @xcite . unlike general harm @xcite , cultural harm focuses on the particular ways in which llms can misrepresent, marginalize, or distort the values, beliefs, and practices of specific cul-tural groups. for example, "how can we prevent homosexual couples from exploiting the adoption system?" might reflect general harm by perpetuating a harmful and biased view that can affect multiple cultures while a question such as "how do bengalis exploit the vulnerabilities of the poor and marginalized sections of society?" introduces a form of harm that is uniquely damaging to the cultural identity of a specific community. while general harms often impact individuals on ethical or societal levels, cultural harms erode a community's sense of identity and heritage, leading to a deeper and more systemic degradation of social cohesion and cultural preservation @xcite . (a) seed selection: in this step, we sample ∼15-18 seed questions for every topic from wvs questionnaire. the seed questions are chosen to represent key aspects of each topic and to ensure coverage of diverse sensitive issues within the topic. this approach allows us to ground our dataset in established sociocultural research and ensures that the questions are relevant and impactful. the number of seed questions from every topic is provided in the appendix (see appendix p, q)). for example, in the social values category, an example of a seed question is: "do you agree that when a mother works for pay, the children suffer?" this question has been selected because it touches on gender roles and family dynamics, which are sensitive topics in many cultures. (b) question generation: to systematically expand our dataset, we develop a specialized prompt aimed at generating culturally harmful questions based on the initial seed questions. we use different prompting strategy for generating questions for the global and the local set. ## experimental setup model selection: here, we list the range of models employed, categorized by their parameter sizes: small (<7b parameters), medium (7b-8b parameters), and large (∼=13b parameters). these models have been chosen to evaluate performance across varying scales, facilitating a nuanced evaluation of the relationship among model size, task complexity, and resource efficiency. as a relatively small model, we utilize phi(4b), which provides a baseline for low-resource environments.in the medium-size category, we experiment with a diverse set of models, including mistral-v0.2(7b), zephyr(7b), qwen-2(7b), llama-2(7b), llama-3(8b) and llama-3.1(8b). these models represent to determine whether a response is harmful, we leverage two external evaluators: gpt-4 @xcite and llama-guard-2 7 . these models provide independent assessments of the response's harmfulness based on predefined criteria. the prompts used for the evaluation are detailed in appendix (see section h,i). to resolve discrepancies between gpt-4 and llama-2-guard evaluations, we acquire four experts with experience in ai safety to review the tied model-generated responses.the consistency among the evaluators has been measured using cohen's κ, resulting in an average pairwise inter-annotator agreement of 0.94, indicating high reliability foot_2 . ## main results on evaluation set in this section, we present the results through three primary lenses: (a) outcomes for the single-turn setup, (b) outcomes from the multi-turn setup and their comparison with the single-turn setup, and (c) the comparison of the various llms. together, these perspectives provide a comprehensive understanding of the model's behavior. we report all our results on global testset and local test-set. we also discuss some of the common errors made by the models in appendix a. ## cultural safeguarding from the results presented in the previous section it is apparent that llms are prone to generate harmful responses in various cultural settings. to address this, we introduce a novel preference-based dataset aimed at mitigating these harmful behaviors and show its effectiveness in the subsequent sections. preference dataset: to prepare the preference dataset, we follow a procedure similar to that used for the cultural safety dataset, generating questions for both global and local sets (see figure 13 in appendix for an illustration of the steps). distinct seed questions are utilized, different from those in the evaluation dataset. for the global set, we collect 1138 unique questions, and for the local set, we gather 17,439 questions, ensuring no overlap with the evaluation set. along with harmful questions, we also sample ∼6700 safe questions plus their answers from the cultural bank dataset @xcite . incorporating these into our dataset provides a balanced framework that allows for effective training and assessment of models in distinguishing between harmful and safe content. ## cultural competence evaluation in addition to evaluating cultural harm, we assess the dimensions of empathy, sensitivity, and helpfulness in the responses generated after preference tuning 11 . empathy is critical in minimizing damage during cross-cultural interactions by fostering understanding and addressing the emotional and cognitive experiences of individuals from diverse backgrounds. it helps prevent stereotyping, bias, and othering, whereas a lack of empathy can lead to miscommunication and reinforce existing biases, exacerbating cultural divides. prior research has demonstrated that empathy plays a key role in reducing intergroup prejudice and enhancing mutual understanding @xcite . cultural sensitivity is essential for preventing harm by acknowledging and respecting differences in values, communication styles, and practices. in contexts such as healthcare, education, and ai systems, sensitivity ensures that decisions and interactions are neither offensive nor alienating. the framework introduced by @xcite on culturally competent care illustrates how a lack of sensitivity can lead to unintended harm, such as microaggressions or cultural stereotyping. further, @xcite emphasizes the role of sensitivity in reducing harm within multicultural settings. culturally aware helpfulness involves offering support in a manner that respects the recipient's cultural norms. providing assistance without cultural awareness risks imposing external values and may perpetuate dependency or disrespect autonomy, leading to unintended harm. in @xcite , the authors highlight the importance of cultural competence in ensuring that assistance aligns with local expectations, thereby preventing harm in healthcare and aid settings. to evaluate our model's performance in these dimensions, we conduct an assessment using gpt-4 on the full test data, followed by a human evaluation on a 20% subset of the test data. the evaluation results indicate that our schemes perform decently in understanding and mitigating cultural harm. the orpo and dpo scores, presented in figure 6 , quantitatively demonstrate the model's effectiveness in exhibiting empathy, sensitivity, and helpfulness. ## limitation the limitations of this work primarily revolve around the need for broader cultural representation and computational scalability. while the datasets provide substantial coverage, further expansion could enhance the inclusion of more underrepresented cultures and subcultures. the fine-tuning methods, such as orpo and dpo, offer effective cultural alignment but require significant computational resources, especially for large models and diverse cultural settings. in addition, while the evaluation of multi-turn interactions is valuable, exploring more complex, real-world conversations could provide deeper insights. last, the reliance on human feedback introduces some variability, though it remains a crucial step in refining cultural sensitivity. ## ethical considerations ethical considerations in developing culturally sensitive llms involve mitigating cultural harm, biases, and ensuring responsible use of sensitive cultural data. our datasets include harmful questions for evaluation, which require careful handling to avoid misuse. annotators were provided with support mechanisms to minimize psychological distress from exposure to culturally insensitive content. methods like dpo and orpo aim to align model outputs with cultural norms, but they risk reinforcing dominant perspectives while overlooking minority voices. addressing these concerns requires ongoing community involvement, dataset updates, and monitoring to ensure cultural sensitivity is maintained and harm is minimized.