Self-Pluralising Culture Alignment for Large Language Models (2025)

Abstract: As large language models (LLMs) become increasingly accessible in many countries, it is essential to align them to serve pluralistic human values across cultures. However, pluralistic culture alignment in LLMs remain an open problem. In this paper, we propose CultureSPA, a Self-Pluralising Culture Alignment framework that allows LLMs to simultaneously align to pluralistic cultures. The framework first generates questions on various culture topics, then yields LLM outputs in response to these generated questions under both culture-aware and culture-unaware settings. By comparing culture-aware/unaware outputs, we are able to detect and collect culture-related instances. These instances are employed to fine-tune LLMs to serve pluralistic cultures in either a culture-joint or culture-specific way. Extensive experiments demonstrate that CultureSPA significantly improves the alignment of LLMs to diverse cultures without compromising general abilities. And further improvements can be achieved if CultureSPA is combined with advanced prompt engineering techniques. Comparisons between culture-joint and culture-specific tuning strategies, along with variations in data quality and quantity, illustrate the robustness of our method. We also explore the mechanisms underlying CultureSPA and the relations between different cultures it reflects.

## introduction large language models, such as @xcite , have gained widespread use due to their extensive knowledge and prowess in downstream tasks @xcite @xcite . given the multicultural nature of our society, it is essential for llms to serve diverse human values and preferences across cultures. however, existing alignment techniques, such as rlhf @xcite and dpo @xcite , do not specifically take cultural diversity into account. with such alignment techniques, llms tend to learn biased human values and preferences @xcite @xcite @xcite . many studies examine how well llms align to serve specific cultures by simulating social surveys on llms @xcite @xcite @xcite @xcite . in these studies, the similarity between the outputs of an llm and realworld survey answers from a specific culture is calculated as the cultural alignment score (cas) between the llm and given culture. findings with cas suggest that llms often exhibit cultural dominance, as shown in figure 1 (culture-unaware prompting), where llama3's outputs naturally align more closely to certain north american and european cultures. to mitigate the reduction of llms in distribu-tional pluralism, efforts are dedicated to pluralistic value alignment in pre-training @xcite @xcite , alignment training @xcite @xcite , and prompt engineering @xcite @xcite @xcite . however, training-based approaches require external cultural data, which are often scarce, especially for underrepresented cultures. meanwhile, prompt engineering methods necessitate careful example selection and can yield inconsistent results @xcite . to address these issues, we propose to explore self-pluralising culture alignment without relying on external cultural resources. our approach is grounded in two key findings: (1) research in prompt engineering shows that llms possess a certain level of internal knowledge about diverse cultures. as illustrated in figure 1 (culture-aware prompting), simply prompting llama3 to align to a given culture is an effective way to enhance its cultural alignment; (2) studies on data synthesis @xcite indicate that llms can generate data using their existing knowledge to improve performance on specific tasks. building on these findings, we explore the following research question: can we harness the internal culture knowledge of llms to enhance their alignment to specific cultures? to this end, we propose cuturespa, a framework that achieves pluralistic culture alignment in llms by "activating" their internal culture knowledge. as illustrated in figure 2 , cuturespa first generates survey questions on diverse culture topics ( §4.1). it then collects llm outputs for these questions under two scenarios: culture-unaware prompting, where the model does not receive specific cultural information, and culture-aware prompting, where the model is prompted to align to a specific culture ( §4.2). samples that exhibit shifted outputs when cultural information is provided are deemed the most representative of a specific culture. culturerelated qa pairs collecting is employed to select such samples ( §4.3). the collected data instances are ultimately used for culture-joint and culturespecific supervised fine-tuning (sft) ( §4.4). we conduct extensive experiments to examine culturespa. experimental results indicate that cul-turespa effectively enhances llm alignment to pluralistic cultures and can be integrated with advanced prompt engineering techniques ( §5.3). a comparison between culture-joint and culturespecific sft strategies demonstrates the superiority of the former ( §5.4). additionally, we explore the mechanism behind culturespa ( §6.1), investigate cross-cultural relationships ( §6.2), and examine the effects of data quality and quantity ( §6.3). we summarize our contributions as follows: ## preliminary in this section, we first define culture and culture alignment, then present the framework used to assess the cultural alignment of llms. ## experiments we conducted extensive experiments to examine the proposed framework against various baselines. ## analysis in addition to the above experiments, we conducted in-depth analyses into the framework to understand how culturespa works. ## conclusion in this paper, we have presented culturespa (self-pluralising culture alignment), a novel framework that improves the cultural alignment of llms without using mass external cultural data. our experiments demonstrate the effectiveness of culturespa, confirming that the internal knowledge of llms related to diverse cultures can be activated to enhance their alignment with specific cultures. comparisons between culture-joint and specific sft, along with variations in data quality and quantity, demonstrate the robustness of our method. further exploration of the mechanisms behind culturespa and the cultural relationships reflected in llm outputs reveals interesting findings.