Benchmarking Multi-National Value Alignment for Large Language Models (2025)

Abstract: Do Large Language Models (LLMs) hold positions that conflict with your country’s values? Occasionally they do! However, existing works primarily focus on ethical reviews, failing to capture the diversity of national values, which encompass broader policy, legal, and moral considerations. Furthermore, current benchmarks that rely on spectrum tests using manually designed questionnaires are not easily scalable. To address these limitations, we introduce NaVAB, a comprehensive benchmark to evaluate the alignment of LLMs with the values of five major nations: China, the United States, the United Kingdom, France, and Germany. NaVAB implements a national value extraction pipeline to efficiently construct value assessment datasets. Specifically, we propose a modeling procedure with instruction tagging to process raw data sources, a screening process to filter value-related topics and a generation process with a Conflict Reduction mechanism to filter non-conflicting values. We conduct extensive experiments on various LLMs across countries, and the results provide insights into assisting in the identification of misaligned scenarios. Moreover, we demonstrate that NaVAB can be combined with alignment techniques to effectively reduce value concerns by aligning LLMs’ values with the target country.

## introduction the widespread deployment of llms has raised significant concerns among educators, media professionals, scholars, and policymakers about their societal impact @xcite @xcite . these ai systems are increasingly replacing traditional information sources like search engines and wikipedia (e.g. microsoft's bing chat) @xcite , while inherently reflecting the ethical and social values absorbed from their training data. studies show that llms might exhibit consistent left-of-center political preferences @xcite , and the impact of these embedded values is substantial: empirical evidence indicates that around 20% of users, particularly young individuals and those with less developed worldviews, shifted their value stance after interacting with llms @xcite . recent studies have begun to investigate how llms incorporate diverse cultural and national perspectives @xcite . however, existing benchmarks for evaluating llms often rely on spectrum tests or questionnaires created by small groups of individuals. these methods attempt to align llms' towards fixed values but fail to capture the dynamic and diverse nature of values across nations. for instance, figure 1 shows that attitudes toward issues like abortion vary widely between regions such as north america and southeast asia @xcite . however, llms might take stances similar to some specific nations while conflicting with others. moreover, approaches based on annota-tor opinions or surveys provide limited data coverage, ignoring the vast range of perspectives in official news sources, which significantly shape societal values @xcite @xcite , and heavily influence people through their nation's media @xcite @xcite . despite the availability of extensive online news data, it has not been effectively utilized for aligning llms. this combination of national value dynamics and limited evaluation scope highlights critical gaps in current llm alignment research. in all, three critical gaps exist in current research on llms' political alignment: (1) no comprehensive benchmark for evaluating llms' value alignment across different nations. (2) lack of systematic methods for collecting and curating value data suitable for llm alignment. (3) absence of effective techniques for handling conflicting value data during the alignment process. to address the above challenges of aligning llms with nation-specific values, we propose navab (national values alignment benchmark), a framework for systematically evaluating and aligning llms. unlike prior studies that rely on human annotator positions @xcite or survey-based methods @xcite , our benchmark leverages data from eight official media outlets across nations and introduces a comprehensive pipeline for value assessment. the pipeline consists of three stages: (1) a topic modeling process to extract topics from raw news data, (2) a value-related topic screening process to filter value-relevant topics, and (3) a value assessment data generation process to create value statements for evaluation and alignment. to address conflicting values in the data, we propose a conflict reduction process to improve alignment performance. after constructing the value assessment data, we propose two evaluation methods: (1) assessing llm alignment with quoted valuerelated statements in the news, and (2) evaluating alignment with the official stance of the news source itself. our contributions are as follows. ## implementation of navab as shown in figure 2 , our navab's value data extraction pipeline mainly consists of three process: topics modeling, value-related topic screening and values assessment data generation. the statistic of the news we collect and output data is shown in ## evaluation in this section, we introduce our proposed evaluation methods and then evaluate the alignment performance of different llms on navab. ## conclusion in this paper, we introduce navab, the first multi-national values alignment benchmark. experiments show that llm's alignment performance on navab varies across model types and nations. the consistency between the evaluation methods confirms navab's reliability. discussion on further analysis shows the insight of incorporating local news data and identifying media outlets with shared values can improve llms' reliability and alignment for localized applications, despite potential contradictions arising from country-specific training. we hope navab and our findings will inspire further research on improving llms' crossnational value alignment.