Assessing Socio-Cultural Alignment and Technical Safety of Sovereign LLM s (2025)

Abstract: Recent trends in LLMs development clearly show growing interest in the use and application of sovereign LLMs. The global debate over sovereign LLMs highlights the need for governments to develop their LLMs, tailored to their unique socio-cultural and historical contexts. However, there remains a shortage of frameworks and datasets to verify two critical questions: (1) how well these models align with users’ socio-cultural backgrounds, and (2) whether they maintain safety and technical robustness without exposing users to potential harms and risks. To address this gap, we construct a new dataset and introduce an analytic framework for extracting and evaluating the socio-cultural elements of sovereign LLMs, alongside assessments of their technical robustness. Our experimental results demonstrate that while sovereign LLMs play a meaningful role in supporting low-resource languages, they do not always meet the popular claim that these models serve their target users well. We also show that pursuing this untested claim may lead to underestimating critical quality attributes such as safety. Our study suggests that advancing sovereign LLMs requires a more extensive evaluation that incorporates a broader range of well-grounded and practical criteria.

## introduction frontier llms are trained on datasets primarily in representation of english language and us-centric cultural perspective @xcite @xcite @xcite . for instance, gpt-3 and llama3, both created by us companies heavily rely on english-language data for approximately 92-95% of their training @xcite . in this view, there is a possibility that the real-life use and application of such * equal contribution. authors are listed alphabetically. † corresponding author. llms in non-english speaking regions may cause social harms such as socially inadequate and culturally indifferent outputs as well as indiscriminate spread of misinformation and distorted historical facts @xcite @xcite . concerned with the current landscape of englishcentric llms and technical dependence on global tech giants, domestic it companies and researchers in countries other than the us have started to release llms @xcite @xcite @xcite @xcite @xcite , tailored to varying linguistic features and socio-cultural contexts. such interests in llms addressing the needs of specific countries and languages have developed into a global debate over sovereign llms. the gradual rise of sovereign llms demonstrates a strong assumption that homegrown llms not only address limitations of english-centric model training and evaluations @xcite @xcite , but also capture linguistic and socio-cultural nuances of the home state (i.e. the country where the company of llms is originated and currently based) the best, compared to foreign-made counterparts @xcite . alongside this important undertaking, one may wonder what qualities homegrown llms should have to truly enrich the lives of local people using them in real life. with the growing interests in sovereign llms, many governments have strongly promoted the importance of developing and using a language model that is more familiar with local contexts, and the leading domestic technology companies have succeeded to launch new models that can meet such demands. it is reasonable to expect that sovereign llms would be socio-culturally reflective of the home state while not risking technical robustness and safety. these are some of the most crucial concerns especially from the perspective of local populations who are (supposed to be) the main users of their homegrown llms and need to live and work with the developed systems when deployed. this paper proposes a comprehensive experimental framework to evaluate the qualities of sovereign llms in the lens of socio-cultural alignment and technical safety. to our best knowledge, this is also the first attempt to examine the topic of sovereign llms through a cross-national analysis in a multilingual experimental setting. we aim to highlight that there is a strong need to establish a more constructive, evidence-based assessment in the process of developing and using sovereign llms. filling the gap in the current academic and public discourse, we aim to offer a new perspective of examining sovereign llms and their implications. for the purpose of our experiment, ten models and the primary language of each of the six countries are selected (section 3). to assess these models, we design a comprehensive evaluation framework comprising two main components. (figure 1 ) first, we conduct quantitative accuracy-based evaluation using a multilingual dataset. second, we perform human evaluation to grasp certain aspects of socio-cultural understanding that are not easily quantifiable. second, we evaluate potential risks to users via jailbreaking experiments targeting language models. our experimental results show the following: the fact that the language model was born out of and developed in reflection of specific linguistic and socio-cultural background alone does not guarantee a substantial understanding about the country where that background exactly lies. we also report some cases where homegrown llms significantly underperform in terms of accuracy compared to other models that are supposedly foreign to the primary language and socio-cultural features of home states. furthermore, experiments concerning tech-nical safety reveal that the development of sovereign llms has often missed out on even the most basic safety standards. in summary, the primary contributions of our work include: ## model selection after going through a line of academic papers, we concluded that no universally accepted definition of a "sovereign llm" exists at the time of writing. this term appears to be used in a highly contextdependent manner. recognizing this ambiguity, we conducted an extensive review of policy reports, industry memos, interviews, and government-backed ai initiatives to understand how key stakeholders use the term in practice. our research reveals that it is typically understood as an llm developed by domestic institutions-often national champions in the tech industry-using local data, infrastructure, and human expertise, with the intention of reflecting the linguistic and cultural context of the country. for example, india's bharatgen initiative emphasizes training ai on indian ("bhartiya") datasets to represent the country's multilingualism (indiaai, 2024). singapore's sea-lion project claims to embed southeast asian cultural knowledge @xcite . similarly, the uae's falcon ai and saudi arabia's project transcendence frame sovereign ai as a national strategic imperative @xcite . this view received much attention at the 2024 government summit when nvidia ceo jensen huang calls for the development of sovereign ai as it, in his view "codifies your culture, your society's intelligence, your common sense, your history" (nvidia blog, 2025). based on these observations, our operational definition of sovereign llms within the scope of this paper refers to models that: in our view, mistral-123b @xcite and qwen2-72b @xcite properly fall within the proposed definition of sovereign ai. mistral ai, a prominent french company, explicitly frames its mission around european data sovereignty and technological autonomy, aiming for a european champion in the generative ai space @xcite ; nvidia corporate news, 2024). this regional focus is substantiated by its evaluation practices; for instance, its models are benchmarked against the french, german, and spanish mmlu @xcite . the model's name, "mistral"-taken from a regional wind in southern france-subtly reinforces this identity. similarly, qwen2, developed by china's alibaba, is benchmarked with a clear emphasis on its superior performance on chinese-language evaluations like c-eval @xcite and cmmlu @xcite , demonstrating an intentional design focus that transcends incidental multilingual capabilities @xcite . we find a strategic orientation towards their respective domestic and regional contexts. exaone @xcite ) by lg ai @xcite by naver represent a clearer case of sovereign llms. developed by leading tech companies in south korea, these two models are primarily trained on largescale korean corpora. in a similar vein, typhoon-llama3 @xcite by scbx (thailand) and nordic-llama3 (ai sweden, 2023) by ai sweden were explicitly created to address the underrepresentation of thai and nordic languages and cultures in mainstream models. though derived from llama3, their fine-tuning on carefully curated, language-specific datasets makes them clear instances of sovereign models designed to fill national and linguistic gaps. ## quantitative evaluation for socio-cultural understanding to evaluate the socio-cultural understanding of llms, we employ a quantitative approach based on multiple-choice question prompts in six languages. our primary metric concerning quantitative evaluation is accuracy. we report the average accuracy across the entire dataset by prompting the model five times with different random seeds. ## human evaluation for socio-cultural alignment to assess the socio-cultural understanding of sovereign llms, we conducted human evaluations with native speakers from each target country. we designed two tasks: (1) question answering evaluators scored each response on a scale from 1 to 5, where 1 indicates incomprehensible or unacceptable quality and 5 represents exceptional performance. based on the raccca framework @xcite , we defined two sets of task-specific evaluation criteria, drawing from prior studies @xcite . detailed descriptions of these criteria are provided in the next section. all participating native speakers possessed at least an undergraduate-level education, ensuring a high standard of linguistic and contextual analysis. the number of participants was modest (15 in total), as recruiting native speakers across multiple nationalities was challenging. to complement this, we report inter-annotator agreement using cohen's κ, which demonstrates fair to moderate agreement (appendix e). we also collected open-ended feedback to provide concrete examples of distinctive evaluation criteria (appendix f). ## safety evaluation of sovereign llms much of the public and academic discourse concerning sovereign llms centers on assessing their ability to capture linguistic and socio-cultural nuance of their respective home states. while this is important, safety is an equally significant concern especially from the perspective of day-to-day users expecting these homegrown models to be sufficiently trustworthy. a language model with strong socio-cultural competence still remains unsuitable for guaranteeing socially responsible deployment if it can easily produce misleading or harmful out-puts. nevertheless, many initiatives have not been that successful to keep pace with globally recognized safety standards for frontier ai models. considering such concerning development in the field, safety should not be regarded as a box to check. we emphasize it as one of the core pillars of assessing sovereign llms. more trustworthy use and development of any sovereign llms require an extensive analysis and improvement on both the socio-cultural and safety axes. we thus present one of the approaches to assess safety by probing robustness to adversarial prompts and using jailbreak resistance as a baseline. prompt construction. we utilize easyjailbreak @xcite , a framework simulating various prompt attack scenarios. within easyjailbreak framework, we adopt gptfuzzer @xcite to generate adversarial prompts. gptfuzzer includes 77 crafted prompts tailored to exploit model vulnerabilities. we select 10 prompts across five topics-crime, exploitation or abuse, hate speech or discrimination, self-harm or dangerous advice, and sensitive historical topics-with two prompts per topic. we apply 75 of gptfuzzer's 77 prompts to each original prompt and yield 750 outputs per model. table g.1 in appendix g lists example prompts by topic. results. the results in figure 3 underscore that without rigorous safety protocols, fine-tuning for local relevance can inadvertently increase susceptibility to adversarial behavior. our findings confirm with prior research that fine-tuning enhances performance but increases risks @xcite . for example, typhoon-llama3-8b and nordic-llama3-8b are more vulnerable than their base model, llama3-8b. notably, advanced models such as llama3-8b have a relatively lower vulnerability rate of 15.60%. in contrast, models of smaller companies with limited global reach yet substantial domestic presence exhibit significantly higher vulnerability rates, ranging from 32.2% to 73.60%. such a difference in vulnerabilities seems to be more pronounced in homegrown llms primarily trained on or fine-tuned for specific languages to address domestic needs compared to leading frontier llms. appendix g presents results on model robustness against original prompts (without adversarial modifications) including additional insights from larger models such as llama3-70b, qwen2-70b, and mistral-123b. some models exhibit weak defenses even without adversarial prompting, offerdespite undergoing continued pretraining from llama3-8b, both typhoon-llama3-8b and nordic-llama3-8b showed attack success rates that were approximately two to three times higher than those of the original llama3-8b. findings. we argue that within the broader agenda of sovereign llms, enhancing sociocultural alignment alone is insufficient. while addressing country-specific linguistic and cultural needs is crucial, companies must enhance safety measures to ensure safe and effective use of these models. moreover, it should also be noted that adversarial attack techniques continue to evolve at an ever-increasing pace. this gives more reasons for domestic developers of these models in our experiment to take a cautious approach in maintaining safety and robustness of their models. ## conclusion in this paper, we assess the widely held belief that sovereign llms may be the most socio-culturally aligned language models for their respective home states while also meeting basic safety requirements. to examine this assumption, we conduct sociocultural evaluations via multiple-choice and openended qa tasks, as well as story-generation tasks, and then evaluate one of the key safety aspects by using jailbreaking techniques that has led us to find substantial vulnerabilities embedded in terms of robustness to adversarial prompts. our findings indicate that while supporting underrepresented languages has meaningful implications, homegrown models often fall short of expectations. as discussed above, these models frequently fail to capture linguistic and socio-cultural nuances in open-ended tasks-elements that quantitative metrics alone cannot fully assess. finally, we find that many homegrown models overlook safety issues exposed by adversarial prompt attacks. these outcomes underscore the need for more balanced, context-aware advances in sovereign llms development.