High-Dimension Human Value Representation in Large Language Models (2025)

Abstract: The widespread application of Large Language Models (LLMs) across various tasks and fields has necessitated the alignment of these models with human values and preferences. Given various approaches of human value alignment, such as Reinforcement Learning with Human Feedback (RLHF), constitutional learning, and safety fine-tuning etc., there is an urgent need to understand the scope and nature of human values injected into these LLMs before their deployment and adoption. We propose UniVar, a high-dimensional neural representation of symbolic human value distributions in LLMs, orthogonal to model architecture and training data. This is a continuous and scalable representation, self-supervised from the value-relevant output of 8 LLMs and evaluated on 15 open-source and commercial LLMs. Through UniVar, we visualize and explore how LLMs prioritize different values in 25 languages and cultures, shedding light on the complex interplay between human values and language modeling.

## introduction the remarkable capabilities of llms have revolutionized general-purpose ai assistants leading to their widespread adoption in many tasks and fields @xcite @xcite . ensuring llms align with ethical and societal values has become as crucial as achieving high task performance @xcite . numerous efforts have been made to imbue ai systems with ethical principles and moral values, from designing robust frameworks for value alignment (e.g., rlhf, rlaif etc.) @xcite @xcite @xcite to incorporate diverse perspectives into training data @xcite @xcite @xcite . these methods aim to make llms more performant, fairer, less toxic, and align better with human values. human values and preferences encompass a wide range, from universal ethical principles to culturally specific values, social etiquette, to industry and domain-specific preferences ( §2.1) and often become the foundation of ai regulations and guidelines. while llms are trained to incorporate these values, differences may emerge due to the crowd-sourced annotations and variations in rlhf efforts across different languages @xcite @xcite . for example, whereas the majority of english language llms produced by north american institutions tend to manifest american coastal liberal values @xcite , and those from chinese institutions might incorporate additional chinese values @xcite @xcite . the values pre-trained in llms are not always clear, and it is uncertain if different models reflect consistent values within a language or culture. to better understand the human values of llms, one can use surveys of human values to query llms @xcite @xcite . while the surveys are useful, they capture an incomplete picture of llm value distributions, as they only explore constrained subspace with a limited number of dimensions. for instance, the cultural values @xcite only uses 6 dimensions to represent a vast variability in human cultures, while the theory of basic values @xcite @xcite and the world value survey (wvs) @xcite @xcite , each represented 19 and 10 dimensions of values, respectively. we argue that such a lowdimension semantic representation will likely fail to give a full picture of human values in llms. instead, we aim a high dimension representation of human value distribution to reflect the complexity of the embedded values in llms. ideally, this representation must be orthogonal to the linguistic patterns and the model architecture. in this paper, we propose universal value representation (univar), the first highdimensional representation of human values in llms. we formulate the value embedding learning problem and adopt a siamese network structure @xcite @xcite @xcite to enable the model to capture values while filtering out irrelevant information. to train univar, we generate 21k value-eliciting questions based on 87 core human values and collect responses from 15 llms in 25 languages, resulting in a diverse dataset of 1m qa pairs. previous works suggest that llms express distinct values across languages @xcite @xcite , we treat language variations as distinct value representations within each model. we assess univar by performing value identification tasks with k-nn and linear probing, and demonstrate that univar embeddings effectively capture value-relevant features in llms. through visualizing univar, we further show how it captures cultural similarities and differences in values within llms (figure 1 ). univar offers systematic and statistical approach to understand value systems of llms. it facilitates explorations of how llms learn and prioritize values in different languages, and is ultimately a powerful tool for more transparent and accountable llms. we release univar models and code. ## conclusion the adoption of llms across various fields necessitates understanding how these models represent human values. our paper introduces univar, a high-dimensional, language-and model-invariant representation, that enables better understanding of the human value aspect in llms. univar allows us to examine how different llms prioritize values across languages and cultures, shedding light on the complex interplay between human values and ai systems. our approach enables us to statistically analyze the value systems embedded in llms, providing transparency and accountability in developing and using ai technologies. this approach helps align llms with human preferences, providing insights into the value systems embedded in these ai technologies.