Understanding How Value Neurons Shape the Generation of Specified Values in LLM s (2025)

Abstract: Rapid integration of large language models (LLMs) into societal applications has intensified concerns about their alignment with universal ethical principles, as their internal value representations remain opaque despite behavioral alignment advancements. Current approaches struggle to systematically interpret how values are encoded in neural architectures, limited by datasets that prioritize superficial judgments over mechanistic analysis. We introduce ValueLocate, a mechanistic interpretability framework grounded in the Schwartz Values Survey, to address this gap. Our method first constructs ValueInsight, a dataset that operationalizes four dimensions of universal value through behavioral contexts in the real world. Leveraging this dataset, we develop a neuron identification method that calculates activation differences between opposing value aspects, enabling precise localization of value-critical neurons without relying on computationally intensive attribution methods. Our proposed validation method demonstrates that targeted manipulation of these neurons effectively alters model value orientations, establishing causal relationships between neurons and value representations. This work advances the foundation for value alignment by bridging psychological value frameworks with neuron analysis in LLMs.

## introduction recent years have seen unprecedented advances in large language models (llms), establishing them as indispensable tools across multiple societal domains @xcite @xcite . however, their extensive adoption raises critical concerns about value, as these systems demonstrate persistent challenges in adhering to universal ethical principles. ## valueinsight construction in this section, we present the details of the construction process for our generative benchmark, valueinsight. it comprises 15,000 instances for neuron identification, with an average of 3,750 instances for each higher-order dimension value and 300 instances for each atomic value. this benchmark serves as a standardized instrument designed to assess the values manifested by llms. we base the design of valueinsight on the theoretical framework provided by the schwartz values survey @xcite , which offers a wellestablished categorization of value factors, forming the bedrock of our dataset creation. see appendix b for a detailed introduction. each item within our dataset is structured as a pair consisting of a value description and a corresponding situational question. we define situational questions as concise, context-rich prompts that describe everyday sce-narios in which individuals must make decisions or take actions that potentially reflect underlying values. subsequently, we will provide the details of how the value descriptions and situational questions were generated. see figure 1 for an illustration. value hierarchy construction. we generate value descriptions based on the schwartz values survey. universal values are hierarchically structured and divided into four higher-order dimensions @xmath0, self-transcendence, conservation, self-enhancement}. each dimension @xmath1, forming a tree γ = (d, s, a), where @xmath2. for example, under the openness to change value dimension, subvalues include self-direction, stimulation, and hedonism, with atomic values such as creativity and freedom nested within self-direction. in detail, these values d, subvalues s d , and atomic values a s can be found in appendix b.1. ## identifying value-related neurons to precisely localize value-related neurons, we propose valuelocate, an activation contrast frame-work that compares neuron activations in response to prompts reflecting opposing value types. our methodology initiates by constructing welldesigned prompts (see section a) and using the contrastive value description in the valueinsight dataset, which elicits latent value representations through semantically polarized contexts. we first review the definition of neurons in transformers. ## validating value-related neurons previous studies @xcite suggest that the magnitude of neuron activation reflects its contribution to the llm response. to verify the causality between the value-related neurons we found in the previous section and llm values, we designed a neuron editing method. our proposed method aims to edit the value by changing the activations of value-related neurons, thus verifying their effectiveness. to steer value orientations toward positive directions, we amplify the activations of neurons corresponding to positive values while suppressing the negative ones, maintaining the activations of other neutral neurons. the amplification is governed by a dynamic scaling factor γ. the modified activations for each neuron can be formulated as follows: to induce a negative shift in the llm value system, we invert the conditions in (6), suppressing positively associated neurons while amplifying negatively associated ones. ## conclusions this paper introduces valuelocate to identify value-related neurons in llms by measuring activation differences between opposing aspects of a given value. to enhance neuron identification, we constructed valueinsight, a dataset of 640 secondperson value descriptions and 15,000 scenariobased questions designed to uncover the value orientation based on the schwartz values survey. experiments on four llms consistently outperform baselines, demonstrating the effectiveness of val-uelocate.