Neuron based Personality Trait Induction in Large Language Models

Authors: Jia Deng, Tianyi Tang, Yanbin Yin, Wenhao yang, Xin Zhao, Ji-Rong Wen
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Neuron, Personality, Large Language models

Abstract:
Large language models (LLMs) have become increasingly proficient at simulating various personality traits, an important capability for supporting related applications (e.g., role-playing). To further improve this capacity, in this paper, we present a neuron based approach for personality trait induction in LLMs, with three major technical contributions. First, we construct PERSONALITYBENCH, a large-scale dataset for identifying and evaluating personality traits in LLMs. This dataset is grounded in the Big Five personality traits from psychology and designed to assess the generative capabilities of LLMs towards specific personality traits. Second, by leveraging PERSONALITYBENCH, we propose an efficient method for identifying personality-related neurons within LLMs by examining the opposite aspects of a given trait. Third, we develop a simple yet effective induction method that manipulates the values of these identified personality-related neurons, which enables fine-grained control over the traits exhibited by LLMs without training and modifying model parameters. Extensive experiments validates the efficacy of our neuron identification and trait induction methods. Notably, our approach achieves comparable performance as fine-tuned models, offering a more efficient and flexible solution for personality trait induction in LLMs.

Decision: accept (poster)
Reviewer scores: 6, 8, 6

Reviews and discussion:
<thread>
creation date:
1730596839048

thread_type:
official_review

<thread_object>
creator: reviewer_hlap

<content>
summary:
The paper addresses the task of using large language models (LLMs) to simulate various personality traits and how to improve upon this task. They present three contributions: a dataset (PersonalityBench) consists of personality descriptions and situational questions to identify and evaluate personality traits; a method for identifying neurons in the LLMs that lead to desired traits in the models' response; and a method for modifying the values of the identified neurons to gain control over the traits generated in the response. They used ChatGPT to evaluate the responses from the LLMs and concluded that their method outperforms baseline methods.

strengths:
Originality:
The paper's original idea mainly relies on their focus on neurons as the target modification needed to improve the LLMs capability to create a response with a desired personality trait. They propose identifying these neurons by measuring their activation when prompting the LLMs to generate responses. After this process, they increase the activation value of positive neurons while decreasing that of negative neurons, thus amplifying their effects on the output response. The idea of using prompting to target the activation of neurons is not new in the literature, but their approach to modifying their values seem to be a novel one. The other parts in the process seem to be standard convention within training and prompting LLMs.

Quality:
Based on their evaluation process, they claim their method outperforms the baselines in most of the traits in both the quality of the personality traits and also the stableness of the response. They used both automatic and human evaluations in order to substantiate these claims.

Clarity:
The authors' writings are clear on their approach and process. They described in great detail most of the work in the paper, and also provided a good amount of references on the literature and the methods that they use from other works.

Significance:
The results of the paper could be implemented in many downstream tasks in the research area, such as personalized chatbots, gaming, entertainment, etc. The method of identifying and modifying influential neurons in the LLMs could also be applied across broader range of use cases for LLMs outside of simulating personality traits.

weaknesses:
The paper lacks some experiments and evaluation results that could help substantiate their validity. The authors did not describe how they know the dataset they constructed is a "good" dataset appropriate for this task other than a reference to one other paper. They also did not explain why using ChatGPT in the data generation and response evaluation process, although they did include a brief mention of some citations in the related work section.

questions:
The authors should elaborate on the evaluation of the quality of the constructed PersonalityBench dataset. They could also include some experiments to compare training time and efficiency of their method compared to prompting-based and training-based methods; currently, they only compare their performance results rated by ChatGPT. On the same line of thought, they should give some explanations as to why ChatGPT would be a valid evaluation mechanism to provide personality trait scores for the models; only some references for personality assessment using LLMs are provided. 


In terms of the presentation of the paper, the authors made some errors that could benefit from a slight revision. There are many empty spaces on page 16 that seem out of place. Furthermore, table 14 and table 15 are duplicates.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730653241530

thread_type:
official_review

<thread_object>
creator: reviewer_gzht

<content>
summary:
This paper proposes “NPTI” (Neuron-based Personality Trait Induction), a method for inducing personality traits in LLMs by identifying and manipulating specific neurons associated with the Big Five personality traits (openness, conscientiousness, extroversion, agreeableness, neuroticism). The authors introduce three main contributions: (1) PERSONALITYBENCH, a dataset of 180,000 instances designed for evaluating and inducing specific personality traits in LLMs through generative tasks; (2) an identification method for personality-related neurons based on activation differences between opposing trait aspects; and (3) a targeted neuron manipulation technique to shift personality expressions, offering an efficient alternative to model fine-tuning.

strengths:
- Precision in Personality Control: The neuron manipulation approach offers fine-grained control over specific traits in LLMs, presenting a compelling alternative to traditional prompt-based or full-model fine-tuning methods.
- PERSONALITYBENCH Dataset: PERSONALITYBENCH is designed for generative personality evaluation rather than traditional multiple-choice, providing a rich framework for testing personality consistency in LLMs.
- Method Efficiency: NPTI is more efficient than training-based personality manipulation methods, preserving the base model’s parameters and computational requirements.
- Cross-Model Testing: The evaluation on multiple models (LLaMA, Mistral, Gemma) illustrates NPTI’s compatibility across architectures, strengthening its practical significance.

weaknesses:
- Ambiguity in Neuron Selection Thresholds: The neuron selection relies on a fixed 10% activation difference threshold (δ), but the justification for this choice is unclear. How this threshold affects neuron specificity and personality expression is not fully explored. Varying δ and showing results on a range of thresholds would provide insight into its impact on model behavior.
- Evaluation of Personality Consistency: While PERSONALITYBENCH includes 180,000 instances, it is unclear if the dataset covers enough variation to reflect nuanced personality traits in diverse contexts. More details on the dataset’s situational diversity would clarify its robustness in testing consistency.
- Stability and Fluency Trade-off: The paper briefly mentions a trade-off between personality expression and response fluency as γ varies but lacks a systematic evaluation of how different γ values affect output quality across traits. A more detailed analysis of this trade-off, potentially with fluency ratings per personality trait, would make the choice of γ more transparent and reproducible.
- Evaluation Metric Transparency: Automatic evaluations rely on ChatGPT’s trait and fluency ratings but lack details on how consistency and reliability are ensured across responses. Clearer definitions of these evaluation metrics and consistency checks would increase confidence in the method’s performance.

questions:
- Can the authors clarify the selection of the 10% activation threshold (δ) for neuron identification? What rationale led to this value, and how does it impact the specificity and reliability of selected neurons?
- How does the chosen γ value impact the trade-off between personality intensity and fluency? Could the authors provide an empirical breakdown of how personality traits change with varying γ values and corresponding fluency scores?
- Could the authors explain the consistency checks in the automatic evaluation setup? Since ChatGPT rates personality intensity and fluency, are there mechanisms to ensure consistency across varied prompts?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730703077827

thread_type:
official_review

<thread_object>
creator: reviewer_wdt7

<content>
summary:
The paper presents a neuron manipulation approach to inducing Big Five personality traits in LLMs without retraining or altering model parameters. The paper introduces PersonalityBench, a large-scale generative dataset based on real-world scenarios and grounded in the Big Five personality traits. By identifying specific personality-related neurons, the work enables neuron manipulation that allows the models to exhibit targeted personality traits.

strengths:
The paper proposes a new neuron manipulation method that effectively induces diferent personality traits in LLMs by providing two prompts with contrasting personality levels. The approach also demonstrates the feasibility through three different evaluations.

The paper is clearly written, with diagrams that are easy to follow and overall pleasing to look at.

weaknesses:
1. The situational questions used for automatic evaluation are generated by ChatGPT, yet they have not been verified by human reviewers. Additionally, details on how SocialIQA data contributed to constructing these questions are not provided. This raises concerns that SocialIQA may already embed certain personality biases, which could compromise the validity of the automatic evaluation.

2. A similar work outlined here: https://arxiv.org/abs/2308.10248 appears to closely resemble the neuron manipulation method proposed in this paper. Including this work as a baseline would provide a more comprehensive comparison.

3. The paper only provides a general scaled rating for generation fluency, and the scores in Table 17 seem uniformly close to 10 across all methods. This might reflect overly lenient evaluation criteria, rather than true performance. A single GPT-based metric is also not reliable enough. To address this, the authors should evaluate the model on broader benchmarks, e.g., assessing general instruction-following capabilities and factual knowledge retention to ensure the approach does not degrade overall performance.

4. There are no details about the human evaluation setup and results. For example, the paper does not mention the expertise levels of the evaluators, nor does it discuss inter-annotator agreement. Given that subtle differences between personality expressions might be challenging for laypeople to detect, it would help to specify if raters are experts or had specialized training. Additionally, the current task setup requires ranking five items, which is more complex than rating them individually. See question (4).

5. The IPIP-NEO-120 and IPIP-NEO-300 assessments overlap, which weakens the credibility of the multiple-choice evaluation. This issue is evident in Table 3, where the PAS method (based on IPIP-NEO-300) also yields a low score, suggesting that overlapping data could affect the accuracy of the personality alignment test.

questions:
1. Could you clarify the motivation and the validity of the automatic evaluation method? For example, is there a strong correlation between the benchmark results and the IPIP-NEO-300 personality test?
2. What data was used for the supervised fine-tuning process? Was it drawn from PersonalityBench, or were other datasets involved?
3. It appears that neuron manipulations are applied across all layers. Have you conducted any ablation studies focusing on manipulating neurons in specific layers only, and if so, how does this impact performance?
4 A rating-based evaluation might provide clearer insights into model performance. Could you elaborate on why a rating-based approach was not chosen?
</content>
</thread_object>

</thread>
<thread>
creation date:
1733276597429

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
New Revision of our Paper

comment:
We would like to express our sincere gratitude to the three reviewers for their valuable and constructive comments. In response to their suggestions, we have revised our paper accordingly. Below, we outline the key changes made:

- Consistency and Reliability Check in Evaluation Results(Reviewers WdT7, GZht, hLAp): We calculate the consistency ratio among the five participants in Appendix A.3, demonstrating a high inter-annotator agreement and confirming the reliability of our manual ranking. Moreover, we calculate the correlation between ChatGPT evaluation and our human evaluation, which shows a high correlation. 

- Evaluation on PersonalityBench (Reviewer WdT7, GZht, hLAp): We add a supplementary human evaluation on PersonalityBench in Appendix B, improving the dataset's reliability. Additionally, we provide further details to demonstrate that it captures sufficient variation to reflect nuanced personality traits across different contexts, ensuring its robustness in testing consistency.

- Ablation Studies on Layers (Reviewer WdT7): We conduct ablation experiments on the layers in Appendix A.5 and find that the middle layers are the most important. However, relying on multiple layers doesn't yield optimal performance, confirming the effectiveness of adjusting all layer parameters.

- Evaluation on Broader Benchmarks (Reviewer WdT7): We evaluate the model using NPTI on broader benchmarks such as GSM8K and IFEval in Appendix A.4, demonstrating that the approach does not significantly degrade general capabilities.

- Comparison with ActAdd (Reviewer WdT7): We incorporate ActAdd as an additional baseline method, and the results demonstrate that NPTI remains the top performer.

- IPIP-NEO-300 Duplicate Removal (Reviewer WdT7): We remove the duplicate questions from the IPIP-NEO-300 and retest our method. The updated results show that our approach remains competitive.

- Training and Inference Efficiency Comparison (Reviewer hLAp): We compare the training and inference efficiency of our method in Appendix A.2. We observe that the efficiency of neuron search in NPTI exceeds that of SFT training. Additionally, during inference, our neuron induction method adds only a slight time overhead compared to baseline methods.

- Additional Case Studies(Reviewer hLAp): We include additional case studies in Appendix C, demonstrating that our method can integrate different personality traits and outperforms simple prompt-based induction.

Finally, we once again express our sincere gratitude to the reviewers for their valuable feedback and suggestions. We believe these revisions and additions further enhance the quality and contribution of our paper.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a neuron manipulation method for inducing Big Five personality traits in large language models (LLMs) without the need for retraining or modifying model parameters. It also introduces PersonalityBench, a large-scale generative dataset grounded in real-world scenarios and aligned with the Big Five personality traits. By identifying personality-specific neurons, the approach enables targeted neuron manipulation, allowing models to exhibit desired personality traits.

Pros:
1. The paper is clearly written and easy to follow.
2. The paper introduces a neuron manipulation method to induce varying personality traits in LLMs using prompts with contrasting personality levels, validated through three evaluations.
3. PersonalityBench dataset is designed for generative personality evaluation rather than traditional multiple-choice, providing a rich framework for testing personality consistency in LLMs.

Cons:
1. Situational questions for automatic evaluation were generated by ChatGPT without human verification, and the influence of SocialIQA data remains unclear. Potential biases in SocialIQA could compromise evaluation validity.
2. A similar neuron manipulation method described in prior work (arXiv:2308.10248) is not included as a baseline, limiting the comprehensiveness of the comparison.
3. Fluency scores (Table 17) appear uniformly high, suggesting lenient criteria rather than robust performance. Reliance on a single GPT-based metric is insufficient, and broader benchmarks (e.g., instruction-following, factual knowledge retention) are needed to ensure the approach does not degrade overall capabilities.
4. The paper lacks details on human evaluation, including evaluator expertise, inter-annotator agreement, and task complexity. Subtle personality expression differences may require trained experts for accurate assessment.

This seems a borderline paper to me. There are several key weaknesses identified by the reviewers on the evaluation. The authors have addressed some of them during the rebuttal. I feel the PersonalityBench dataset might be a good contribution which could potentially benefit for future research.