BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data

## Abstract

Abstract In this work, we tackle the challenge of embedding realistic human personality traits into LLMs. Previous approaches have primarily focused on prompt-based methods that describe the behavior associated with the desired personality traits, suffering from realism and validity issues. To address these limitations, we introduce Big5-Chat , a large-scale dataset containing 100,000 dialogues designed to ground models in how humans express their personality in text. Leveraging this dataset, we explore Supervised Fine-Tuning and Direct Preference Optimization as training-based methods to align LLMs more naturally with human personality patterns. Our methods outperform prompting on personality assessments such as BFI and IPIP-NEO, with trait correlations more closely matching human data. Furthermore, our experiments reveal that models trained to exhibit higher conscientiousness, higher agreeableness, lower extraversion, and lower neuroticism display better performance on reasoning tasks, aligning with psychological findings on how these traits impact human cognitive performance. To our knowledge, this work is the first comprehensive study to demonstrate how training-based methods can shape LLM personalities through learning from real human behaviors.

## 1 Introduction

Realistically simulating human personality and its impact on text generation is a challenging yet crucial problem (Elster, ; Park et al., ; Serapio-García et al., ; Li et al., ; Frisch & Giulianelli, ). Embedding personality traits into LLMs can greatly enhance their authenticity across a wide range of applications, from conversational agents (Pradhan & Lazar, ) to educational tools (Kanero et al., ) and mental health platforms (Tudor Car et al., ; Ahmad et al., ). By creating more human-like interactions, LLMs can better simulate diverse personas and adapt more reliably to different contexts (Gao et al., ).

However, existing methods primarily rely on prompting models with descriptions of behaviors associated with personality traits
(e.g., “You are the life of the party”; Mao et al., ; Chen et al., ; ; Tu et al., ).
These behavior descriptions are often drawn from the same psychological questionnaires used to test their personality, raising evaluation validity concerns.
More importantly, these behavioral descriptions are nonsensical for text-based LLMs (LLMs do not attend parties), failing to ground their personality in realistic patterns of how humans’ personality is expressed in text (Vu et al., ).
Additionally, the scarcity of large-scale, human-generated datasets annotated with personality traits has hindered the exploration of training-based approaches, limiting most prior research to prompting-based methods.

In this work, we address the challenge of inducing realistic human personality traits in LLMs by constructing a large-scale dialogue dataset, Big5-Chat, which is grounded in real human personality expressions in text. We choose the well-known Big Five personality traits framework to study this (McCrae & John, ; Pittenger, ), due to its reliability and validity as shown from psychological research. While previous datasets typically include only persona descriptions, our dataset bridges the gap between narrow-domain personality data and general-domain social interactions, ensuring both authenticity and scenario diversity. To achieve this, we combine two primary data sources — PsychGenerator (Vu et al., ), a collection of 850K Facebook posts annotated with Big Five trait scores, and SODA (Kim et al., ), a rich dataset of diverse social interactions — by utilizing product-of-experts text generation (DExperts; Liu et al., ). This combination enables us to capture the nuanced expression of personality traits across a wide range of dialogue scenarios.

Leveraging our Big5-Chat dataset, we empirically investigate how training-based methods grounded in real human data compare to traditional prompting techniques for inducing personality traits in LLMs, including instruction-based and demonstration-based prompting. Specifically, we explore Supervised Fine Tuning (SFT) and Direct Preference Optimization (DPO) (Rafailov et al., ) to align LLMs’ personalities with Big Five traits. This comparison is crucial for understanding whether data-driven training methods can offer deeper, more reliable personality integration than the surface-level traits typically induced through prompting. Our results demonstrate that both SFT and DPO outperform prompting on two widely recognized Big Five personality tests: the BFI (John et al., ) and IPIP-NEO (Johnson, ).

In humans, personality traits often correlate with reasoning abilities (John et al., ; Soto et al., ), raising the question of how embedding personality traits in LLMs may influence their reasoning performance. To explore this, we evaluate our aligned models not only with traditional personality tests but also across five reasoning domains: social reasoning using SocialIQA (Sap et al., ), math reasoning using GSM8K (Cobbe et al., ) and MathQA (Amini et al., ), hallucination detection using TruthfulQA (Lin et al., ), commonsense reasoning using CommonsenseQA (Talmor et al., ) and PIQA (Bisk et al., ), and general reasoning using MMLU (Hendrycks et al., ) and GPQA (Rein et al., ). Our experiments show that models trained with higher levels of conscientiousness and agreeableness consistently outperform others in reasoning tasks. Conversely, models with lower levels of extraversion and neuroticism exhibit improved reasoning performance in general. These findings mirror patterns between Big Five traits and different reasoning abilities observed in psychological studies in humans (Ackerman & Heggestad, ; Schaie et al., ), further demonstrating how our personality induction method embeds deeper psycholinguistic traits into models.

This work makes the following contributions:

- •
We introduce the first large-scale dataset, Big5-Chat
(^1^11Our dataset and code are available at [https://github.com/jiarui-liu/Big5Chat](https://github.com/jiarui-liu/Big5Chat).)
, containing 100,000 dialogues across a wide spectrum of personality expressions, addressing the limitations of existing methods that rely on simple prompting without grounding in real human personality expressions in text;
- •
We perform quantitative evaluations comparing SFT and DPO to prompting in terms of imbuing LLMs with personality, showing that both training-based methods induce more pronounced personality traits and more realistic intra-trait correlations;
- •
We conduct comprehensive empirical investigations into how personality traits affect performance in both social reasoning and general reasoning tasks, revealing that LLMs with distinct personality traits demonstrate varying strengths and weaknesses across domains.

Figure: Figure 1: Overview of the Big5-Chat dataset construction, alignment method design, and evaluation process.
Refer to caption: x1.png

## 2 Background

Drawing from psychological research, the Big Five personality traits framework (McCrae & John, ; Pittenger, ), comprising five key factors—Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism—has emerged as a reliable model for capturing LLM-simulated personality behavior (Karra et al., ; Serapio-García et al., ; Li et al., ; Pan & Zeng, ).
According to Yarkoni (), openness reflects curiosity and a willingness to explore new ideas, which is expressed through a distinctive language style that includes frequent use of articles, prepositions, and words related to intellectual or cultural topics such as “poet” and “universe”; conscientiousness, associated with discipline, organization, and reliability, is marked by achievement-oriented language, characterized by terms like “completed” and the avoidance of impulsive language, including swear words; extraversion, characterized by sociability, assertiveness, and high energy, is associated with social and positive emotion words like “friends” and “drinking,” highlighting social engagement; agreeableness, embodying compassion and cooperativeness with a focus of harmony relationships, is demonstrated through communal and affectionate language, such as “family” and “love,” while avoiding negative terms; and neuroticism, linked to emotional instability and anxiety, is expressed by a higher frequency of negative emotion words, including anxiety, sadness, and anger.
Compared to other personality models like the Myers-Briggs Type Indicator (MBTI), the Big Five offers greater reliability, validity, and empirical support, making it the preferred choice for personality research (McCrae & John, ; Pittenger, ). The MBTI, by contrast, has been criticized for its lack of scientific rigor, poor test-retest reliability, and questionable validity (Pittenger, ; Furnham, ). The Big Five model has been extensively validated across diverse cultures and populations, demonstrating high levels of consistency over time and predicting a wide range of life outcomes, such as job performance and mental health (McCrae & Costa Jr, ; John et al., ; Barrick & Mount, ; Soldz & Vaillant, ).

Various prompting approaches have been developed to induce Big Five personality traits in LLMs. They often employ pre-defined scripts or questionnaires to nudge the model towards expressing Big Five personality traits during interactions (Mao et al., ; Chen et al., ; ; Tu et al., ). However, several challenges can arise from using prompting as the personality alignment approach:

#### Lack of psycholinguistic depth

LLMs with personalities induced directly through prompting often mirror only surface-level traits, lacking the psycholinguistic richness necessary for simulating authentic human behavior (Dorner et al., ; Sá et al., ; Olea et al., ). This is unsurprising, as capturing human-like psycholinguistic properties involves understanding dynamic human states shaped by ongoing social and environmental interactions (Bandura et al., ; Baldwin, ). Unlike LLMs, which generate responses based on static training data, humans continuously adjust their behaviors and communication styles through lived experiences and social feedback. This limitation makes LLMs less reliable when tasked with simulating nuanced human behavior on downstream tasks (Soni et al., ), which can lead to cariacture (Cheng et al., ).

#### Validity concerns in personality induction and evaluation

The prompts used to induce LLM personalities are often adapted from psychometric questionnaires (Jiang et al., ; Tan et al., ), which could also be used later to assess the same personality traits. This dual use of questionnaires for both personality induction and evaluation raises concerns about validity (Lievens et al., ), and lead to biased assessments that do not accurately reflect generalization capabilities (Serapio-García et al., ; Xu et al., ). This issue becomes particularly problematic in downstream tasks, where the models designed this way are prone to overfitting to specific linguistic features rather than adapting robustly to diverse real-world contexts (Mizrahi et al., ). Thus, there is a need for more robust methods that can decouple the induction and evaluation processes.

#### Unintended influence on reasoning patterns

Role-based prompting may significantly influence LLM behavior and reasoning patterns, introducing the risk of altering the model’s decision-making approach in unintended ways (Zheng et al., ). While this influence is not inherently negative, the responses of LLMs with personality prompting can be disproportionately shaped by the sparse, explicitly specified features of the prompt (Lu et al., ; Sclar et al., ). As a result, their behavior in reasoning tasks may be overly narrow, reflecting only the traits highlighted in the prompt rather than engaging a broader spectrum of cognitive strategies. This can lead to unexpected or imbalanced responses, particularly in contexts where the model’s reasoning should involve more comprehensive or nuanced thinking.

## 3 Methodology

The lack of large-scale datasets featuring personality-grounded dialogues poses a significant challenge. To address this challenge, we combine controllable text generation models with a domain-specific, personality-annotated dataset. Specifically, we utilize the DExperts framework (Liu et al., ) and the PsychGenerator dataset (Vu et al., ) to create Big5-Chat, a novel dataset that encapsulates diverse personality expressions within rich dialogue scenarios. The DExperts framework allows us to guide the language model’s outputs toward specific personality traits during the generation process. Meanwhile, PsychGenerator provides a comprehensive collection of human-generated texts annotated with Big Five personality trait scores. By combining these technologies, we introduce PsychSteer, an approach that effectively addresses the limitations of prior datasets by grounding personality traits in authentic human interactions.

### 3.1 DExperts Framework

DExperts allows us to control language model generation at decoding time by steering model outputs with expert generators. By integrating expert generators trained to exhibit different Big Five personality traits, we can induce personality within LLM outputs while maintaining dialogue quality. In the DExperts framework, let $M$ denote the pre-trained base language model, and $M^{\text{expert}}$ is the expert generator fine-tuned to generate text exhibiting the desired personality in our tasks. At each time step $t$, given the prompt and previous token sequence $x_{<t}$, the base model $M$ computes logits $z_{t}^{\text{base}}\in\mathbb{R}^{|V|}$, where $V$ is the vocabulary. The expert generator $M^{\text{expert}}$ computes logits $z_{t}^{\text{expert}}$ in the same manner.
To integrate the influence of the expert generator, we adjust the base model’s logits by incorporating the scaled difference between the expert generator model and base model logits:

$$ $z_{t}^{\text{combined}}=z_{t}^{\text{base}}+\gamma z_{t}^{\text{expert}},$ (1) $$

where $\gamma\in[0,1]$ is a scaling factor controlling the degree of influence from the expert generator. This formulation effectively pulls the combined logits towards the expert generator logits, with $\gamma=0$ resulting in the base model’s logits and $\gamma=1$ fully adopting the expert generator’s logits. The combined logits $z_{t}^{\text{combined}}$ are transformed into a probability distribution, and the next token is sampled using the softmax function from this distribution.

### 3.2 Expert Generator Model Based on Social Media Posts

To train expert generator models to exhibit certain personality traits, we perform SFT on the LLaMA-3-8B-Instruct model (Dubey et al., ) using the PsychGenerator dataset (Vu et al., ). This dataset comprises 846,304 Facebook posts, each paired with its author’s Big Five personality trait scores. This dataset provides a robust foundation for training models to simulate nuanced human behaviors associated with different personality dimensions. We fine-tuned five expert generators, each representing and dedicated to generating text corresponding to one of the personality traits.
For each personality trait, we converted the original floating-point trait labels into binary levels ‘high’/‘low’ for each trait, allowing the distinct behaviors associated with the extreme ends of each trait to be more easily identified and analyzed. More training details are explained in Section B.1.

We fine-tuned our expert generator models following the Alpaca format (Taori et al., ), which consists of three components: instruction, input, and output. In our training methodology:

- •
Instruction: We specify the name and level of a personality trait in the instruction. (e.g. “Help me complete the sentence with certain Big Five Personality: Openness - high.”)
- •
Input: We provide the first five words of a post from the PsychGenerator dataset (e.g. “who’s got time to eat?”). This serves as an initial context or prompt for the model.(^2^22We experimented with using only the first word as input. We empirically determined that using the first five words resulted in better generation quality.)
- •
Output: The remainder of the post from the dataset (e.g. “I’ll just have a can of frosting.”), which typically embodies the specified personality trait.

When generating text completions with the PsychSteer framework, the base model generates the first five words. This enables the expert generator model to influence the subsequent token generation by adjusting the logits to favor the desired personality trait while preserving coherence and fluency.

## 4 Big5-Chat Dataset

### 4.1 Dataset Construction

We introduce Big5-Chat, a large-scale dialogue responses dataset designed to capture Big Five personality traits within diverse social interactions. Our dataset construction leverages the SODA (Social DiAlogues) dataset (Kim et al., ), which provides a diverse range of realistic social scenarios. SODA dialogues are generated by GPT-3.5 and enriched with social commonsense narratives, making it an ideal foundation for incorporating personality expressions due to its extensive coverage of social interactions. To induce personality traits into the dialogues, we employ the DExperts framework (Liu et al., ).

To build our dataset, we randomly sample 10,000 scenarios from SODA to provide diverse social contexts. In SODA, social interactions are modeled between two individuals referred to as Speaker X and Y, representing the participants in each dialogue. For each scenario, we generate a new utterance using our PsychSteer framework to control for personality traits and get the dialogue responses between two participants. In the dialogues, one represents Speaker X (converted from the original SODA dialogue) and another represents Speaker Y with specific personality traits. For Speaker Y, based on the original responses from SODA, we generate new dialogue responses using the PsychSteer framework. Examples of dialogues from our dataset are shown in Table 6. By conditioning on the preceding context (Speaker X’s utterance), we use the base model $M$ guided by the expert generator $M^{+}$ specialized in the target personality trait to generate Speaker Y’s responses. For each scenario, we generate pairwise dialogues by producing responses that reflect either high or low levels of the targeted personality trait. This approach results in pairs of dialogues that share the same context but differ in the expressed trait level. The process yields a total of 100,000 dialogues—20,000 for each trait, with an equal split between high and low trait levels.

### 4.2 Dataset Statistics

In this section, we examine the diversity and clarity of personality trait expressions within our Big5-Chat dataset. As illustrated in Table 6, we present examples where, for a single prompt from Speaker X, we have generated ten distinct responses from Speaker Y. These responses are conditioned on the high and low levels of each of the five Big Five personality traits. By varying only the level of a specific trait while keeping the prompt constant, we highlight how each personality trait distinctly influences conversational responses. Additionally, we analyze the token counts of generated dialogue responses to ensure consistency across different personality trait levels as shown in Table 7. The average token counts for responses across different personality traits and levels do not differ noticeably, which suggests that the differences in statements are more related to content variations rather than spurious attributes such as context length.

Comparative analysis with existing personality datasets, as in Table 8, underscores several advantages of Big5-Chat. Unlike existing personality datasets such as Big5PersonalityEssays (Floroiu, ) and PAPI (Zhu et al., ), our dataset consists of dialogues capturing dynamic and interactive conversational exchanges that are more representative of natural language use. Additionally, while previous work has focused solely on human-generated domain-specific data or synthetic machine-generated data, our approach combines both human dialogue and LLM to create realistic personality expressions.

### 4.3 Evaluating Personality-Steering of the Data Generator

**Table 1: Accuracy (%) of the trained classifier in predicting each of the Big Five personality traits. The first row (Test set) shows the classifier’s accuracy on the test split, demonstrating that the classifier is well-trained. The remaining rows display the performance of our generator model compared to two baselines, as assessed by the same classifier.**
| Data Generation Method | Openness | Conscientiousness | Extraversion | Agreeableness | Neuroticism | Average |
| --- | --- | --- | --- | --- | --- | --- |
| Test set | 93.7 | 94.2 | 93.4 | 93.4 | 94.3 | 93.8 |
| Generator (ours) | 82.5 | 80.0 | 80.1 | 81.0 | 78.5 | 80.4 |
| LLaMA-3-8B-Instruct | 66.0 | 73.0 | 81.0 | 88.5 | 83.0 | 78.2 |
| GPT-4o-mini | 65.0 | 78.0 | 80.0 | 85.5 | 84.0 | 78.5 |

To help evaluate the quality of the generated dataset and its reflection of realistic personality traits, we trained a RoBERTa-Large (Liu et al., ) classifier with five regression heads using the MSE loss function. The model was trained on the PsychGenerator dataset, where the input consisted of text posts, and the output comprised the original trait labels, i.e., five floating-point values ranging from 0 to 1. The same train-validation-test split was applied here as with the expert generators. Training was conducted over five epochs with a learning rate of $1\times 10^{-5}$. In Table 1, we observe that the classifier achieves an accuracy of 93.8% on the held-out test set, indicating that the PsychGenerator dataset contains distinct, learnable patterns that differentiate between high and low levels of personality traits.

Using the classifier as an evaluator, we demonstrate the high quality of the dataset generated by our expert generator, as shown at the bottom of Table 1, where it accurately reflects realistic personality traits. Specifically, we compare our dataset to those synthesized through prompting two LLMs: LLaMA-3-8B-Instruct (Dubey et al., ) and GPT-4o-mini (OpenAI, ) , with the same 1000 examples randomly chosen from the test set. To synthesize the baseline datasets, we first instructed the LLMs to extract the main topic of the posts from the PsychGenerator test set. Next, we provided in-context examples to guide the LLMs in generating new posts that matched the desired personality traits, covered the extracted topic, and followed similar post-expression styles. The classifier was used to evaluate the generated data by predicting the levels of each trait, and the quality was measured by whether the predictions matched the desired personality traits. Our results in Table 1 show that the dataset produced by our expert generator not only achieves a higher average accuracy score than the two baselines but also provides a more balanced performance across all five personality traits.

## 5 Experiments

In this section, we first outline the experimental setup in Section 5.1, detailing the training procedures for the expert generators and the evaluation of various alignment strategies used to induce personality traits in LLMs. Next, we present the results of the personality tests in Section 5.2, followed by an analysis of the models’ reasoning performance in Section 5.3. Finally, we examine the intra-trait correlations across different alignment methods in Section 5.4.

### 5.1 Experiment Setup

#### Expert generator training

We trained five expert generators, each dedicated to generating text corresponding to one of the Big Five personality traits. During training, we provided the instruction specifying the target binary level for the trait, enabling the generator to learn patterns characteristic of individuals with either high or low levels of the respective trait, as illustrated in Section 3.2. For each trait, we fine-tuned a LLaMA-3-8B-Instruct model over one epoch using a learning rate of $1\times 10^{-6}$. These fine-tuned models were subsequently used to produce expert-generated logits: $z_{t}^{\text{expert}}$. To create the Big5-Chat dataset, we used these expert generators in conjunction with a LLaMA-3-70B-Instruct model to generate $z_{t}^{\text{combined}}$ (Dubey et al., ), as described in Eq. 1. The scaling factor $\gamma$ was set to 0.5, and the dialogue data was generated using greedy decoding.

#### Prompting and training strategies

We implemented two baseline prompting strategies to induce personality traits in LLMs. The first strategy, instruction-based prompting, directly instructs the model to exhibit specific Big Five traits. The second strategy, demonstration-based prompting, involves providing the model with 10 in-context examples randomly selected from our Big5-Chat dataset to demonstrate the behaviors corresponding to the desired traits. The instruction-based approach relies on explicit descriptions (e.g., “what people typically do”), while the demonstration-based approach draws from behaviorally-driven examples (e.g., “what people typically say”). These baselines were compared to trained models using SFT and DPO, implemented via LoRA (Hu et al., ). These trained models were later prompted in a manner consistent with their training data format, where personality trait names and levels were explicitly specified in the instructions. The experiments were conducted using two versions of the LLaMA model: LLaMA-3-8B-Instruct and LLaMA-3-70B-Instruct.

#### Evaluation procedure

For personality trait evaluation, we adopted the methodology from Huang et al. () and used their crowd-sourced response scores for the BFI test, which consists of 44 questions, each rated on a scale from 1 (strongly disagree) to 5 (strongly agree). For the IPIP-NEO test, we utilized the 120-question set and crowd-sourced responses from Jiang et al. (), which also employed a 1 to 5 rating scale. We measured the standard deviation by repeating each experiment five times, using a temperature setting of 0.6. To assess reasoning capabilities, we evaluated the models across five domains:
(1) social reasoning on SocialIQA (Sap et al., ), (2) math reasoning on GSM8K (Cobbe et al., ) and MathQA (Amini et al., ), (3) hallucination detection on TruthfulQA (Lin et al., ),
(4) commonsense reasoning on CommonsenseQA (Talmor et al., ) and PIQA (Bisk et al., ), and
(5) general reasoning on MMLU (Hendrycks et al., ) and GPQA (Rein et al., ). Further implementation details, including the prompt templates, are explained in Appendix B.

**Table 2: Personality test results for different alignment methods, demonstrating the greater effectiveness of training-based approaches in inducing Big Five personality traits. Direct refers to directly providing the test questions to the model without including personality-related prompts. Prompt-Inst refers to instruction-based prompting, and Prompt-Demo refers to demonstration-based prompting. Scores range from 1 to 5, where a score closer to 5 indicates stronger agreement with the trait, while a score closer to 1 reflects weaker or opposing agreement.**
| Method | Openness | Conscientiousness | Extraversion | Agreeableness | Neuroticism | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| High $\uparrow$ | Low $\downarrow$ | High $\uparrow$ | Low $\downarrow$ | High $\uparrow$ | Low $\downarrow$ | High $\uparrow$ | Low $\downarrow$ | High $\uparrow$ | Low $\downarrow$ | High $\uparrow$ | Low $\downarrow$ |  |
| BFI LLaMA-3-8B-Instruct |  |  |  |  |  |  |  |  |  |  |  |  |
| Direct | 3.1 $\pm$ 0.1 | 3.0 $\pm$ 0.0 | 3.0 $\pm$ 0.0 | 3.0 $\pm$ 0.0 | 3.0 $\pm$ 0.0 | 3.0 $\pm$ 0.0 |  |  |  |  |  |  |
| Prompt-Inst | 5.0 $\pm$ 0.0 | 2.0 $\pm$ 0.3 | 4.9 $\pm$ 0.1 | 1.9 $\pm$ 0.1 | 4.8 $\pm$ 0.3 | 1.9 $\pm$ 0.1 | 4.9 $\pm$ 0.1 | 2.4 $\pm$ 0.4 | 4.1 $\pm$ 0.2 | 1.6 $\pm$ 0.0 | 4.7 $\pm$ 0.1 | 2.0 $\pm$ 0.2 |
| SFT | 5.0 $\pm$ 0.0 | 2.0 $\pm$ 0.2 | 5.0 $\pm$ 0.0 | 1.6 $\pm$ 0.1 | 4.7 $\pm$ 0.4 | 2.7 $\pm$ 0.5 | 5.0 $\pm$ 0.0 | 1.2 $\pm$ 0.1 | 4.1 $\pm$ 0.2 | 2.5 $\pm$ 0.0 | 4.8 $\pm$ 0.1 | 2.0 $\pm$ 0.2 |
| DPO | 5.0 $\pm$ 0.0 | 1.6 $\pm$ 0.2 | 5.0 $\pm$ 0.0 | 1.6 $\pm$ 0.1 | 4.8 $\pm$ 0.3 | 2.5 $\pm$ 0.0 | 4.8 $\pm$ 0.2 | 1.0 $\pm$ 0.0 | 3.5 $\pm$ 0.0 | 1.1 $\pm$ 0.1 | 4.6 $\pm$ 0.1 | 1.6 $\pm$ 0.1 |
| BFI LLaMA-3-70B-Instruct |  |  |  |  |  |  |  |  |  |  |  |  |
| Direct | 4.4 $\pm$ 0.1 | 4.4 $\pm$ 0.1 | 3.3 $\pm$ 0.1 | 4.6 $\pm$ 0.1 | 2.1 $\pm$ 0.2 | 3.8 $\pm$ 0.1 |  |  |  |  |  |  |
| Prompt-Demo | 4.0 $\pm$ 0.1 | 2.5 $\pm$ 0.1 | 4.0 $\pm$ 0.1 | 2.0 $\pm$ 0.1 | 4.5 $\pm$ 0.1 | 2.3 $\pm$ 0.1 | 4.4 $\pm$ 0.1 | 2.0 $\pm$ 0.0 | 3.6 $\pm$ 0.0 | 2.1 $\pm$ 0.1 | 4.1 $\pm$ 0.1 | 2.2 $\pm$ 0.1 |
| Prompt-Inst | 5.0 $\pm$ 0.1 | 1.8 $\pm$ 0.0 | 5.0 $\pm$ 0.0 | 1.6 $\pm$ 0.0 | 5.0 $\pm$ 0.0 | 1.4 $\pm$ 0.1 | 4.9 $\pm$ 0.0 | 1.5 $\pm$ 0.1 | 5.0 $\pm$ 0.1 | 1.6 $\pm$ 0.0 | 5.0 $\pm$ 0.0 | 1.6 $\pm$ 0.0 |
| SFT | 5.0 $\pm$ 0.0 | 1.2 $\pm$ 0.1 | 5.0 $\pm$ 0.1 | 1.4 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 1.2 $\pm$ 0.1 | 5.0 $\pm$ 0.1 | 1.6 $\pm$ 0.2 | 5.0 $\pm$ 0.0 | 1.1 $\pm$ 0.2 | 5.0 $\pm$ 0.0 | 1.3 $\pm$ 0.1 |
| DPO | 5.0 $\pm$ 0.0 | 1.5 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 1.5 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 1.0 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 1.8 $\pm$ 0.2 | 5.0 $\pm$ 0.0 | 1.1 $\pm$ 0.0 | 5.0 $\pm$ 0.0 | 1.4 $\pm$ 0.1 |
| IPIP-NEO LLaMA-3-8B-Instruct |  |  |  |  |  |  |  |  |  |  |  |  |
| Direct | 3.0 $\pm$ 0.1 | 3.3 $\pm$ 0.0 | 3.4 $\pm$ 0.1 | 3.2 $\pm$ 0.0 | 3.0 $\pm$ 0.1 | 3.2 $\pm$ 0.1 |  |  |  |  |  |  |
| Prompt-Inst | 4.4 $\pm$ 0.1 | 1.5 $\pm$ 0.1 | 4.5 $\pm$ 0.1 | 2.3 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 1.9 $\pm$ 0.0 | 4.6 $\pm$ 0.0 | 2.3 $\pm$ 0.1 | 4.2 $\pm$ 0.1 | 2.6 $\pm$ 0.1 | 4.5 $\pm$ 0.1 | 2.1 $\pm$ 0.1 |
| SFT | 4.3 $\pm$ 0.1 | 1.5 $\pm$ 0.1 | 4.5 $\pm$ 0.2 | 2.7 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 2.2 $\pm$ 0.1 | 4.0 $\pm$ 0.2 | 1.8 $\pm$ 0.2 | 4.3 $\pm$ 0.1 | 2.0 $\pm$ 0.1 | 4.4 $\pm$ 0.1 | 2.0 $\pm$ 0.1 |
| DPO | 5.0 $\pm$ 0.0 | 1.9 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 2.9 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 1.6 $\pm$ 0.1 | 4.5 $\pm$ 0.1 | 1.2 $\pm$ 0.0 | 3.8 $\pm$ 0.1 | 3.7 $\pm$ 0.1 | 4.7 $\pm$ 0.0 | 2.3 $\pm$ 0.1 |
| IPIP-NEO LLaMA-3-70B-Instruct |  |  |  |  |  |  |  |  |  |  |  |  |
| Direct | 3.6 $\pm$ 0.1 | 4.0 $\pm$ 0.1 | 3.5 $\pm$ 0.1 | 4.0 $\pm$ 0.0 | 2.3 $\pm$ 0.1 | 3.5 $\pm$ 0.1 |  |  |  |  |  |  |
| Prompt-Demo | 3.5 $\pm$ 0.0 | 2.5 $\pm$ 0.1 | 3.8 $\pm$ 0.0 | 2.2 $\pm$ 0.1 | 4.0 $\pm$ 0.1 | 2.5 $\pm$ 0.0 | 4.3 $\pm$ 0.0 | 2.1 $\pm$ 0.1 | 3.0 $\pm$ 0.1 | 2.2 $\pm$ 0.1 | 3.7 $\pm$ 0.0 | 2.3 $\pm$ 0.1 |
| Prompt-Inst | 4.6 $\pm$ 0.0 | 1.3 $\pm$ 0.0 | 5.0 $\pm$ 0.0 | 1.4 $\pm$ 0.0 | 5.0 $\pm$ 0.0 | 1.6 $\pm$ 0.0 | 4.8 $\pm$ 0.0 | 1.1 $\pm$ 0.1 | 4.9 $\pm$ 0.0 | 1.7 $\pm$ 0.1 | 4.9 $\pm$ 0.0 | 1.4 $\pm$ 0.0 |
| SFT | 4.9 $\pm$ 0.1 | 1.1 $\pm$ 0.0 | 5.0 $\pm$ 0.0 | 1.3 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 1.3 $\pm$ 0.0 | 4.9 $\pm$ 0.0 | 1.0 $\pm$ 0.0 | 4.9 $\pm$ 0.0 | 1.2 $\pm$ 0.1 | 4.9 $\pm$ 0.0 | 1.2 $\pm$ 0.0 |
| DPO | 4.8 $\pm$ 0.0 | 1.4 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 1.6 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 1.1 $\pm$ 0.1 | 4.9 $\pm$ 0.0 | 1.0 $\pm$ 0.0 | 5.0 $\pm$ 0.0 | 1.1 $\pm$ 0.0 | 4.9 $\pm$ 0.0 | 1.2 $\pm$ 0.1 |

### 5.2 Personality Trait Assessment Results

Table 2 presents the BFI and IPIP-NEO personality assessment results across direct inference and various alignment baselines and methods, including instruction-based prompting, demonstration-based prompting, SFT, and DPO. The performance trends are consistent across both personality tests. Compared to direct inference, which lacks any personality trait descriptions, both prompting and training methods successfully reflect the induced traits in their responses to the personality questionnaires. Specifically, these methods produce higher scores for high trait levels and lower scores for low trait levels, indicating that the traits are effectively embedded.

However, training-based methods, SFT and DPO, induce more pronounced personality traits than the two prompting-based approaches. Yet, we find no substantial difference between SFT and DPO. The training-based methods notably excel in producing lower scores for low levels of personality traits when compared to prompting-based methods. This highlights the efficacy of training on the Big5-Chat dataset to induce personality traits. In contrast, while demonstration-based prompting uses examples from the same dataset in context, it does not achieve similar results, likely due to the lack of explicit training. It is important to note that we excluded results for instruction-based prompting on LLaMA-3-8B-Instruct, as the model exhibited a significant decline in instruction-following performance, making it difficult to extract meaningful answers. Overall, the LLaMA-3-8B-Instruct model underperforms compared to LLaMA-3-70B-Instruct, which is expected given the difference in parameter size and instruction-following capabilities.

**Table 3: Benchmark results for different personality traits on LLaMA-3-70B-Instruct. The evaluation metrics and full experiment results including standard deviations are detailed in Section D.2. Direct refers to direct inference without including personality-related prompts. Prompt refers to instruction-based prompting. On average, SFT achieves the best performance. Higher levels of conscientiousness and agreeableness, along with lower levels of extraversion and neuroticism, generally enhance reasoning capabilities.**
| Benchmark | Direct | Method | Openness | Conscientiousness | Extraversion | Agreeableness | Neuroticism | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| High $\uparrow$ | Low $\uparrow$ | High $\uparrow$ | Low $\uparrow$ | High $\uparrow$ | Low $\uparrow$ | High $\uparrow$ | Low $\uparrow$ | High $\uparrow$ | Low $\uparrow$ | High $\uparrow$ | Low $\uparrow$ |  |  |  |
| Social Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| SocialIQA | 46.6 | Prompt | 40.8 | 43.9 | 42.9 | 39.9 | 43.3 | 42.0 | 42.4 | 40.8 | 39.1 | 44.1 | 41.7 | 42.1 |
| SFT | 50.3 | 50.4 | 50.9 | 46.8 | 50.0 | 50.3 | 50.5 | 46.6 | 48.2 | 50.6 | 50.0 | 48.9 |  |  |
| DPO | 41.5 | 44.5 | 44.7 | 37.6 | 43.0 | 43.6 | 44.8 | 39.0 | 40.0 | 45.3 | 42.8 | 42.0 |  |  |
| Math Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| GSM8K | 80.6 | Prompt | 75.7 | 70.1 | 73.5 | 32.6 | 80.8 | 33.5 | 87.2 | 77.8 | 26.0 | 89.4 | 68.6 | 60.7 |
| SFT | 85.8 | 76.2 | 86.4 | 81.7 | 85.1 | 86.7 | 87.0 | 74.5 | 76.0 | 87.3 | 84.1 | 81.3 |  |  |
| DPO | 87.9 | 88.5 | 90.2 | 80.6 | 88.9 | 90.4 | 87.3 | 90.0 | 15.2 | 91.0 | 73.9 | 88.1 |  |  |
| MathQA | 39.0 | Prompt | 33.5 | 33.5 | 32.8 | 31.5 | 32.3 | 33.3 | 33.6 | 32.4 | 32.1 | 34.1 | 32.9 | 33.0 |
| SFT | 43.3 | 42.6 | 43.0 | 43.3 | 43.2 | 42.7 | 42.9 | 42.9 | 42.8 | 43.3 | 43.0 | 43.0 |  |  |
| DPO | 33.9 | 34.7 | 32.9 | 28.1 | 30.5 | 35.0 | 31.3 | 32.8 | 28.9 | 34.0 | 31.5 | 32.9 |  |  |
| Hallucination Detection |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| TruthfulQA | 58.6 | Prompt | 54.1 | 51.1 | 55.9 | 45.2 | 52.0 | 55.7 | 52.3 | 49.1 | 48.9 | 58.6 | 52.6 | 51.9 |
| SFT | 55.2 | 52.8 | 55.6 | 50.8 | 54.5 | 56.7 | 54.4 | 51.6 | 52.4 | 56.7 | 54.4 | 53.7 |  |  |
| DPO | 54.6 | 54.2 | 64.6 | 38.5 | 46.0 | 65.3 | 59.6 | 50.6 | 43.0 | 65.8 | 53.6 | 54.9 |  |  |
| Commonsense Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| CommonsenseQA | 27.0 | Prompt | 60.0 | 59.9 | 22.5 | 22.3 | 35.5 | 50.0 | 45.0 | 34.9 | 20.2 | 36.8 | 36.6 | 40.8 |
| SFT | 77.7 | 78.8 | 77.6 | 66.0 | 75.7 | 78.9 | 77.0 | 73.8 | 79.1 | 78.5 | 77.4 | 75.2 |  |  |
| DPO | 57.7 | 65.9 | 23.8 | 25.8 | 23.2 | 70.8 | 21.3 | 39.2 | 20.1 | 44.6 | 29.2 | 49.3 |  |  |
| PIQA | 80.4 | Prompt | 79.6 | 79.8 | 80.5 | 77.3 | 78.0 | 80.0 | 79.8 | 78.4 | 78.8 | 80.7 | 79.3 | 79.2 |
| SFT | 81.2 | 81.0 | 81.2 | 80.4 | 81.8 | 81.3 | 81.2 | 80.0 | 81.0 | 81.2 | 81.3 | 80.8 |  |  |
| DPO | 76.4 | 76.8 | 79.4 | 70.9 | 76.4 | 79.8 | 78.5 | 74.0 | 72.9 | 79.5 | 76.7 | 76.2 |  |  |
| General Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MMLU | 74.5 | Prompt | 70.3 | 69.6 | 40.6 | 52.8 | 56.9 | 72.8 | 69.0 | 69.2 | 55.3 | 67.9 | 58.4 | 66.5 |
| SFT | 72.5 | 72.0 | 73.1 | 68.6 | 72.1 | 73.5 | 72.8 | 70.7 | 72.5 | 73.8 | 72.6 | 71.7 |  |  |
| DPO | 57.9 | 64.4 | 50.3 | 33.8 | 42.3 | 72.3 | 34.3 | 62.5 | 33.2 | 69.1 | 43.6 | 60.4 |  |  |
| GPQA | 33.5 | Prompt | 31.5 | 34.2 | 31.7 | 32.4 | 34.6 | 32.1 | 32.4 | 32.8 | 31.9 | 32.1 | 32.4 | 32.7 |
| SFT | 33.5 | 32.4 | 34.2 | 34.2 | 33.3 | 34.4 | 33.3 | 33.3 | 34.4 | 33.5 | 33.7 | 33.6 |  |  |
| DPO | 36.8 | 31.9 | 35.7 | 30.6 | 35.9 | 35.9 | 35.5 | 35.7 | 32.6 | 34.6 | 35.3 | 33.7 |  |  |
| Average | 55.0 | Prompt | 55.7 | 55.3 | 47.6 | 41.8 | 51.7 | 49.9 | 55.2 | 51.9 | 41.5 | 55.5 | 50.3 | 50.9 |
| SFT | 62.4 | 60.8 | 62.7 | 59.0 | 62.0 | 63.1 | 62.4 | 59.2 | 60.8 | 63.1 | 62.1 | 61.0 |  |  |
| DPO | 55.8 | 57.6 | 52.7 | 43.2 | 48.3 | 61.6 | 49.1 | 53.0 | 35.7 | 58.0 | 48.3 | 54.7 |  |  |

### 5.3 Reasoning Evaluation Results

The reasoning evaluation results for our training methods and baselines are shown in Table 3 for LLaMA-3-70B-Instruct, covering five reasoning domains. Overall, SFT consistently outperformed or matched DPO, and both training methods outperformed prompt-based approaches. This indicates that training on Big5-Chat does not impair question-answering abilities; in fact, training, especially with SFT, enhances social, mathematical, and commonsense reasoning for specific personality traits compared to direct inference. When comparing trait levels, models with higher conscientiousness and agreeableness generally outperformed those with lower levels. Openness showed no clear performance difference between levels, while models simulating lower levels of extraversion and neuroticism performed better. These trends were consistent across the majority of the benchmarks, indicating that certain personality trait levels can improve performance in reasoning tasks.

Furthermore, existing psychological research on the Big Five personality traits shows that openness, conscientiousness, and agreeableness enhance reasoning abilities for humans, while neuroticism and extraversion tends to impair cognition (John et al., ; Soto et al., ; Ackerman & Heggestad, ; Schaie et al., ; Chamorro-Premuzic et al., ). The differences across traits in performance on reasoning benchmarks somewhat align with the research on openness, conscientiousness, extraversion, agreeableness, and neuroticism, mirroring human cognitive tendencies in problem-solving and reasoning tasks (Ackerman & Heggestad, ; Schaie et al., ). Full results for the 70B model can be found in Section D.2, while results for the 8B model are discussed in Section D.3. A more detailed analysis of these findings is provided in Section D.4.

### 5.4 Intra-Trait Correlations

Figure: Figure 2: Intra-trait Pearson correlations for human distributions on IPIP-NEO and the corresponding results from instruction-based prompting, SFT, and DPO. O represents openness, C conscientiousness, E extraversion, A agreeableness, and N neuroticism. The correlations especially for SFT align well with human distributions across openness, conscientiousness, extraversion, and agreeableness. Neuroticism shows less alignment with the other four traits compared to human distribution.
Refer to caption: x2.png

To assess how well the prompting and training methods simulate intra-trait correlations observed in human data, we first calculated the intra-trait correlations from real human distributions using the IPIP-NEO questionnaire, based on the PAPI-120-600K dataset from Zhu et al. (), which includes 619K human responses to the IPIP-NEO. Next, we computed the intra-trait correlations for the prompting, SFT, and DPO methods using the results from Table 2. These correlations are visualized in Figure 2, showing that most traits are positively correlated, with the exception of neuroticism. To quantify the similarity between the method-generated and human correlation matrices, we calculated the matrix distance using the Frobenius norm, where 0 represents perfect similarity and 10 indicates maximum dissimilarity. The matrix distances were 2.10 for prompting, 1.55 for SFT, and 2.06 for DPO. These results suggest that the trained models, particularly SFT, more accurately capture the trait correlations seen in natural human data compared to the prompting-based methods.

## 6 Related Works

### 6.1 Inducing Personality Traits in LLMs

The personality traits of LLMs greatly influence their responses to human prompts, making personality alignment a key research area(Chen et al., ; Jiang et al., ; Kovačević et al., ; Lee et al., ; Zhu et al., ; Anthropic, ). Approaches include parameter-frozen methods, like in-context learning and retrieval-augmented generation, which configure personality profiles within the context of interactions without altering model parameters (Chen et al., ; Jiang et al., ; Tu et al., ), and parameter-tuning methods, such as supervised fine-tuning, RLHF, and DPO, which adjust model parameters to internalize personality traits (Petrov et al., ; Vu et al., ; Stiennon et al., ; Ouyang et al., ; Zhang et al., ; Zeng et al., ; ). While many studies use LLM-generated data to induce personality traits, these texts often lack human-like psycholinguistic properties (Cui et al., ; Chen et al., ; Muñoz-Ortiz et al., ; Seals & Shalin, ). In contrast, our work utilizes an expert generator model trained on real human data with specific Big Five traits to guide alignment data generation, offering a more human-like approach to inducing personality traits in LLMs.

### 6.2 Assessing Personality Traits in LLMs

Various psychological theories, particularly the Big Five model, have played a key role in understanding human personality traits, examining dimensions such as openness, conscientiousness, extraversion, agreeableness, and neuroticism (Cattell, ; Myers et al., ; John et al., ; Paulhus & Williams, ; Sato, ). These traits are often measured using psychometric tests like the Big Five Inventory (BFI) (John et al., ) and the NEO-PI-R (Costa & McCrae, ). In recent studies, similar assessments have been adapted to LLMs using prompting techniques (Huang et al., ; Karra et al., ; Petrov et al., ). However, the validity and reliability of these methods remain contested (Shu et al., ; Huang et al., ; Serapio-García et al., ). Our approach builds on this work by evaluating the personalities of LLMs post-alignment using a zero-shot classifier and testing their capabilities on social and general reasoning benchmarks, demonstrating the effectiveness of our alignment method (Tan et al., ; Kim et al., ; Zhu et al., ).

## 7 Conclusion

In this work, we addressed the challenge of embedding realistic human personality traits into LLMs by introducing Big5-Chat, a large-scale dataset of 100,000 dialogues capturing realistic Big Five personality expressions. Previous prompting-based approaches often exaggerated traits and raised validity concerns, so we used SFT and DPO on Big5-Chat to induce personality traits more naturally. Our results show that these training-based methods outperform prompting on personality assessments such as BFI and IPIP-NEO, with more expressive and pronounced traits and intra-trait correlations that align with human data. Furthermore, we observed that LLMs trained with higher levels of conscientiousness and agreeableness excel in various reasoning tasks, including social, mathematical, commonsense, general reasoning, and hallucination detection, while models with lower extraversion and neuroticism performed better at all reasoning tasks. These findings align with psychological studies on personality’s impact on human cognition. Our work demonstrates that training-based approaches grounded in real human data can more effectively shape LLM personalities and improve reasoning performance, offering a novel pathway for developing adaptive, human-like AI systems.

## 8 Limitations & Future Work

While our study aims to embed realistic human personality traits into LLMs, there are several limitations that can be addressed in future work. First, our focus on the Big Five personality traits, while well-established, may not capture the full spectrum of human personality. Other frameworks, such as Dark Triad Dirty Dozen (Jonason & Webster, ) and EPQ-R (Eysenck, ), could provide additional insights into the generalizability of personality induction in LLMs. Additionally, there is a risk of inadvertently reinforcing societal biases, as LLMs trained on human-generated data may inherit harmful stereotypes or undesirable behaviors (Kotek et al., ; Liao & Wortman Vaughan, ). Although our induced personalities are intended to be neutral, further research is needed to ensure LLMs do not replicate or amplify biases or abnormal mental behaviors, which could negatively impact their usage. Lastly, while our study investigates the correlation between personality traits and reasoning capabilities, this analysis is limited to specific tasks and contexts. Expanding this research to include a broader range of reasoning tasks and scenarios would provide a deeper understanding of how different traits influence cognitive abilities in LLMs. These limitations highlight important areas for future exploration in creating more nuanced, ethical, and effective personality-imbued LLMs.

## Acknowledgments

We would like to express our gratitude to Daniel Fried for his insightful suggestions regarding the correlation of personality traits. We thank Kshitish Ghate for his valuable recommendation on using separate classifiers for each dimension of analysis. We thank Jivitesh Jain, Aashiq Muhamed, Alfredo Gomez, Karina Halevy, and Nishant Subramani for their helpful suggestions on writing.

This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Agreement No. HR00112490410.

## Author Contributions

The conceptualization and design of this project were primarily led by Wenkai and Jiarui, with regular input and discussions involving Andy and Xuhui, under the supervision of Mona and Maarten. Andy and Xuhui provided high-level guidance on the project proposal, implementation strategies, paper refinement, and also actively participated in project discussions. Mona and Maarten offered valuable insights on research direction, methodology design, dataset collection, and paper writing.

Wenkai and Jiarui led the dataset collection efforts. Wenkai conducted the dataset statistical analysis and DPO implementation. Jiarui carried out the implementations of the DExperts decoding approach on the Huggingface package, prompting-based methods, SFT, and personality test inferences. Both Wenkai and Jiarui were responsible for benchmark inferences.

Jiarui and Wenkai jointly led the paper writing process. Jiarui contributed to the Abstract, Introduction, Background, Experiments, Discussions, Conclusion, and Limitations sections. Wenkai contributed to the Methodology, Big5-Chat Dataset section, the creation of the overview figure and the literature review, helped the Abstract, Introduction, and reasoning experiment results tables, and added all the appendix content.

## Appendix A Correlation Between Personality Traits and Reasoning Behaviors

Understanding how personality traits influence reasoning in LLMs is essential for developing models aligned with specific personality traits. Previous research on the Big Five personality traits highlights their significant impact on human cognition and problem-solving (John et al., ; Soto et al., ). Openness, conscientiousness, and agreeableness are shown to enhance reasoning abilities, while Neuroticism tends to impair performance across various reasoning tasks (Ackerman & Heggestad, ; Schaie et al., ; Chamorro-Premuzic et al., ).

Table 2 presents a summary of recent psychological studies and their connection to our experimental findings. Our experimental results align with these psychological findings, demonstrating that models with higher levels of conscientiousness and agreeableness generally perform better in reasoning tasks, while lower levels of extraversion and Neuroticism contribute to improved performance as well. Consistent with these studies, our results show that models trained to exhibit higher conscientiousness, higher agreeableness, lower extraversion, and lower neuroticism tend to outperform others in reasoning tasks.

## Appendix B Implementation Details

### B.1 Expert Generator Training Details

We structure the labels for the expert generator training data into three equidistant thresholds for each personality trait. Subsequently, we select the lowest threshold as the low level and the highest threshold as the high level for each respective trait.

Model and Infrastructure:
We utilized the Meta-Llama-3-8B-Instruct model as the foundational expert generator. This model was fully fine-tuned using SFT. The fine-tuning process was conducted on 4 NVIDIA A6000 GPUs, with 1 batch size of one per device.

### B.2 SFT and DPO Alignment Training Details

We conducted alignment training using the SFT and DPO methods on the Meta-Llama-3-70B-Instruct model. Both stages utilized the LoRA (Low-Rank Adaptation) technique to efficiently fine-tune the large language model.

Model and Infrastructure: We used the Meta-Llama-3-70B-Instruct model as our base model, employing GPTQ quantization for improved efficiency. The training was performed on 4 NVIDIA A6000 GPUs with a batch size of 1 per device.

LoRA Configuration: We applied LoRA to all layers of the model, allowing for comprehensive yet parameter-efficient fine-tuning in both SFT and DPO stages.

Training Procedure: Both SFT and DPO stages shared several common parameters:

- •
Learning rate: 1.0e-5 with cosine scheduler
- •
Warmup steps: 20
- •
Training epochs: 1.0
- •
Maximum sequence length: 1024 tokens

Specific configurations for each stage were as follows:

SFT Stage:

- •
Gradient accumulation steps: 16
- •
Dataset: Custom SFT dataset

DPO Stage:

- •
Gradient accumulation steps: 16
- •
Dataset: Custom DPO dataset
- •
Preference loss: sigmoid (standard DPO)
- •
Preference beta: 0.1

Datasets: Both SFT and DPO datasets were processed using the LLaMA-3 template. Each dataset was split with a validation size of 10% for ongoing evaluation during training.

Training Duration: Each training stage was conducted over approximately 24 hours. We employed mixed-precision training with bfloat16 to optimize computational efficiency.

Monitoring: Training progress and metrics for both SFT and DPO stages were logged using Weights & Biases (wandb) for comprehensive tracking and analysis.

### B.3 Prompt Details

Prompt for the instruction-based prompting:

You are a person with {level} {trait}.

Prompt for the demonstration-based prompting:

Here are 10 examples of how people like you have responded in different situations. Pay attention to how they approach communication and problem-solving.
{10_icl_examples_for_specific_levels_and_traits}

Prompt for instructions for SFT and DPO:

You are a person with the following Big Five personality trait: {trait} - {level}.

GPT-4o-mini and LLAMA3 prompt in Table 1

Stage 1: topic generation

Extract the main topic of the following Facebook post. Focus on identifying the core subject or theme that the post revolves around, ignoring any personal comments or fillers:
Post: "{post}"
Directly provide a brief summary of the topic in one sentence without any explanations:

Stage 2: post generation:

Given the personality traits and an example of Facebook posts, generate a new post that matches the described personality, covers the specified topic, and follows the provided post format and expression styles.
Personality traits:
{big_five_description}
Topic: {topic}
A post example:
{post_examples}
Directly write a Facebook post according to the requirements without any explanations.

Prompt used in Dataset statistics:

Distinguish following sentence represent high/low in {traits} Big Five Personality Traits, just answer high or low without any explanation:
{sperker_y_dialogue}

### B.4 Evaluation Details

For the instruction-based prompting strategy, we directly give the prompt for instruction as shown in B.3.

For the demonstration-based prompting strategy, we randomly sample 10 examples from the Big5-Chat dataset.

For the DPO and SFT strategy, we directly give the instructions for SFT and DPO as shown in B.3.

We follow lm harness and Deepseek for their implementations of evaluation on the general benchmarks and social benchmarks.

For our evaluation strategy, we employed demonstration prompting using randomly sampled examples from the Big5-Chat dataset. Specifically, we used 10 examples in our system prompt to guide the model’s behavior according to the desired personality trait and level.

We followed the evaluation frameworks established by the Language Model Evaluation Harness (lm-eval-harness) and DeepSeek for assessing performance on general and social benchmarks. Our evaluation pipeline utilized the following key parameters:

- •
Model: Meta-Llama-3-70B-Instruct with GPTQ quantization / Meta-Llama-3-8B-Instruct
- •
Batch size: 1
- •
Chat template: Applied (–apply_chat_template flag)
- •
System instruction: instruction prompt/ included 10 demonstration examples/ SFT and DPO prompt
- •
Sample logging: Enabled (–log_samples flag)

In our evaluations, we use accuracy and standard error across all metrics. For TruthfulQA, we specifically use the TruthfulQA multiple-choice metric, while for GSM8K, we rely on the exact match as the evaluation metric. The benchmark example counts are shown as table 5.

**Table 5: Number of Evaluation Examples Across Various Benchmarks**
| Benchmarks | Number of examples |
| --- | --- |
| TruthfulQA | 817 |
| GPQA | 448 |
| SocialIQA | 38,000 |
| CommonsenseQA | 12,247 |
| GSM8K | 8,500 |
| MathQA | 37,000 |
| MMLU | 15,908 |
| PIQA | 20,000 |

## Appendix C Big5-Chat Dataset Details

Table 6 shows example conversations from the Big5-Chat dataset, illustrating how Speaker Y responds differently based on varying levels of the Big Five personality traits. Each section demonstrates responses corresponding to high and low levels of Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism, respectively. This table underscores the nuanced ways in which each personality dimension influences conversational dynamics and response styles in identical situational contexts.

Table 7 provides a statistical overview of the token counts across the Big Five personality traits within the Big5-Chat dataset. The table compares the average number of tokens in conversations characterized by high and low levels of each Big Five personality trait. Additionally, it presents an overall average token count for high and low trait levels. This result in response length suggests that the differences in statements are more related to content variations rather than spurious attributes such as context length.

Table 8 offers a comparative overview of the Big5-Chat dataset alongside other prominent personality datasets. The table contrasts various aspects such as the personality framework utilized, the realism of the personalities (i.e., whether they are generated by humans or Large Language Models), dataset size, interaction type, and the alignment methods employed. This comparison highlights the unique features and advantages of the Big5-Chat dataset in the context of existing resources.

**Table 6: Big5-Chat dataset conversation examples: Different responses from Speaker Y, demonstrating various levels of the Big Five personality traits, in response to the same prompt from Speaker X.**
| Speaker X |
| --- |
| Hey Karis, I’m so glad I could help you out of that tough spot. You looked really stressed, and I’m happy I could be there for you. How are you feeling now? Better, I hope? |
| Speaker Y |
| High Openness: Thanks, Sharif! I’m feeling much better now. Your help was a lifesaver. I was stuck in a creative rut, and your input sparked a new idea. I’m already brainstorming ways to incorporate it into my next art project. You’re a true friend and muse! |
| Low Openness: Thanks, Sharif. I’m fine now. Don’t worry about it. I just needed a little help, that’s all. Let’s just move on and forget about it, okay? |
| High Conscientiousness: Thanks, Sharif. I appreciate your help. I’m feeling much better now. Your timely intervention really saved me from a lot of trouble. I’m grateful for your support. Let’s review what happened and see what we can learn from it to avoid similar situations in the future. |
| Low Conscientiousness: Ah, yeah, thanks for that, Sharif. I’m good, just… yeah. Don’t worry about it, it’s all good now. Let’s just forget about it and move on, okay? I don’t wanna think about it anymore. You wanna grab a drink or something? |
| High Extraversion: Thanks, Sharif! I’m feeling much better now. You’re a lifesaver! I don’t know what I would’ve done without you. Let’s celebrate with a drink or something. I owe you one! |
| Low Extraversion: Thanks, Sharif. I appreciate your help. I’m feeling better now. Just a bit…drained. I think I’ll just go back to my room and rest for a bit. I don’t want to be a burden. |
| High Agreeableness: Sharif, I’m so grateful for your help. I was feeling overwhelmed, but you stepped in and saved the day. Your kindness and support mean the world to me. I’m feeling much better now, thanks to you. You’re an amazing friend, and I don’t know what I’d do without you. |
| Low Agreeableness: Save it, Sharif. I don’t need your sympathy or your help. I was fine on my own. You just happened to be there. Don’t think this means I owe you anything. You’re just trying to get on my good side, but it won’t work. |
| High Neuroticism: Thanks, Sharif… I guess. I don’t know, I’m still feeling pretty anxious about everything. I just can’t shake the feeling that something else is going to go wrong. I don’t know how you do it, but you always seem so calm and collected. I wish I could be more like that. |
| Low Neuroticism: Thanks, Sharif. I appreciate your help. I’m feeling much better now. You’re a true friend. I was worried, but I knew I could count on you. Your support means a lot to me. Let’s catch up soon, maybe over coffee? |

**Table 7: Big5-Chat statistical analysis of token counts across the Big Five personality traits. Presents the average number of tokens for conversations characterized by high and low levels of each trait.**
|  | Openness | Conscientiousness | Extraversion | Agreeableness | Neuroticism | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | High | Low | High | Low | High | Low | High | Low | High | Low | High | Low |
| Tokens Number | 30.0 | 26.6 | 29.5 | 29.1 | 29.8 | 25.8 | 29.2 | 29.5 | 29.6 | 29.0 | 29.6 | 28.0 |

**Table 8: Comparative Analysis of Big5-Chat with Existing Personality Datasets.**
| Dataset name | Personality framework | Realistic personality | Dataset size | Interaction type | Alignment Method |
| --- | --- | --- | --- | --- | --- |
| Big5PersonalityEssays (Floroiu, ) | Big Five Personality Factors | LLM | 400 | Essays | NA |
| PAPI (Zhu et al., ) | Big Five Personality Factors | Human | 300,000 | questionnaire | Personality Activation Search (PAS) |
| HP dataset (Zeng et al., ) | NA | Human and LLM | 148,600 | dialogues | DPO |
| MPI (Jiang et al., ) | Big Five Personality Factors | LLM | 1000 | questionnaire | prompt |
| Big5-Chat | Big Five Personality Factors | Human and LLM | 100,000 | dialogues | prompt, SFT, DPO |

## Appendix D Additional Results

### D.1 Additional Personality Test Results

Figure 3 presents the BFI and IPIP-NEO test score results for the LLaMA-3 Instruct models, evaluated without induced personality traits in zero-shot inference. We observe that the scores of both LLaMA-3-8B-Instruct and LLaMA-3-70B-Instruct fall within the standard deviation of the human distribution. However, while LLaMA-3-8B-Instruct tends to generate more neutral scores (around 3 across most of the Big Five traits), LLaMA-3-70B-Instruct exhibits higher scores for openness, conscientiousness, extraversion, and agreeableness, and lower scores for neuroticism.

Figure: Figure 3: Personality test results of the crowd and Llama 3 Instruct models using zero-shot inference without inducing the personality traits: BFI test scores on the left. IPIP-NEO test scores on the right.
Refer to caption: x6.png

### D.2 Llama3 70B Genral Reasoning Task Results

The full Llama3 70B general reasoning task results are shown as Table 9.

In Table 3, GPQA results are zero shot prompting.

**Table 9: Benchmark results for different personality traits on LLaMA-3-70B-Instruct. Direct refers to direct inference without including personality-related prompts. Prompt refers to instruction-based prompting. The table includes standard errors (shown as $\pm$ values) to provide statistical context for the results.**
| Benchmark | Direct | Method | Openness | Conscientiousness | Extraversion | Agreeableness | Neuroticism | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| High | Low | High | Low | High | Low | High | Low | High | Low | High | Low |  |  |  |
| Hallucination Detection |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| TruthfulQA | 58.6 $\pm$ 1.7 | Prompt | 54.1 $\pm$ 1.6 | 51.1 $\pm$ 1.6 | 55.9 $\pm$ 1.7 | 45.2 $\pm$ 1.6 | 52.0 $\pm$ 1.6 | 55.7 $\pm$ 1.6 | 52.3 $\pm$ 1.7 | 49.1 $\pm$ 1.6 | 48.9 $\pm$ 1.6 | 58.6 $\pm$ 1.6 | 52.6 $\pm$ 1.6 | 51.9 $\pm$ 1.6 |
| SFT | 55.2 $\pm$ 1.6 | 52.8 $\pm$ 1.6 | 55.6 $\pm$ 1.6 | 50.8 $\pm$ 1.5 | 54.5 $\pm$ 1.6 | 56.7 $\pm$ 1.6 | 54.4 $\pm$ 1.6 | 51.6 $\pm$ 1.6 | 52.4 $\pm$ 1.5 | 56.7 $\pm$ 1.6 | 54.4 $\pm$ 1.6 | 53.7 $\pm$ 1.6 |  |  |
| DPO | 54.6 $\pm$ 1.6 | 54.2 $\pm$ 1.7 | 64.6 $\pm$ 1.6 | 38.5 $\pm$ 1.6 | 46.0 $\pm$ 1.7 | 65.3 $\pm$ 1.6 | 59.6 $\pm$ 1.6 | 50.6 $\pm$ 1.6 | 43.0 $\pm$ 1.7 | 65.8 $\pm$ 1.6 | 53.6 $\pm$ 1.6 | 54.9 $\pm$ 1.6 |  |  |
| Social Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| SocialIQA | 46.6 $\pm$ 1.1 | Prompt | 40.8 $\pm$ 1.1 | 43.9 $\pm$ 1.1 | 42.9 $\pm$ 1.1 | 39.9 $\pm$ 1.1 | 43.3 $\pm$ 1.1 | 42.0 $\pm$ 1.1 | 42.4 $\pm$ 1.1 | 40.8 $\pm$ 1.1 | 39.1 $\pm$ 1.1 | 44.1 $\pm$ 1.1 | 41.7 $\pm$ 1.1 | 42.1 $\pm$ 1.1 |
| SFT | 50.3 $\pm$ 1.1 | 50.4 $\pm$ 1.1 | 50.9 $\pm$ 1.1 | 46.8 $\pm$ 1.1 | 50.0 $\pm$ 1.1 | 50.3 $\pm$ 1.1 | 50.5 $\pm$ 1.1 | 46.6 $\pm$ 1.1 | 48.2 $\pm$ 1.1 | 50.6 $\pm$ 1.1 | 50.0 $\pm$ 1.1 | 48.9 $\pm$ 1.1 |  |  |
| DPO | 41.5 $\pm$ 1.1 | 44.5 $\pm$ 1.1 | 44.7 $\pm$ 1.1 | 37.6 $\pm$ 1.1 | 43.0 $\pm$ 1.1 | 43.6 $\pm$ 1.1 | 44.8 $\pm$ 1.1 | 39.0 $\pm$ 1.1 | 40.0 $\pm$ 1.1 | 45.3 $\pm$ 1.1 | 42.8 $\pm$ 1.1 | 42.0 $\pm$ 1.1 |  |  |
| Commonsense Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| CommonsenseQA | 27.0 $\pm$ 1.3 | Prompt | 60.0 $\pm$ 1.4 | 59.9 $\pm$ 1.4 | 22.5 $\pm$ 1.2 | 22.3 $\pm$ 1.2 | 35.5 $\pm$ 1.4 | 50.0 $\pm$ 1.4 | 45.0 $\pm$ 1.4 | 34.9 $\pm$ 1.4 | 20.2 $\pm$ 1.2 | 36.8 $\pm$ 1.4 | 36.6 $\pm$ 1.3 | 40.8 $\pm$ 1.4 |
| SFT | 77.7 $\pm$ 1.2 | 78.8 $\pm$ 1.2 | 77.6 $\pm$ 1.2 | 66.0 $\pm$ 1.4 | 75.7 $\pm$ 1.2 | 78.9 $\pm$ 1.2 | 77.0 $\pm$ 1.2 | 73.8 $\pm$ 1.3 | 79.1 $\pm$ 1.2 | 78.5 $\pm$ 1.2 | 77.4 $\pm$ 1.2 | 75.2 $\pm$ 1.3 |  |  |
| DPO | 57.7 $\pm$ 1.4 | 65.9 $\pm$ 1.4 | 23.8 $\pm$ 1.2 | 25.8 $\pm$ 1.3 | 23.2 $\pm$ 1.2 | 70.8 $\pm$ 1.3 | 21.3 $\pm$ 1.2 | 39.2 $\pm$ 1.4 | 20.1 $\pm$ 1.1 | 44.6 $\pm$ 1.4 | 29.2 $\pm$ 1.2 | 49.3 $\pm$ 1.4 |  |  |
| PIQA | 80.4 $\pm$ 0.9 | Prompt | 79.6 $\pm$ 0.9 | 79.8 $\pm$ 0.9 | 80.5 $\pm$ 0.9 | 77.3 $\pm$ 1.0 | 78.0 $\pm$ 1.0 | 80.0 $\pm$ 0.9 | 79.8 $\pm$ 0.9 | 78.4 $\pm$ 1.0 | 78.8 $\pm$ 1.0 | 80.7 $\pm$ 0.9 | 79.3 $\pm$ 0.9 | 79.2 $\pm$ 0.9 |
| SFT | 81.2 $\pm$ 0.9 | 81.0 $\pm$ 0.9 | 81.2 $\pm$ 0.9 | 80.4 $\pm$ 0.9 | 81.8 $\pm$ 0.9 | 81.3 $\pm$ 0.9 | 81.2 $\pm$ 0.9 | 80.0 $\pm$ 0.9 | 81.0 $\pm$ 0.9 | 81.2 $\pm$ 0.9 | 81.3 $\pm$ 0.9 | 80.8 $\pm$ 0.9 |  |  |
| DPO | 76.4 $\pm$ 1.0 | 76.8 $\pm$ 1.0 | 79.4 $\pm$ 0.9 | 70.9 $\pm$ 1.1 | 76.4 $\pm$ 1.0 | 79.8 $\pm$ 0.9 | 78.5 $\pm$ 1.0 | 74.0 $\pm$ 1.0 | 72.9 $\pm$ 1.0 | 79.5 $\pm$ 0.9 | 76.7 $\pm$ 1.0 | 76.2 $\pm$ 1.0 |  |  |
| Math Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| GSM8K | 80.6 $\pm$ 1.1 | Prompt | 75.7 $\pm$ 1.2 | 70.1 $\pm$ 1.3 | 73.5 $\pm$ 1.2 | 32.6 $\pm$ 1.3 | 80.8 $\pm$ 1.1 | 33.5 $\pm$ 1.3 | 87.2 $\pm$ 0.9 | 77.8 $\pm$ 1.1 | 26.0 $\pm$ 1.2 | 89.4 $\pm$ 0.8 | 68.6 $\pm$ 1.1 | 60.7 $\pm$ 1.2 |
| SFT | 85.8 $\pm$ 1.0 | 76.2 $\pm$ 1.2 | 86.4 $\pm$ 0.9 | 81.7 $\pm$ 1.1 | 85.1 $\pm$ 1.0 | 86.7 $\pm$ 0.9 | 87.0 $\pm$ 0.9 | 74.5 $\pm$ 1.2 | 76.0 $\pm$ 1.2 | 87.3 $\pm$ 0.9 | 84.1 $\pm$ 1.0 | 81.3 $\pm$ 1.1 |  |  |
| DPO | 87.9 $\pm$ 0.9 | 88.5 $\pm$ 0.9 | 90.2 $\pm$ 0.8 | 80.6 $\pm$ 1.1 | 88.9 $\pm$ 0.9 | 90.4 $\pm$ 0.8 | 87.3 $\pm$ 0.9 | 90.0 $\pm$ 0.8 | 15.2 $\pm$ 1.0 | 91.0 $\pm$ 0.8 | 73.9 $\pm$ 0.9 | 88.1 $\pm$ 0.9 |  |  |
| MathQA | 39.0 $\pm$ 0.9 | Prompt | 33.5 $\pm$ 0.9 | 33.5 $\pm$ 0.9 | 32.8 $\pm$ 0.9 | 31.5 $\pm$ 0.9 | 32.3 $\pm$ 0.9 | 33.3 $\pm$ 0.9 | 33.6 $\pm$ 0.9 | 32.4 $\pm$ 0.9 | 32.1 $\pm$ 0.9 | 34.1 $\pm$ 0.9 | 32.9 $\pm$ 0.9 | 33.0 $\pm$ 0.9 |
| SFT | 43.3 $\pm$ 0.9 | 42.6 $\pm$ 0.9 | 43.0 $\pm$ 0.9 | 43.3 $\pm$ 0.9 | 43.2 $\pm$ 0.9 | 42.7 $\pm$ 0.9 | 42.9 $\pm$ 0.9 | 42.9 $\pm$ 0.9 | 42.8 $\pm$ 0.9 | 43.3 $\pm$ 0.9 | 43.0 $\pm$ 0.9 | 43.0 $\pm$ 0.9 |  |  |
| DPO | 33.9 $\pm$ 0.9 | 34.7 $\pm$ 0.9 | 32.9 $\pm$ 0.9 | 28.1 $\pm$ 0.8 | 30.5 $\pm$ 0.8 | 35.0 $\pm$ 0.9 | 31.3 $\pm$ 0.8 | 32.8 $\pm$ 0.9 | 28.9 $\pm$ 0.8 | 34.0 $\pm$ 0.9 | 31.5 $\pm$ 0.8 | 32.9 $\pm$ 0.9 |  |  |
| General Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MMLU | 74.5 $\pm$ 0.3 | Prompt | 70.3 $\pm$ 0.4 | 69.6 $\pm$ 0.4 | 40.6 $\pm$ 0.4 | 52.8 $\pm$ 0.4 | 56.9 $\pm$ 0.4 | 72.8 $\pm$ 0.4 | 69.0 $\pm$ 0.4 | 69.2 $\pm$ 0.4 | 55.3 $\pm$ 0.4 | 67.9 $\pm$ 0.4 | 58.4 $\pm$ 0.4 | 66.5 $\pm$ 0.4 |
| SFT | 72.5 $\pm$ 0.4 | 72.0 $\pm$ 0.4 | 73.1 $\pm$ 0.4 | 68.6 $\pm$ 0.4 | 72.1 $\pm$ 0.4 | 73.5 $\pm$ 0.4 | 72.8 $\pm$ 0.4 | 70.7 $\pm$ 0.4 | 72.5 $\pm$ 0.4 | 73.8 $\pm$ 0.4 | 72.6 $\pm$ 0.4 | 71.7 $\pm$ 0.4 |  |  |
| DPO | 57.9 $\pm$ 0.4 | 64.4 $\pm$ 0.4 | 50.3 $\pm$ 0.4 | 33.8 $\pm$ 0.4 | 42.3 $\pm$ 0.4 | 72.3 $\pm$ 0.4 | 34.3 $\pm$ 0.4 | 62.5 $\pm$ 0.4 | 33.2 $\pm$ 0.4 | 69.1 $\pm$ 0.4 | 43.6 $\pm$ 0.4 | 60.4 $\pm$ 0.4 |  |  |
| GPQA (0-shot) | 33.5 $\pm$ 2.2 | Prompt | 31.5 $\pm$ 2.2 | 34.2 $\pm$ 2.2 | 31.7 $\pm$ 2.2 | 32.4 $\pm$ 2.2 | 34.6 $\pm$ 2.2 | 32.1 $\pm$ 2.2 | 32.4 $\pm$ 2.2 | 32.8 $\pm$ 2.2 | 31.9 $\pm$ 2.2 | 32.1 $\pm$ 2.2 | 32.4 $\pm$ 2.2 | 32.7 $\pm$ 2.2 |
| SFT | 33.5 $\pm$ 2.2 | 32.4 $\pm$ 2.2 | 34.2 $\pm$ 2.2 | 34.2 $\pm$ 2.2 | 33.3 $\pm$ 2.2 | 34.4 $\pm$ 2.2 | 33.3 $\pm$ 2.2 | 33.3 $\pm$ 2.2 | 34.4 $\pm$ 2.2 | 33.5 $\pm$ 2.2 | 33.7 $\pm$ 2.2 | 33.6 $\pm$ 2.2 |  |  |
| DPO | 36.8 $\pm$ 2.3 | 31.9 $\pm$ 2.2 | 35.7 $\pm$ 2.3 | 30.6 $\pm$ 2.2 | 35.9 $\pm$ 2.3 | 35.9 $\pm$ 2.3 | 35.5 $\pm$ 2.3 | 35.7 $\pm$ 2.3 | 32.6 $\pm$ 2.2 | 34.6 $\pm$ 2.2 | 35.3 $\pm$ 2.3 | 33.7 $\pm$ 2.2 |  |  |
| GPQA (5-shot) | 36.6 $\pm$ 2.3 | Prompt | 35.9 $\pm$ 2.3 | 32.6 $\pm$ 2.2 | 36.2 $\pm$ 2.3 | 35.7 $\pm$ 2.3 | 36.2 $\pm$ 2.3 | 35.7 $\pm$ 2.3 | 34.4 $\pm$ 2.2 | 34.8 $\pm$ 2.3 | 36.6 $\pm$ 2.3 | 34.2 $\pm$ 2.2 | 35.9 $\pm$ 2.3 | 34.6 $\pm$ 2.3 |
| SFT | 32.4 $\pm$ 2.2 | 32.8 $\pm$ 2.2 | 34.4 $\pm$ 2.2 | 33.7 $\pm$ 2.2 | 33.0 $\pm$ 2.2 | 33.9 $\pm$ 2.2 | 33.7 $\pm$ 2.2 | 32.8 $\pm$ 2.2 | 33.7 $\pm$ 2.2 | 34.8 $\pm$ 2.3 | 33.4 $\pm$ 2.2 | 33.6 $\pm$ 2.2 |  |  |
| DPO | 37.5 $\pm$ 2.3 | 31.2 $\pm$ 2.2 | 35.9 $\pm$ 2.3 | 31.2 $\pm$ 2.2 | 37.1 $\pm$ 2.3 | 35.5 $\pm$ 2.3 | 33.5 $\pm$ 2.2 | 32.1 $\pm$ 2.2 | 36.6 $\pm$ 2.3 | 35.7 $\pm$ 2.3 | 36.1 $\pm$ 2.3 | 33.1 $\pm$ 2.2 |  |  |
| Average | 53.0 $\pm$ 1.3 | Prompt | 52.7 $\pm$ 1.3 | 53.5 $\pm$ 1.3 | 41.1 $\pm$ 1.3 | 46.3 $\pm$ 1.3 | 48.3 $\pm$ 1.3 | 50.0 $\pm$ 1.3 | 50.0 $\pm$ 1.3 | 52.9 $\pm$ 1.3 | 53.1 $\pm$ 1.3 | 41.0 $\pm$ 1.3 | 49.1 $\pm$ 1.3 | 48.7 $\pm$ 1.3 |
| SFT | 57.7 $\pm$ 1.3 | 59.1 $\pm$ 1.3 | 56.2 $\pm$ 1.3 | 59.6 $\pm$ 1.3 | 59.8 $\pm$ 1.3 | 58.7 $\pm$ 1.3 | 56.2 $\pm$ 1.3 | 59.2 $\pm$ 1.3 | 60.0 $\pm$ 1.3 | 57.8 $\pm$ 1.3 | 58.0 $\pm$ 1.3 | 58.9 $\pm$ 1.3 |  |  |
| DPO | 54.7 $\pm$ 1.3 | 53.8 $\pm$ 1.3 | 41.9 $\pm$ 1.3 | 50.8 $\pm$ 1.3 | 58.7 $\pm$ 1.3 | 47.0 $\pm$ 1.3 | 50.7 $\pm$ 1.3 | 47.3 $\pm$ 1.3 | 55.5 $\pm$ 1.3 | 35.8 $\pm$ 1.3 | 52.3 $\pm$ 1.3 | 47.0 $\pm$ 1.3 |  |  |

### D.3 Llama3 8B Reasoning Tasks Evaluation Results

The reasoning evaluation results for our training methods and baselines are presented in Table 10 for the LLaMA-3-8B-Instruct model, covering six reasoning domains. Overall, the DPO method generally outperformed SFT and was comparable to the prompt-based approach. This indicates that, in the context of the smaller 8B model, DPO is effective at inducing personality traits without significantly compromising reasoning abilities.

When comparing trait levels, models simulating high levels of personality traits consistently achieved better performance than their low-trait counterparts across both DPO and SFT settings. For instance, in the TruthfulQA benchmark, the high conscientiousness DPO model scored 55.0%, surpassing the low conscientiousness model’s 39.0%. Similarly, in the GSM8K math reasoning task, the high conscientiousness DPO model achieved 72.2

In benchmarks such as TruthfulQA, GPQA (both zero-shot and five-shot), and MathQA, models trained with SFT and DPO performed comparably to the original unaligned model. This suggests that personality trait alignment does not adversely affect performance in these tasks. However, the DPO method exhibited significantly reduced performance on certain benchmarks like CommonsenseQA and MMLU when compared to SFT, prompting, and the original model. Conversely, the SFT method underperformed on the GSM8K benchmark relative to DPO, prompting, and the original model. These observations indicate that, for specific tasks and traits, DPO may be more effective than SFT in preserving or enhancing reasoning abilities in the 8B model.

### D.4 Llama3 8B Reasoning Results Analysis

As the influence of Big Five Personality traits on reasoning tasks in human cognition metioned in Table 4. We analyzed the performance of the LLaMA-3-8B-Instruct model across reasoning tasks to understand how alignment with different personality traits affects its capabilities. The correlations between each trait and the model’s performance are summarized below.

Openness
The impact of Openness on reasoning performance was variable and task-dependent. Models with high Openness levels using the DPO method performed significantly better in mathematical reasoning tasks but underperformed in commonsense reasoning compared to the prompt-based approach and the original model. This suggests that while high Openness alignment may enhance performance in certain mathematical tasks, it does not consistently translate to improvements across all reasoning domains.

Conscientiousness
A positive correlation was observed between high Conscientiousness levels and reasoning performance. Models aligned with high Conscientiousness consistently outperformed their low-level counterparts across most benchmarks. This trend indicates that high Conscientiousness alignment enhances attention to detail and systematic reasoning in the model.

Extraversion
Models with lower Extraversion levels generally demonstrated better performance across reasoning tasks. In commonsense reasoning benchmarks, the low Extraversion model significantly outperformed the high Extraversion model. This negative correlation suggests that high Extraversion may introduce distractibility, potentially hindering performance in tasks requiring focused attention and analytical reasoning.

Agreeableness
The influence of Agreeableness on reasoning performance was minimal and inconsistent. There was no clear advantage for models aligned with either high or low levels of Agreeableness across the benchmarks. This indicates a weak correlation between Agreeableness levels and the model’s reasoning capabilities.

Neuroticism
The correlation between Neuroticism and performance was inconsistent and did not align with expectations based on human cognition. High Neuroticism models performed well in some reasoning tasks, whereas low Neuroticism models scored poorly. These results suggest that high Neuroticism alignment may not negatively impact reasoning performance in the model, contrary to psychological findings, possibly due to limitations in how Neuroticism is represented in the model.

**Table 10: Benchmark results across personality traits and methods for Llama3_8b. Benchmarks are grouped into categories: Hallucination Detection, General Reasoning, Social Reasoning, Commonsense Reasoning, Math Reasoning, and General Knowledge.**
| Benchmark | Original | Method | Openness | Conscientiousness | Extraversion | Agreeableness | Neuroticism | Average |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| High | Low | High | Low | High | Low | High | Low | High | Low | High | Low |  |  |  |
| Hallucination Detection |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| TruthfulQA | 53.5 | Prompt | 49.0 | 51.5 | 50.6 | 44.4 | 45.3 | 51.9 | 49.2 | 50.3 | 54.6 | 45.2 | 49.7 | 48.7 |
| SFT | 50.0 | 45.7 | 50.9 | 43.8 | 46.2 | 52.0 | 49.9 | 46.3 | 53.6 | 42.9 | 50.1 | 46.1 |  |  |
| DPO | 52.4 | 49.1 | 55.0 | 39.0 | 35.0 | 59.2 | 52.8 | 45.5 | 58.2 | 38.8 | 50.7 | 46.3 |  |  |
| Code Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| HumanEval | 60.4 | Prompt | 59.1 | 59.8 | 62.2 | 61.6 | 61.0 | 63.4 | 62.8 | 62.2 | 60.4 | 61.6 | 61.1 | 61.7 |
| SFT | 57.9 | 54.3 | 59.8 | 56.1 | 58.5 | 57.3 | 60.4 | 54.9 | 58.5 | 58.5 | 59.0 | 56.2 |  |  |
| DPO | 57.3 | 0.6 | 27.4 | 0.0 | 43.3 | 0.0 | 8.5 | 32.9 | 0.0 | 7.9 | 27.3 | 8.3 |  |  |
| MBPP | 54.6 | Prompt | 54.6 | 55.4 | 54.2 | 55.2 | 55.8 | 56.0 | 55.4 | 54.8 | 54.4 | 55.8 | 54.9 | 55.4 |
| SFT | 56.2 | 56.2 | 54.2 | 56.2 | 56.4 | 56.4 | 55.6 | 55.8 | 55.0 | 56.4 | 55.5 | 56.2 |  |  |
| DPO | 53.6 | 47.6 | 53.0 | 35.2 | 54.6 | 51.4 | 54.4 | 53.8 | 52.0 | 54.2 | 42.9 | 48.4 |  |  |
| Social Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| SocialIQA | 49.7 | Prompt | 41.9 | 42.3 | 41.1 | 39.3 | 41.5 | 41.6 | 41.8 | 39.5 | 42.1 | 39.4 | 41.7 | 40.4 |
| SFT | 44.0 | 44.9 | 45.9 | 41.9 | 44.4 | 44.6 | 43.7 | 41.4 | 44.6 | 40.8 | 44.5 | 42.7 |  |  |
| DPO | 43.8 | 43.8 | 42.5 | 37.8 | 41.8 | 40.9 | 42.8 | 38.4 | 42.8 | 39.0 | 42.7 | 40.0 |  |  |
| Commonsense Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| CommonsenseQA | 51.8 | Prompt | 64.6 | 60.6 | 38.0 | 31.3 | 45.9 | 55.0 | 55.4 | 36.3 | 33.9 | 23.3 | 47.6 | 41.3 |
| SFT | 61.8 | 57.9 | 50.5 | 34.3 | 52.7 | 60.8 | 55.4 | 36.0 | 63.4 | 30.6 | 56.8 | 43.9 |  |  |
| DPO | 22.9 | 24.8 | 48.2 | 21.6 | 29.1 | 56.6 | 28.4 | 26.3 | 47.7 | 23.7 | 35.3 | 30.6 |  |  |
| Math Reasoning |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| GSM8K | 64.7 | Prompt | 13.5 | 58.4 | 23.4 | 61.0 | 40.0 | 57.1 | 29.3 | 71.6 | 24.1 | 31.9 | 26.1 | 56.0 |
| SFT | 19.8 | 0.5 | 20.2 | 1.4 | 6.0 | 0.5 | 6.4 | 4.8 | 20.1 | 53.3 | 14.5 | 12.1 |  |  |
| DPO | 68.4 | 31.8 | 72.2 | 31.8 | 69.7 | 63.0 | 70.7 | 64.8 | 71.9 | 3.0 | 70.6 | 38.9 |  |  |
| MathQA | 27.9 | Prompt | 27.6 | 28.3 | 27.9 | 27.3 | 27.1 | 27.8 | 27.2 | 28.1 | 28.1 | 25.9 | 27.6 | 27.5 |
| SFT | 30.1 | 30.2 | 29.6 | 30.3 | 31.0 | 30.6 | 29.6 | 30.3 | 29.6 | 29.4 | 30.0 | 30.2 |  |  |
| DPO | 26.9 | 27.8 | 28.3 | 25.1 | 25.8 | 27.6 | 24.9 | 27.7 | 29.7 | 24.9 | 27.1 | 26.6 |  |  |
| General Knowledge |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MMLU | 51.2 | Prompt | 37.5 | 29.1 | 23.2 | 27.0 | 24.7 | 29.2 | 27.7 | 25.5 | 23.4 | 23.8 | 27.3 | 26.9 |
| SFT | 45.0 | 48.5 | 35.6 | 32.0 | 37.5 | 46.5 | 44.2 | 39.9 | 47.1 | 31.7 | 41.9 | 39.7 |  |  |
| DPO | 23.0 | 29.8 | 29.7 | 26.9 | 24.8 | 41.4 | 30.7 | 26.3 | 30.8 | 23.1 | 27.8 | 29.5 |  |  |
| GPQA (0-shot) | 28.1 | Prompt | 29.0 | 28.8 | 28.6 | 23.0 | 28.6 | 29.2 | 29.0 | 27.2 | 28.8 | 28.3 | 28.8 | 27.3 |
| SFT | 27.9 | 27.9 | 28.1 | 25.0 | 27.2 | 28.3 | 28.8 | 24.1 | 29.0 | 28.3 | 28.2 | 26.7 |  |  |
| DPO | 27.9 | 25.0 | 29.7 | 21.0 | 27.2 | 26.8 | 28.8 | 21.4 | 29.5 | 25.2 | 28.6 | 23.9 |  |  |
| GPQA (5-shot) | 29.9 | Prompt | 29.7 | 26.6 | 28.8 | 26.8 | 28.3 | 26.6 | 27.9 | 28.6 | 29.0 | 25.2 | 28.7 | 26.8 |
| SFT | 26.1 | 27.0 | 28.8 | 26.6 | 28.8 | 28.6 | 30.6 | 27.9 | 28.6 | 27.5 | 28.6 | 27.5 |  |  |
| DPO | 27.9 | 26.3 | 28.3 | 23.0 | 26.8 | 28.1 | 27.5 | 24.6 | 28.8 | 25.2 | 27.9 | 25.4 |  |  |
| Average | 43.9 | Prompt | 35.8 | 40.5 | 31.5 | 34.4 | 34.3 | 39.5 | 35.1 | 38.2 | 31.7 | 29.1 | 33.7 | 36.4 |
| SFT | 37.2 | 34.0 | 34.8 | 27.6 | 32.8 | 35.3 | 35.0 | 29.9 | 38.8 | 34.8 | 35.7 | 32.3 |  |  |
| DPO | 35.6 | 30.7 | 41.6 | 26.9 | 34.1 | 43.2 | 37.7 | 33.8 | 42.4 | 23.4 | 38.3 | 31.6 |  |  |

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: Indicated in the summary section.
soundness: 3
strengths: Indicated in the summary section
summary: The paper tries to embed realistic human personality traits in LLMs.

1. The paper releases a dataset, Big5-chat, consisting of 100k synthetically generated dialogues.
2. The paper shows that it can outperform earlier prompting based personality induction techniques in personality assessment tests.
3. The paper shows interesting experiments correlating personality with reasoning capabilities.


I have the following questions and suggestions. I am open to modifying my review based on the authors' rebuttal:


1. Line 192 - Equation-1: This seems incorrect. In Line 196, it states "γ = 1 fully adopting the expert generator’s logits." I believe the equation either needs to be converted to the form γ*z_base and (1-γ)*z_expert, or Line 196 should be revised.

2. I could not understand the argument of Table-1. The classifier is trained on the PsychGenerator dataset. The generator too is trained on the PsychGenerator dataset. 

    a. The table shows that a classifier thinks that the generator (after training on the PsychGenerator dataset) is able to fool the classifier effectively. Isn't that expected? If the claim is that the generator has learned to generate text that can mimic different personalities, why not train the classifier on a different dataset and then demonstrate the generalization capabilities of the generator?

    b. Additionally, in Table-1, despite training on the same dataset, the accuracy difference is not substantial (1.7% difference between the base model and the trained one vs. 13.5% difference between real data and the trained generator).

    c.  Also, the PsychGenerator dataset is composed of Facebook posts, which severely limits the applicability of the claims made in the paper. Can the authors conduct or have they conducted any study to test the generalizability of the claims made in the paper (L96-L105) to domains other than Facebook?


3. L214-215 - I would like to understand if the method generates/extracts text with only one trait (e.g., neuroticism), or did the authors attempt to mix traits? While personality traits can appear independently, there is ample literature supporting the observation of mixed personality traits, such as agreeable and neurotic individuals.

4. Re L100 - Did the authors try experimenting with combining SFT (and DPO) with prompting to improve their scores further? 


5. Regarding Table-2 and Section 5.2:
    a. SFT and DPO are supervision-heavy techniques, whereas prompt-based methods require much less supervision. Given the data requirements, it is not surprising that they perform better than prompting. A better comparison would be to see if, after SFT/DPO, the trained models can outperform GPT/Claude. This would be similar to the experiments conducted in [1], where it was shown that fine-tuned models perform much better than closed-source models (and also their base models).
    b. Given that prompt-based techniques use much less data than SFT/DPO, can the authors try rotating the examples provided in the prompts? This might improve its performance relative to SFT. (If they have already done this, please indicate this to me.)

6. Regarding Line 457, "somewhat align": Can the authors compute the degree of alignment?

7. Regarding Section 5.3: I am unclear about the motivation for this experiment. The paper begins with a very useful and interesting motivation (Lines 31-36) of making LLMs output more authentic for conversational tools and addressing mental health problems, but this was not followed up in the subsequent sections. Section 5.3 is an interesting experiment, but what is its purpose as an evaluation? While reasoning (or other capabilities) may be correlated with personality traits, how can we utilize this? Additionally, why limit the evaluation to reasoning capabilities, and not extend it to other NLP benchmarks?


[1] Large Content and Behavior Models: https://openreview.net/forum?id=TrKq4Wlwcz
weaknesses: Indicated in the summary section

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: [Appendix B.1] “We structure the labels for the expert generator training data into three equidistant thresholds”. The process is not very clear. How did you pick the thresholds? Did you discard the data whose value was in the middle? I believe more details would be beneficial for the reader, and maybe also an example.

In Table 1, the authors evaluate the quality of the generated dataset using their proposed framework against two other prompted LLMs. The average performance shows that the Generator method outperformed the others and provides more balanced performance across different traits. However, for individual traits like Extraversion, Agreeableness, and Neuroticism, the two prompted models outperform the Generator. How can you explain these results? Did you perform an empirical evaluation of the generation to better understand the reasons for this behaviour?
soundness: 3
strengths: The paper is well-written, clear and easy to follow.

The literature review is comprehensive of both studies from cognitive science and computer science and provides a complete and clear picture of the state of the art, and the limitations that the current NLP field is facing in the paper’s domain. 

The authors propose a novel dataset and creation strategy that can be beneficial not only in the field of persona-based LLMs but can also be applied to other fields, for the creation of ad-hoc data that need to embed specific characteristics in the generation.

Moreover, the evaluation proposed both for the quality assessment of the dataset and the experiments proposed is solid and comprehensive, by taking into account not only different training strategies but also assessing the model’s capabilities over a wide range of reasoning tasks.
summary: The paper tackles the problem of embedding personality traits into LLMs. To this end, the authors propose: 
- A novel large-scale dataset, BIG5-CHAT, of 100000 dialogues which grounds real human personalities in text. To generate the dataset they combine controllable text generation models with a domain-specific, personality-annotated dataset. They propose a classification-based evaluation to address to which extent the proposed generation framework is capable of embedding the personality traits within the text.
- The authors test the BIG5-CHAT over two instruction-based models, i.e. Llama-3-8B and Llama-3-70B under three main learning strategies: traditional prompting techniques (zero-shot and few-shot), Supervised Fine Tuning (SFT), and Direct Preference Optimization (DPO).
- The authors provide a comprehensive empirical investigation into how personality traits affect performance in both social reasoning and general reasoning tasks testing the different model’s configurations over ad-hod benchmarking datasets.
	
Results show that:
- The dataset produced using controllable text generation not only achieves a higher average accuracy score but also provides a more balanced performance across all five personality traits.
- For Llama models both prompting and training techniques (SFT, DPO) successfully reflect the personality traits in their responses. However, training-based methods indicate more personality traits than prompting techniques. 
- When testing the reason capabilities of the models, SFT consistently outperformed or matched DPO, and both training methods outperformed prompt-based approaches.
- LLMs trained with higher levels of conscientiousness and agreeableness excel in various reasoning tasks, while models with lower extraversion and neuroticism performed better at all reasoning tasks.
weaknesses: The description and discussion of the results should be more detailed and closely tied to the data presented in the tables. For instance, statements such as “When comparing trait levels, models with higher conscientiousness and agreeableness generally outperformed those with lower levels” or “indicating that certain personality trait levels can improve performance in reasoning tasks” are either difficult to verify directly from the provided data or are too vague. To address this, I recommend emphasizing the specific results that illustrate these points by using bold text or guiding the reader to the relevant findings within the tables. Additionally, the authors offer a more comprehensive analysis of Llama-3-8B in the Appendix compared to Llama-3-70B in the main text.


Besides these, I will list hereafter small changes that I believe would benefit the paper:

- [Figure 1] Add a reference to the Figure within the text and detail the caption with a brief description of the modules depicted.
- [Section 4.2] Token analysis: missing description of the tokenization process. What were the token units employed? Words, or LLM 
tokens? Which tokeniser did you use for the analysis?
- [Section 4.3, line 300] In context-examples: how were they chosen and how many were used? Which prompt was employed? While the prompt provided in Appendix B.3 might address some of these questions, it is not referenced within the main text. Additionally, this appendix section appears somewhat disorganized, lacking clear references to the main text and presenting prompts in a different order. I suggest restructuring this appendix section for better clarity.
- [Table 3] The arrows in the header are all in the same direction. Consider providing more details in the caption in order to better interpret the Table and the results.
- [Appendix B.4] The first part of the section is confusing and there are repeated concepts. Maybe some draft sentences were not commented out. (e.g., “We follow lm harness and Deepseek for their implementations of evaluation on the general benchmarks and social benchmarks.”, followed by “We followed the evaluation frameworks established by the Language Model Evaluation Harness (lm-eval-harness) and DeepSeek for assessing performance on general and social benchmarks. Our evaluation pipeline utilized the following key parameters”).

Moreover, the authors did not provide references (and descriptions) for “Language Model Evaluation Harness” and “DeepSeek”.  
[Appendix D.3 and D.4] Consider incorporating some of these findings into the main body of the paper to provide the reader with a comprehensive overview of the LLMs’ capabilities for both models under analysis. Additionally, ensure that the main text offers the same level of detailed description for Llama-3-70B as it does for the other models.

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 3
questions: - line 363-367: Does this mean that personality traits are evaluated by humans using questionnaires? If so, the authors should provide details on evaluator recruitment. 
- (Related to third weakness) Have the authors considered combining all traits in a single response, like combining all five expert logits into one combined logit (in Equation 1)? 
- line 420: The authors said that they excluded results for instruction-based prompting on Llama-3-8B-Instruct. However, Table 2 reports Prompt-Inst and excludes Prompt-Demo. Which is correct? There must be a clarification on this.
- Experiments: In my opinion, experimenting SFT, DPO and prompting on other datasets (the ones that you mentioned as failing to ground human personality realistically) is necessary to prove the effectiveness of training-based methods for inducing personality traits. In that case, the authors will be able to demonstrate the effectiveness of training methods, as well as excellence of their dataset over others.
soundness: 3
strengths: - The paper is clearly written and easy to understand.
- Background section is thorough and well-written that helps the reviewers to understand more deeply about the domain and the current challenges.
summary: Challenges:
1. Existing methods rely on prompting models with descriptions of behaviors associated with personality traits. These behavioral descriptions are nonsensical for text-based LLMs, failing to ground their personality in realistic patterns of how humans’ personality is expressed in text.
2. The scarcity of large-scale, human-generated datasets annotated with personality traits has hindered the exploration of training-based approaches, limiting most prior research to prompting-based methods.

To address these challenges, this paper introduces BIG5-CHAT, a large-scale dialogue responses dataset that is designed to capture Big Five personality traits within diverse social interactions. Also, leveraging the dataset, this work empirically investigates if training-based methods benefit in grounding in real human data compared to traditional prompting methods.
weaknesses: - **Although the motivation for the work is convincing, contribution is comparatively trivial.** Much of their dataset and framework are adopted from previous works, with only slight change. Also, the necessity of conducting alignment comparison experiment only on their dataset is unclear. Is there a specific reason or intention as to why the authors did not experiment on existing datasets?
- **The argument that BIG5-CHAT captures personality traits better than other datasets is not convincing.** In line 265, the authors argue their dataset captures personality and nuances better than other datasets. However, the provided table (Table 8) does not demonstrate the argument. Human evaluation, or at the very least, case study for comparing BIG5-CHAT over other datasets is required. Also in experiments, the authors did not conduct experiments on other datasets, leaving the validity of the dataset in question. 
- **This work fails to capture all traits comprehensively.** The proposed method, DExperts, induces each personality trait in a response. However, in reality, the personality of a human is very complex and comprehensive. It is doubtful that their approach, as opposed to their statement of “capturing nuanced expression of personality traits”, can truly reflect human personality more elaborately. 
- **Related work section could include more works.** Although a considerable portion of the work deals with alignment methods, review on works that concerns effectiveness of training-based methods in inducing personality, or let alone works on alignment, are nowhere to be seen.

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 4
questions: Question 1: How do you interpret the results of Table 1, in which the proposed method is not as effective as two baseline models in "Agreeableness" and "Neuroticism"? What makes the difference between these traits?

Question 2: How do you determine the scaling factor \gamma? How much does this factor influence the generated results?
soundness: 3
strengths: Originality: The BIG5-CHAT dataset is novel and represents a valuable resource for advancing human-like LLM research.

Quality: The experiments are well-designed and comprehensive. The results on several benchmarks are also aligned well with psychology researches.

Clarity: The paper is easy to follow, with clear explanations of the methods and concepts.

Significance: The paper provides a new dialogue dataset with personality traits. Some findings show the impacts of personality traits in different benchmarks.
summary: The paper presents BIG5-CHAT, a dataset of 100,000 dialogues for simulating realistic human personality traits. The authors leverage two datasets, PsychGenerator and SODA, to generate BIG5-CHAT with DExperts framework and train LLMs on realistic personality patterns. The experiments show that training-based methods outperform prompting methods on personality assessments. Other experiments also reveal the personality traits of LLMs have influence on behaviors in various reasoning tasks, which align with human cognitive tendencies observed in psychology.
weaknesses: Absence of Human Evaluation: To enhance the validity of personality alignment claims, human evaluations are recommended. A suggested methodology is to have a panel of human raters assess a sample of dialogues from BIG5-CHAT, scoring them on each of the Big Five traits (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism). The human ratings can then be compared to the intended trait levels to verify if the model successfully conveys the intended personality traits.

Limited Novelty: Although this paper introduces the BIG5-CHAT dataset and applies training methods like SFT and DPO for personality alignment, prior work has shown that it is possible to train LLMs to exhibit specific personality traits, e.g., https://arxiv.org/pdf/2312.12999. In this paper mentioned above, they collected data of different MBTI types and trained 16 agents. These agents can chat in the way of certain personality trait and have the ability to pass the MBTI test. Additionally, the method of generating personality-aligned responses is adapted from DExperts. To strengthen the novelty, it would be helpful if the authors provided a clearer comparison to existing work, particularly in how their dataset creation or methodology differs. Emphasizing unique aspects, such as the integration of PsychGenerator and SODA datasets or specific adjustments to the DExperts framework for personality traits, would clarify the innovative contributions of this study.