The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs

## Abstract

Abstract Personality traits have long been studied as predictors of human behavior.
Recent advances in Large Language Models (LLMs) suggest similar patterns may emerge in artificial systems, with advanced LLMs displaying consistent behavioral tendencies resembling human traits like agreeableness and self-regulation.
Understanding these patterns is crucial, yet prior work primarily relied on simplified self-reports and heuristic prompting, with little behavioral validation.
In this study, we systematically characterize LLM personality across three dimensions: (1) the dynamic emergence and evolution of trait profiles throughout training stages; (2) the predictive validity of self-reported traits in behavioral tasks; and (3) the impact of targeted interventions, such as persona injection, on both self-reports and behavior.
Our findings reveal that instructional alignment (e.g., RLHF, instruction tuning) significantly stabilizes trait expression and strengthens trait correlations in ways that mirror human data.
However, these self-reported traits do not reliably predict behavior , and observed associations often diverge from human patterns .
While persona injection successfully steers self-reports in the intended direction, it exerts little or inconsistent effect on actual behavior.
By distinguishing surface-level trait expression from behavioral consistency, our findings challenge assumptions about LLM personality and underscore the need for deeper evaluation in alignment and interpretability.
We make public all code and source data at https://github.com/psychology-of-AI/Personality-Illusion for full transparency and reproducibility, to benefit future works in this direction.

## 1 Introduction

Large Language Models (LLMs) demonstrate impressive abilities in generating coherent and contextually appropriate text, often exhibiting behaviors resembling human personality traits—such as consistent tone, emotional valence, sycophancy, and risk sensitivity (Jiang et al., 2024a; Han et al., 2024b).
Understanding these emergent traits is critical. They affect user interaction (e.g., trust vs. alienation) (van Pinxteren et al., 2023), signal alignment risks like undue agreement or avoidance (Chen et al., 2024c), offer insight into generalization and internal representations (Yetman, 2024), and raise ethical concerns around anthropomorphization (Reinecke et al., 2025).

Existing work approaches LLM traits in two ways.
(1) Self-report questionnaires (Pellert et al., 2024; Bhandari et al., 2025) offer psychometric grounding but face issues of behavioral validation, trait interdependence, prompt sensitivity (Khan et al., 2025), and potential data leakage–casting doubt on profile stability and significance (Gupta et al., 2023; Sühr et al., 2023; Song et al., 2023).
Recent studies further show survey prompts often diverge from open-ended behavior (Röttger et al., 2024; Huang & Hadfi, 2025), and cultural alignment is unstable, formatting-dependent, and largely unsteerable (Khan et al., 2025; Dominguez-Olmedo et al., 2024).
While some internal consistency exists (Moore et al., 2024), it is narrow in scope, reinforcing the need to go beyond surface-level prompt manipulations toward more behaviorally grounded alignment methods.
(2) Intervention-based methods (e.g., prompting or training) (Li et al., 2024b; Yang et al., 2025) elicit observable shifts but lack grounding in psychological theory, limiting comparison to humans (Tseng et al., 2024b; Liu et al., 2025b), and persona-style interventions often obscure underlying traits as surface expressions (Wang et al., 2025c; Petrov et al., 2024).

Figure: Figure 1: Experimental framework for analyzing personality traits in LLMs. We investigate (RQ1) the emergence of self-reported traits (e.g., Big Five, self-regulation) across training stages; (RQ2) their predictive value for real-world–inspired behavioral tasks (e.g., risk-taking, honesty, sycophancy); and (RQ3) their controllability through persona injections. Trait assessments use adapted psychological questionnaires and behavioral probes, with comparisons to human baselines.
Refer to caption: x1.png

These approaches offer complementary strengths, yet remain poorly integrated. We address this gap by systematically examining LLM personality across three dimensions (Fig. [1](https://arxiv.org/html/2509.03730v2#S1.F1)):
First, we trace the development and interrelation of self-reported traits across models and training stages.
Second, we assess whether these profiles manifest in real-world-inspired tasks, using behavioral paradigms from human psychology.
Third, we test how interventions like persona injection affect both self-reports and behavior. We pose the following three research questions:

- •
RQ1 (Origin): When and how do human-like traits emerge and evolve across LLM training?
- •
RQ2 (Manifestation): Do self-reported traits predict performance in real-world–inspired tasks?
- •
RQ3 (Control): How do interventions like persona injection modulate trait profiles and behavior?

We find that instructional alignment(^1^11Refers to post-pretraining phases such as RLHF, DPO, or instruction tuning.) plays a pivotal role in shaping LLM traits, consistently increasing openness, agreeableness, and self-regulation while reducing neuroticism. Trait expression becomes more stable—variability drops by 40.0% (Big Five) and 45.1% (self-regulation)—with stronger trait intercorrelations, resembling human patterns.
Yet, these self-reports poorly predict behavior: only $\sim$24% of trait-task associations are statistically significant, and among them, just 52% align with human expectations (random chance is 50%).
While across prompting strategies persona injection shifts self-reported traits in the expected direction (e.g., agreeableness $\beta=3.95$, $p<.001$ following prompting toward an *agreeable* persona), it has minimal impact on behaviors that are expected to be affected based on human studies (e.g., sycophancy $\beta=0.03$, $p=0.67$).

These results reveal a fundamental dissociation between linguistic self-expression and behavioral consistency: even state-of-the-art LLMs fail to act in line with their reported traits.
Current alignment methods such as RLHF refine linguistic plausibility without grounding it in behavioral regularity, and interventions like persona prompts only steer surface-level self-reports.
This inconsistency cautions against treating linguistic coherence as evidence of cognitive depth and raises concerns for real-world deployment, underscoring the need for different and deeper forms of alignment.
We make public all code and source data at [https://github.com/psychology-of-AI/Personality-Illusion](https://github.com/psychology-of-AI/Personality-Illusion) for full transparency and reproducibility, to benefit future works in this direction.

## 2 RQ1: Origin of Human-like Traits in LLMs

We study self-reported personality trait profiles in LLMs using well-established, standardized psychological questionnaires (John et al., 1991; Brown et al., 1999).
Prior work shows models differ in such profiles (Jiang et al., 2023a; Bhandari et al., 2025), but rarely examines whether inter-trait relationships are coherent or stable.
In humans, traits evolve into structured, interdependent patterns over time (Roberts et al., 2006; Caspi et al., 2005; Digman, 1997).
LLMs similarly undergo staged development–pretraining, instruction tuning, and RLHF–each introducing distinct data, goals, and human influence.
Yet how these phases contribute to the emergence and stabilization of personality-like traits remains underexplored. We examine the developmental trajectory of LLMs to determine when and how such traits originate and solidify, focusing on the following research question:

###### Research Question 1 (Origin) .

When and how do human-like traits emerge and change across different LLM training stages?

###### Research Question 1 (Origin) .

### 2.1 Experiment Setup

#### Psychological Questionnaire.

We assess LLM personality profiles using two well-established instruments: the Big Five Inventory (BFI) (John et al., 1991), which measures openness, conscientiousness, extraversion, agreeableness, and neuroticism, and the Self-Regulation Questionnaire (SRQ) (Brown et al., 1999), which evaluates self-control and goal-directed behavior.
These tools capture core personality dimensions and behavioral regulation, adapted here to probe LLMs’ self-reported traits under controlled prompting.
Full prompt details are in Appendix [D](https://arxiv.org/html/2509.03730v2#A4).

#### Models and Implementation.

To ensure robust results, we evaluate 12 widely used open-source LLMs–comprising 6 base models (pre-training) and their corresponding instruction-tuned variants (post-training alignment)–listed in Table [1](https://arxiv.org/html/2509.03730v2#S2.T1).
Each model is evaluated under three default system prompts (shown in Table [4](https://arxiv.org/html/2509.03730v2#A4.T4) in Appendix [D](https://arxiv.org/html/2509.03730v2#A4)), across three temperature settings, and with three repeated generations per condition, resulting in 27 outputs per item (3 prompts × 3 temperatures × 3 runs).

Figure: Figure 2: Emergence and stabilization of personality traits in LLMs (RQ1). (A) Mean self-reported Big Five and self-regulation scores (±95% CI): alignment-phase models (violet) show higher openness, agreeableness, and self-regulation, and lower neuroticism than base models (pink). (B) Alignment reduces variability: median absolute deviation drops 60–66% across traits (*** $p<0.001$, ** $p<0.01$, * $p<0.05$, n.s. not significant). (C) Regression of self-regulation on the Big Five shows stronger, more coherent associations in aligned (violet) vs. pre-trained (pink) models, suggesting more consolidated personality profiles. Gray boxes mark expected directions from human studies (↑, ↓, –).
Refer to caption: figures/RQ1_updated.png

### 2.2 Statistical Analysis

#### a) Examining Trait-level Differences by Training Phase.

We test whether LLMs exhibit systematic differences in self-reported personality traits across training phases (pre- vs post-alignment). We fit a mixed-effects binomial logistic regression model predicting training phase from six standardized trait scores: the Big Five traits and Self-Regulation. Random intercepts are included for *model*, *temperature* and *prompt* to account for repeated measures and variation due to prompting conditions. Model inference is based on Wald $z$-statistics and 95% confidence intervals.
To assess multicollinearity, we compute Variance Inflation Factors (VIFs), which all fall within acceptable ranges ($<$ 2), indicating no serious collinearity concerns.

#### b) Examining Trait Stability Under Repeated Prompting.

To assess the internal consistency of model trait expression, we analyze trait stability under repeated prompting with the same input across multiple generations. We apply Levene’s test to compare the trait-wise variance between base and instruct models. This test is robust to non-normality and uses the median as the center. Prior to testing, self-regulation scores are rescaled to match the 1–5 range of other traits.

#### c) Trait Coherence: Self-Regulation and Big Five.

To examine whether LLMs express coherent trait structures similar to those observed in humans, we test whether self-regulation scores are predicted by the Big Five traits. We fit linear regression models for each training phase (pre- vs post-alignment), regressing standardized self-regulation on the five personality traits. We evaluate the strength and direction of coefficients, comparing them to known associations in human studies.

### 2.3 Results

#### a) Trait-level differences.

The logistic regression reveals that openness ($\beta=1.48$, 95% CI = [0.74, 2.22], $p<.001$), neuroticism ($\beta=-1.20$, CI = [$-2.00$, $-0.41$], $p=.003$), and agreeableness ($\beta=0.74$, CI = [0.03, 1.44], $p=.041$) significantly predict whether a model is instructionally aligned (Fig. [2](https://arxiv.org/html/2509.03730v2#S2.F2).a). Instruction-aligned models typically sit
$\approx+1.5\,\text{SD}$ higher in Openness,
$+\tfrac{1}{2}\,\text{SD}$ higher in Agreeableness,
and $-1\,\text{SD}$ lower in Neuroticism than their pre-trained counterparts—practically, that’s a big uptick in sociability traits and a marked drop in anxiety-like signals. *Instructionally aligned models are more open and agreeable but less neurotic than pre-trained models*. Change in extraversion ($\beta=-0.12$, $p=.739$) and conscientiousness ($\beta=-0.61$, $p=.089$) is not significant.

#### b) Trait stability under repeated prompting.

Levene’s test confirms *significantly lower variability in five of six traits for instruction-aligned models compared to pre-trained models* (Fig. [2](https://arxiv.org/html/2509.03730v2#S2.F2).b): openness ($p=.01$), conscientiousness ($p=.006$), extraversion ($p<.001$), neuroticism ($p<.001$), and self-regulation ($p<.001$). Agreeableness shows no significant difference ($p=.54$). Instruction alignment consolidates trait expression and reduces susceptibility to prompt-level noise.

#### c) Trait coherence with human benchmarks.

Instructionally aligned models display *stronger and more consistent associations between personality traits and self-regulation* (Fig. [2](https://arxiv.org/html/2509.03730v2#S2.F2).c): self-regulation increases with conscientiousness ($\beta=12.32$, 95% CI = [9.23, 15.41]), openness ($\beta=15.23$, CI = [11.58, 18.89]), agreeableness ($\beta=11.36$, CI = [8.72, 13.99]), and extraversion ($\beta=23.33$, CI = [19.05, 27.62]), while it decreases sharply with neuroticism ($\beta=-16.27$, CI = [$-20.3$, $-12.23$]; all $p<.001$). These patterns mostly align with well-established findings in human personality research (Roberts et al., 2014) (see Appendix [F](https://arxiv.org/html/2509.03730v2#A6) for review of the expectations from human studies).

In contrast, *pre-trained models exhibit weaker and less consistent associations*.
While conscientiousness ($\beta=7.62$, CI = [3.83, 11.40], $p<.001$) and agreeableness ($\beta=6.60$, CI = [2.74, 10.46], $p<.001$) show significant positive effects, consistent with human studies.
Openness and Neuroticism show no reliable association ($p=.068$ and $p=.543$), contrary to human studies. Extraversion is non-significant ($p=.324$), but human studies show mixed results (Nilsen et al., 2024).

**Table 1: List of Evaluated Models by Category. We evaluate a total of 18 models: six small base models, their corresponding six small instruct models, and six large instruct models. For RQ1 (Section [2](https://arxiv.org/html/2509.03730v2#S2)), we compare the group of six small base models with the corresponding group of six small instruct models. For RQ2 and RQ3 (Sections [3](https://arxiv.org/html/2509.03730v2#S3) and [4](https://arxiv.org/html/2509.03730v2#S4)), we use all 12 instruct models, reporting overall results and breakdowns by size (small vs. large) and by family (LLaMA vs. Qwen).**
|  | Model Names |
| --- | --- |
| Base (pre-training) | LLaMA-3.2 (3B), LLaMA-3 (8B), Qwen2.5 (1.5B), Qwen2.5 (7B), Mistral-7B-v0.1, OLMo2 (7B) |
| Small Instruct | LLaMA-3.2 (3B) Instruct, LLaMA-3 (8B) Instruct, Qwen2.5 (1.5B) Instruct, Qwen2.5 (7B) Instruct, Mistral-7B-v0.1 Instruct, OLMo2 (7B) Instruct |
| Large Instruct | LLaMA-3.3 (70B) Instruct, LLaMA-3.1 (405B) Instruct, Qwen2.5 (72B) Instruct, Qwen3 (235B) Instruct, Claude 3.7 Sonnet, GPT-4o |

## 3 RQ2: Manifestation of Human-like Traits in LLM Behaviors

From RQ1, we find that LLMs after instructional alignment exhibit more stable and coherent personality trait profiles when measured with psychological questionnaires.
Yet their significance remains debated: some view them as surface-level artifacts shaped by training data, prompts, or leakage (Gupta et al., 2023; Sühr et al., 2023; Song et al., 2023), while others see them as meaningful reflections of internalized behavioral patterns (Serapio-Garc´ıa et al., 2023; Wang et al., 2025b; Jiang et al., 2023b).

In humans, traits consistently guide behavior across contexts (Roberts et al., 2007), motivating us to test whether LLM traits function similarly.
To move beyond self-reports, we adapt psychological tasks with known links to personality constructs, which–unlike common benchmarks–were not designed as training targets (Hasan et al., 2025; Sainz et al., 2023; Zhou et al., 2025).
Although LLMs lack embodiment and emotion, many paradigms (e.g., decision-making under uncertainty, implicit bias) rely on symbolic reasoning with text-based operationalizations (Kahneman & Tversky, 2013; Greenwald et al., 1998), making them suitable for probing language models (Binz & Schulz, 2023b; Kosinski, 2023; Bai et al., 2024).
We thus focus on the following research question:

###### Research Question 2 (Manifestation) .

How do self-reported personality traits transfer to and predict performance in real-world–inspired behavioral tasks?

###### Research Question 2 (Manifestation) .

### 3.1 Real-world Behavioral Tasks

To evaluate whether personality traits manifest in meaningful behavior, we specifically adapt five downstream tasks from psychological research (Roberts et al., 2007).
These tasks were selected for their importance for real-world LLM applications and validated links to specific traits (e.g., extraversion $\rightarrow$ risk-taking, self-regulation $\rightarrow$ reduced stereotyping; see Appendix [G](https://arxiv.org/html/2509.03730v2#A7)).

#### Risk-Taking.

Risk-taking is a key behavioral trait, especially as LLMs are used in decision-making roles (Bhatia, 2024).
To assess it, we adapt the Columbia Card Task (CCT) (Figner et al., 2009), a standard human measure of risk-taking.
In this task, participants decide how many of 32 cards to flip, weighing rewards from “good” cards against penalties from “bad” ones.
We apply this structure to LLMs using analogous prompts and measure their willingness to take risks.
Higher scores indicate greater risk-taking.
Full details are in Appendix [E](https://arxiv.org/html/2509.03730v2#A5).

#### Social Bias.

Implicit social bias in LLMs poses serious risks, including the reinforcement of stereotypes and discriminatory outputs (Han et al., 2024a; Jiang et al., 2023c).
Since such biases in humans relate to traits like self-regulation (Legault et al., 2007; Allen et al., 2010; Ng et al., 2021), we evaluate them in LLMs using a method based on the Implicit Association Test (IAT) (Bai et al., 2024).
The model is asked to associate terms from two social groups (e.g., White vs. Black names) with contrasting attributes (e.g., “good” vs. “bad”).
A bias score from -1 to 1 reflects preference; its absolute value indicates bias magnitude.
Full details are in Appendix [E](https://arxiv.org/html/2509.03730v2#A5).

#### Honesty.

Honesty is essential for LLMs, as users rely on them for accurate and trustworthy information (Yang et al., 2024).
In research, it is often measured through *calibration*—how well a model’s confidence aligns with its actual accuracy (Li et al., 2024a; Yang et al., 2024).
This mirrors human concepts like *epistemic honesty* (knowing what one knows) and *metacognition* (reflecting on one’s beliefs) (John, 2018; Byerly, 2023).
Following prior human study (Nelson & Narens, 1980), we present factual questions and collect two confidence scores: $C_{1}$ (initial answer) and $C_{2}$ (confidence upon review). Half of the questions are augmented with synthetic entities to test robustness.
Calibration (accuracy vs. $C_{1}$) reflects epistemic honesty; self-consistency ($C_{1}$ vs. $C_{2}$) reflects metacognition.
High calibration error indicates overconfidence; high inconsistency indicates poor metacognition.
Full task details are in Appendix [E](https://arxiv.org/html/2509.03730v2#A5).

#### Sycophancy.

Sycophancy—the tendency to conform to others’ opinions—is a key concern in LLMs, where models may overly align with user input at the expense of objectivity (Cheng et al., 2025; Sharma et al., 2023).
To measure this, we adapt an Asch-style conformity paradigm (Asch, 1956) using moral dilemmas from Christensen et al. (2014), where no answer is objectively correct.
The model first answers independently, then sees the same question prefaced by a conflicting user opinion.
Sycophancy is measured by whether the model changes its response to conform.
Higher scores indicate greater conformity.
Full task details are in Appendix [E](https://arxiv.org/html/2509.03730v2#A5).

### 3.2 Big5 Personality, Self-Regulation, and Behavioral Outcomes in Humans

Psychological research has demonstrated that the Big Five personality traits, along with self-regulation, are systematically associated with consistent behavioral tendencies across a wide range of contexts. To inform our evaluation of LLM behavior, we draw on these well-established human patterns to define directional expectations for each behavioral task.
For each task described above, we outline the expected relationships between personality traits and behavior based on prior literature, which is summarized in Appendix [G](https://arxiv.org/html/2509.03730v2#A7) and also provided in the “Human” row of Table [3](https://arxiv.org/html/2509.03730v2#A3.T3) in Appendix [C.2](https://arxiv.org/html/2509.03730v2#A3.SS2).

### 3.3 Experiment Setup

Since instruction-tuned models exhibit more stable and coherent trait profiles (shown in RQ1), we evaluate the 12 instruction-tuned models listed in Table [1](https://arxiv.org/html/2509.03730v2#S2.T1) on our five behavioral tasks.
We follow the same evaluation procedure as in RQ1: for each task, we test across three default system prompts, three temperature settings, and three random seeds, resulting in 27 generations per condition.

### 3.4 Statistical Analysis

For each LLM and each behavioral task, we fit a mixed-effects model with self-reported traits (e.g., openness, extraversion, self-regulation) as fixed effects and random intercepts for *temperature* and *persona prompt* to account for repeated generations and clustering.
From the fitted models, we take the fixed-effect coefficients and compute a per–trait–task alignment indicator equal to 1 if the coefficient’s sign matches the a priori human-expected direction and 0 otherwise.
We then aggregate these binary indicators by taking their mean at the desired level (per model, per task, or per trait), where 100% indicates perfect alignment, 50% indicates chance-level alignment, and values below 50% indicate systematic misalignment.
We report these aggregated point estimates as means with 95% confidence intervals obtained via a clustered nonparametric bootstrap with 2,000 replicates, resampling the relevant unit of variation (traits when aggregating across traits; tasks when aggregating across tasks) to account for within-model dependence. Further details are provided in Appendix [C.1](https://arxiv.org/html/2509.03730v2#A3.SS1).

### 3.5 Results

Figure: Figure 3: Alignment Between LLMs and Humans Across Personality Traits, Behavioral Tasks, and Model Types. Each panel shows the percentage of cases where LLM self-reports were directionally aligned with behavioral task in accordance with directions expected from human subjects (*Achieved alignment*, colored bars), with the remaining proportion indicating the *Gap to 100%* (light shading). The first panel summarizes alignment in expected association between self-reports and behavioral tasks by self-reported personality traits, the second by behavioral task, and the third by model name, grouped by model family and ordered by increasing parameter size. Percentages above bars indicate the exact alignment proportion. Line at 50% represents random behavior (i.e., % alignment expected by chance). Error bars represent 95% confidence intervals (CIs).
Refer to caption: figures/align_plot_error_bars.png

We find that LLMs’ stable self-reported personality traits do not consistently predict behavior in downstream tasks, and when significant associations emerge, they often diverge from established human behavioral patterns (Figure [3](https://arxiv.org/html/2509.03730v2#S3.F3)).

#### Alignment Across Traits, Tasks and Models.

In Figure [3](https://arxiv.org/html/2509.03730v2#S3.F3), alignment proportions vary across traits, tasks, and models. For personality traits (left), alignment ranges from 45–62%, with *agreeableness* showing the highest alignment (62%) and *neuroticism* the lowest (45%). In all cases, the estimated 95% CIs overlap with 50% level expected by chance under random directional alignment. Behavioral tasks (middle) show even more uniform scores across dimensions, typically between 45–57%. Model-level results (right) reveal that the alignment for most model is no better than chance (e.g., 43–50% for smaller LLaMA and Qwen models). Larger models show somewhat higher alignment (e.g., 64% for Claude-3.7, 68% for GPT-4o, and 82% for Qwen-235B), but except for the largest Qwen model, the CIs overlap with chance. These patterns suggest no alignment between self-report vs. behavior associations for all small to medium sized LLMs, and only modest levels of alignment for some of the biggest LLMs. We do note a higher alignment for Qwen-235B that reached statistical significance.

Figure: Figure 4: Alignment based on Mixed-Effects Models estimating LLM Personality Trait Effects on Task Behavior. Each panel shows mixed-effects model coefficients for LLMs’ self-reported personality traits predicting behavior across five tasks, with results presented for all models, small models, large models, the LLaMA family, and the Qwen family. Blue cells indicate effects aligned with human expectations, while red cells indicate effects in the opposite direction. Split diagonal cells mark cases where human expectations are unclear; blue is on top for positive coefficients and on the bottom for negative. Color intensity reflects effect magnitude, with darker shades indicating stronger effects. Significance is denoted as ^† $p<0.1$, * $p<0.05$, ** $p<0.01$, and *** $p<0.001$. The detailed numerical values are provided in Table [3](https://arxiv.org/html/2509.03730v2#A3.T3) in the Appendix [C](https://arxiv.org/html/2509.03730v2#A3).
Refer to caption: x2.png

#### Alignment Patterns Within Behavioral Tasks.

The heatmap in Figure [4](https://arxiv.org/html/2509.03730v2#S3.F4) visualizes further details. The alignment (blue) and misalignment (red) is shown within each behavioral task group.
The results are also grouped by *Small* and *Large* models and by *Qwen* and *LLaMA* families for which we have 4 individual LLMs of varying sizes.
We observe local, non-systematic patterns of partial alignment between self-reported *Openness* and behavioral tasks around *Stereotyping*, *Self-Reflective Honesty*, and *Sycophancy* (uniformly blue columns), though effects rarely reach statistical significance.
For *Epistemic Honesty* we observe alignment with self-reported *Extroversion*, *Neuroticism*, and *Self-regulation* (uniformly blue columns), but again with few statistically significant associations.
At the LLM-family level, *Qwen family* uniquely displays consistent alignment of all self-reported traits with *Self-Reflective Honesty*.
Still, these results underscore that *alignment patterns are rare and inconsistent*, with both alignment and misalignment varying across traits, tasks, and architectures.

These results highlight that *LLMs’ self-reported traits rarely translate into behavior–alignment hovers near chance for small–mid models and is sporadic even for frontier ones* (with only a narrow, isolated exception). This dissociation between linguistic self-presentation and action limits behavioral controllability and weakens questionnaires as proxies for downstream behavior.

## 4 RQ3: Controllability

RQ2 revealed that LLMs exhibit stable and coherent self-reported personality traits, but these do not reliably predict behavior in downstream tasks.
When associations are statistically significant, they frequently diverge from patterns observed in human behavioral psychology.
This suggests a fundamental disjunction: unlike humans, LLMs lack intrinsic goals, motivations, or consistent internal states, and their behavior appears more contingent on prompt structure and context than on stable traits.
*Instructional alignment may shape self-reports, but this alignment is often superficial.*
For example, a model that self-reports low risk-taking may still act inconsistently in decision-making contexts.
Such inconsistencies highlight the fragility of LLM personality expressions and suggest that self-reports alone are poor indicators of behavioral tendencies. Given this, we ask: if self-reports are unreliable, can we instead control behavior more directly? Specifically, can targeted interventions—such as persona injection—shape both trait self-reports and real-world task behaviors in more human-like and consistent ways?

###### Research Question 3 (Control) .

How do intervention methods (e.g., persona injection) influence self-reported trait profiles and their behavioral manifestations?

###### Research Question 3 (Control) .

### 4.1 Experiment Setup

To evaluate our research question, we replicate RQ1 and RQ2 procedures, using the BFI and SRQ questionnaires for self-reports and two behavioral tasks—sycophancy and risk-taking—that showed the most counterintuitive patterns in RQ2. While self-regulation is typically linked to reduced risk-taking in humans (Duell et al., 2016), and agreeableness predicts sycophantic tendencies (Nettle & Liddle, 2008), these associations were weak or absent in RQ2.

Instead of default personas, we introduce trait-specific personas to test whether explicit personality prompting enhances alignment between self-reports and behavior. We conduct two experiments: (1) Agreeableness Persona, assessing its impact on self-reported traits and sycophantic behavior; and (2) Self-Regulation Persona, evaluating effects on self-reports and risk-taking behavior. Personas are constructed by sampling representative trait keywords, following three different prompting strategies established in prior LLM personality research (Jiang et al., 2024a; Serapio-Garc´ıa et al., 2023; Dash et al., 2025). Implementation details are provided in Table [10](https://arxiv.org/html/2509.03730v2#A8.T10) in the Appendix [H](https://arxiv.org/html/2509.03730v2#A8).

### 4.2 Statistical Analysis

We test whether LLMs exhibit systematic differences in self-reported traits and real-world behaviors before and after trait-specific persona injection. For each of the three prompting strategies, we fit separate binomial logistic regression models to predict persona condition (trait-specific persona vs. default).
For the self-report analysis, all six trait scores are used as predictors. For the behavioral analysis, we use the downstream task performance (sycophancy or risk-taking) as a single predictor.
All predictors are standardized, and within each prompting strategy, we include prompt variation, sampling temperature, and model as control variables. Inference is based on Wald z-statistics and 95% confidence intervals, shown in Figure [5](https://arxiv.org/html/2509.03730v2#S4.F5).

### 4.3 Results

Figure: Figure 5: Trait-Specific Personas Are Detectable via Self-Reports but Not Behavior. Coefficient estimates (95% CI) from logistic regressions predict persona condition (Agreeableness or Self-Regulation vs. Default) using either six self-reported traits or one behavioral measure (sycophancy or risk-taking). Results are shown across three prompting strategies, indicated by color intensity (Appendix [H](https://arxiv.org/html/2509.03730v2#A8)). Significance levels (* $p<0.05$, ** $p<0.01$, *** $p<0.001$, n.s.) are marked on each bar. Across strategies, self-reports reliably reveal persona presence, whereas behavioral measures do not, indicating limited transfer of persona effects to downstream behavior.
Refer to caption: figures/RQ3_c.png

#### Self-Report.

Trait-specific personas lead to strong alignment on their target traits. When injecting the agreeableness persona, logistic regression reveals a significant increase in self-reported agreeableness ($\beta\approx 3.6\text{ to }4.4$, $p<.001$).
Similarly, injecting the self-regulation persona results in a significant increase in self-reported self-regulation ($\beta\approx 2.2\text{ to }2.9$, $p<.05$).
These results confirm that self-reported traits reliably reflect the intended persona in self-report scenarios.

However, the inter-trait relationships do not fully align with the patterns observed in RQ1 (Figure [2](https://arxiv.org/html/2509.03730v2#S2.F2)), where extraversion, openness, conscientiousness, and agreeableness were meaningfully positively correlated, and neuroticism was negatively associated.
In contrast, we find that injecting agreeableness produces an inconsistent effect on self-regulation ($\beta\approx-0.44\text{ to }0.50$, some n.s., up to $p<.05$), while injecting self-regulation reduces agreeableness ($\beta\approx-1.1\text{ to }-1.8$, $p<.05$) and openness ($\beta\approx-2.2\text{ to }-2.8$, $p<.001$).
Additionally, the self-regulation persona has little and often non-significant effect on neuroticism or extraversion.
Notably, conscientiousness shows a strong and significant increase when the self-regulation persona is applied ($\beta\approx 4.2\text{ to }4.8$, $p<.001$), exceeding even the effect on self-regulation itself.

#### Behavioral Task.

In contrast to the strong alignment observed in self-reports, behavioral measures show limited sensitivity to persona injection.
When using downstream behavior to predict whether a persona was applied, logistic regression models yield mostly non-significant results for both cases.
Specifically, sycophantic responses provide weak and inconsistent evidence for predicting whether the agreeableness persona was used ($\beta\approx-0.05$ to $0.32$, n.s. to $p<.001$), and risk-taking behavior similarly fails to reliably distinguish the self-regulation condition ($\beta\approx-0.14$ to $0.20$, n.s.).

These findings suggest that while LLMs exhibit clear changes in how they self-report personality traits under different personas, those changes do not consistently manifest in behavior.
The weak predictive power of real-world tasks highlights a key limitation in the behavioral controllability of LLMs: surface-level trait alignment does not necessarily translate to deeper, goal-driven consistency.
This points to a dissociation between linguistic self-presentation and action-oriented decision behavior.

## 5 Discussion

Our study reveals a notable gap between surface-level trait expression and actual behavior in LLMs.
Although instruction tuning and persona prompts stabilize self-reported traits, these do not reliably translate to consistent downstream behavior.
This challenges the view of LLMs as behaviorally grounded and suggests that current alignment methods favor linguistic plausibility over functional reliability.
We discuss this dissociation across three dimensions: (1) linguistic–behavioral divergence, (2) diagnosis through psychologically grounded frameworks, and (3) the illusion of coherence created by current alignment and prompting.

#### Linguistic-Behavioral Dissociation in LLMs.

Our findings highlight a dissociation between linguistic self-expression and behavioral consistency in LLMs.
While LLMs can simulate personality traits through language (Cao & Kosiński, 2023), these traits likely arise from surface-level pattern matching rather than internalized motivations—unlike human personality, which is grounded in cognitive and affective processes (McCrae & John, 1992).
Moreover, LLMs lack temporal consistency and exhibit high prompt sensitivity (Bodroža et al., 2024a).
This disconnect is further supported by recent findings that survey-based evaluations—though often linguistically coherent—fail to predict open-ended model behavior or reflect genuine psychological dispositions (Röttger et al., 2024; Dominguez-Olmedo et al., 2024).
Such dissociation cautions against interpreting linguistic coherence as evidence of cognitive or behavioral depth, particularly in sensitive domains like mental health (Treder et al., 2024; Fedorenko et al., 2024; Heston, 2023).

#### Testing with a Psychologically Grounded Framework.

Data contamination is a well-recognized issue in LLM evaluation, and one might worry that models trained on broad human data have already encountered the kinds of questionnaires and tasks we use.
However, our framework is tested with a different goal: instead of assessing LLMs’ particular knowledge set, we test whether they can organize knowledge coherently.
This distinction is critical.
(1) Even if an LLM has been exposed to these tasks or related materials (e.g., personality-relevant information) during training, exposure alone does not enable it to form coherent mappings between knowledge and behavior–and our results show that such coherence is clearly lacking, a limitation that traditional open benchmarks cannot reveal.
(2) Unlike open benchmarks or explicit goals (e.g., math ability), which often become optimization targets for LLM training, the tasks we adapt were rarely used as such goals during training and thus better reveal genuine shortcomings (Hasan et al., 2025; Sainz et al., 2023; Zhou et al., 2025).
(3) Finally, in RQ3 we show that the dissociation between surface-level knowledge and coherent behavior persists across perturbations and prompting strategies, underscoring the robustness of our findings.

#### Illusions of Coherence through Alignment and Prompting.

Our results show that alignment methods such as RLHF or DPO, as well as persona-based prompting, can stabilize linguistic self-reports and modulate surface-level identity expression. However, these interventions do not reliably translate into deeper behavioral regularity. Instruction-tuned models remain highly sensitive to superficial prompt variations and cultural framings (Khan et al., 2025), while persona effects often degrade over extended interactions (Raj et al., 2024). In practice, models may produce responses that appear psychologically plausible or socially aligned (Peters & Matz, 2024; Holmes et al., 2024), yet lack the underlying stability and intentionality needed for consistent behavior (Lee et al., 2021). This gap highlights that current alignment techniques shape outputs rather than dispositions, creating an illusion of coherence without genuine behavioral grounding.

#### Toward Behaviorally-Grounded Alignment.

To move beyond surface-level coherence, future alignment work should explicitly target behavioral regularity.
One promising direction is a potential for reinforcement learning from behavioral feedback (RLBF), where models are rewarded based on consistent performance in psychologically grounded tasks—e.g., maintaining honesty under uncertainty or resisting social conformity—rather than on text fluency alone.
Another is the development of behaviorally evaluated checkpoints, assessing models not just via linguistic benchmarks but through temporal stability and context-consistent behavior across interaction sequences.
Finally, deeper alignment may require interventions at the representational level, such as modifying latent activations or embedding spaces to reflect specific behavioral traits (Serapio-Garc´ıa et al., 2023; Cao & Kosiński, 2023).
These strategies could help shift alignment efforts from shaping model outputs to shaping model dispositions—crucial for deploying LLMs in settings where functional reliability matters.

## 6 Conclusion

Our study provides a first step toward a comprehensive behavioral examination of human-like traits in LLMs, revealing a critical dissociation between linguistic self-expression and behavioral consistency.
While instruction tuning induces stable and psychologically coherent self-reports, these traits only weakly predict downstream behavior, and persona interventions fail to produce robust behavioral change.
The findings challenge the assumption that self-reported traits reflect internal alignment and suggest that current alignment strategies primarily shape surface-level outputs.
Future work shall move beyond textual coherence to evaluate deeper, behaviorally grounded model traits.

## 7 Limitations and Future Work

We highlight several limitations of this work and potential directions for future exploration.
First, the self-report part of our study focuses on the Big Five Inventory (BFI) due to its widespread use, interpretability, and established links to real-world psychological and behavioral tasks.
Still, alternative survey frameworks such as HEXACO are also compatible and may certain introduce additional dimensions for analysis (Bhandari et al., 2025).
Beyond personality inventories, complete motivational frameworks such as Schwartz’s Basic Human Values (PVQ-RR) can be incorporated to elicit value priorities and test their behavioral expression; these provide a complementary lens on model “goals” that is theoretically related—but not reducible—to traits (Schwartz, 1992).
Future work should apply the research methods in this work, to probe wider self-report surveys and their potential behavioral manifestations.
Second, our analysis is in mainstream transformer-based, non-reasoning models.
Recent research has demonstrated the strengths of alternative architectures (Gu & Dao, 2023) as well as emerging similarities between reasoning models and human cognition (de Varda et al., 2025).
Future work should extend these evaluations to reasoning models and other architectures such as Mamba and Mixture-of-Experts (MoE), to investigate whether the personality illusion discovered in this work transfers there.
Last, we examine four well-designed behavioral tasks in this study, chosen for their importance to real-world LLM applications and their established connection to personality traits.
Given the growing attention to machine behavior (Rahwan et al., 2019a), we encourage closer collaboration between psychologists and computer scientists to design additional high-quality behavioral tasks tailored to LLMs, thereby enriching insights within this framework.

## 8 Background and Related Work

#### LLM Anthropomorphism & Personalities.

Historically, research on LLMs – and AI systems more broadly – has been guided by analogies to the human brain (Hassabis et al., 2017; Zhao et al., 2023).
This framing continues to shape contemporary work, fueling LLM anthropomorphism: attempts to identify human-like characteristics in models’ language, behavior, and reasoning (Xiao et al., 2025; Epley et al., 2007).
When approached with care, anthropomorphism can deepen human understanding of LLMs, suggest directions of improvement, and inspire better systems of human-AI interaction (Ma et al., 2025; Waytz et al., 2014; Xie et al., 2023).
At the same time, recent work warns against over-anthropomorphism (Ibrahim & Cheng, 2025; Shanahan, 2023; Placani, 2024), especially in real-world, applied settings (Schaaff & Heidelmann, 2024; Ibrahim et al., 2025).
Over-anthropomorphism risks miscalibrating users’ trust (Mireshghallah et al., 2024; Cohn et al., 2024; Sun & Wang, 2025), fostering misconceptions about capabilities (Steyvers et al., 2025), or even encouraging emotional over-reliance on AI systems (Akbulut et al., 2025; Zhou et al., 2024; Shunsen et al., 2024).
Given this two-sidedness of LLM anthropomorphism (Reinecke et al., 2025; Peter et al., 2025), a central fundamental question arises: do LLMs in fact exhibit stable human-like traits – or “personalities” – at all?

#### Measuring LLM Personalities.

To explore this question, early work adapted established psychological self-report inventories such as the Big Five Survey (John et al., 1991) to LLMs, finding that the resulting profiles often resembled human norms under certain conditions (Miotto et al., 2022; Huang et al., 2023; Wang et al., 2024c; Serapio-García et al., 2025).
This initial finding motivated larger-scale studies, which show that different LLM families generally display consistent but distinct personalities (Lee et al., 2025; tse Huang et al., 2024a; b), while still struggling with more nuanced traits such as emotional reasoning (Huang et al., 2024).
However, such apparent “personalities” remain fragile: small variations in temperature, random seed, or context can yield substantial shifts in trait scores, undermining stability across diverse real-world cases (Bodroža et al., 2024b; Li et al., 2025b).
Moreover, LLMs frequently default to socially desirable profiles, e.g. scoring unusually high on agreeableness and low on neuroticism, reflecting a bias toward positive stereotypes rather than neutral personality baselines (Bodroža et al., 2024b; Salecha et al., 2024).
While these studies provide important insights into how LLMs align with or diverge from human personality constructs, they rely heavily on self-report measures.
This raises questions about the reliability of such responses (Zou et al., 2025; Turpin et al., 2023) and whether they meaningfully transfer to real-world, interactive scenarios.

#### Controlling LLM Personalities.

Beyond merely measuring intrinsic traits, researchers have increasingly turned to controlling them, through persona injection: steering an LLM to adopt a specified character or profile (Zhang et al., 2018; Tseng et al., 2024a; Chen et al., 2024a).
Two main paradigms dominate: (1) role-playing, where an LLM simulates a persona (e.g. “a doctor” or “Shakespeare”) (Li et al., 2023; Park et al., 2023; Shanahan et al., 2023), and (2) personalization, where responses are adapted to the user’s own profile (Liu et al., 2025a; Zollo et al., 2025; Chen et al., 2024b).
Approaches vary in mechanism.
Prompt-based techniques range from lightweight prefix instructions to persona-augmented context descriptions (Nighojkar et al., 2025; Kamruzzaman & Kim, 2025; Zheng et al., 2024b).
Training-based methods, by contrast, adjust parameters directly, such as fine-tuning models on trait-annotated dialogues to induce Big Five profiles (Li et al., 2025a; Ji et al., 2025b).
More recently, researchers propose latent-control approaches: persona vectors that identify interpretable directions in activation space (e.g. sycophancy, hallucination) and can be toggled at inference (Chen et al., 2025), or direct activation interventions that align outputs to desired personality profiles (Zhu et al., 2025; Panickssery et al., 2024).
Empirical evaluations confirm that LLMs can convincingly role-play distinct characters (Wang et al., 2025b; Cao & Kosinski, 2024a; Wang et al., 2024b; Cao & Kosinski, 2024b), explicit enough that humans are often able to recognize the intended personas (Jiang et al., 2024b).
Still, these abilities degrade as personas grow more complex or nuanced (Wang et al., 2025b; Zheng et al., 2024a).
Persona injection has also been applied to downstream tasks, enabling models to adopt personas better suited for domain-specific applications (Tan et al., 2024; Olea et al., 2024; He, 2024), yet such applications often prioritize performance metrics over careful evaluation of whether the persona injection itself is effective.

#### Psychology of AI & Machine Psychology.

Zooming out toward a broader picture, as AI systems are aligned to be more human-like in their language and reasoning, researchers have begun treating them as subjects of psychological inquiry, giving rise to an emergent field of “machine psychology” or “AI psychology” (Hagendorff et al., 2024; Rahwan et al., 2019b).
This perspective urges going beyond traditional performance benchmarks to ask: how can we use tools from psychology to probe and understand the behavioral and cognitive patterns of AI models?
Current approaches center around applying human psychological experiments – such as theory-of-mind tasks (Kosinski, 2024; van Duijn et al., 2023; Kim et al., 2023; Pi et al., 2024), reasoning biases (Lampinen et al., 2024; Han et al., 2024b; O’Leary, 2025; Yu et al., 2024), and moral judgment scenarios (Ji et al., 2025a; Garcia et al., 2024; Takemoto, 2024) – to LLMs, to reveal emergent capacities (Wei et al., 2022) and understand failure modes (Song et al., 2025) of LLMs that are otherwise not obvious from standard NLP tasks (Bubeck et al., 2023; Binz & Schulz, 2023a; Shiffrin & Mitchell, 2023; Hernández-Orallo et al., 2014).
Designing these experiments require significant caution to ensure validity, as many psychological tasks carry implicit assumptions and cultural context that do not cleanly transfer to machines (Pellert et al., 2024; Löhn et al., 2024), and LLM-specific concerns arise, including potential training-data contamination, the absence of lived experience, and the need for ensuring reliability of measures (Pellert et al., 2024; Mitchell & Krakauer, 2023).
Looking forward, machine psychology should combine behavioral experiments with interpretability methods (Wang et al., 2025a; Lindsey et al., 2025), so as to link observed behaviors to underlying model mechanisms and better explain why LLMs succeed or fail in ways that resemble – or diverge from – human cognition.

## 9 Acknowledgment

This work is supported by the Caltech Linde Center for Science, Society, and Public Policy (LCSSP).
Anima Anandkumar is Bren Professor of Computing and Mathematical Sciences at Caltech.
R. Michael Alvarez is Flintridge Foundation Professor of Political and Computational Social Science at Caltech.

## References

- Akbulut et al. (2025)
Canfer Akbulut, Laura Weidinger, Arianna Manzini, Iason Gabriel, and Verena Rieser.
All too human? mapping and mitigating the risks from anthropomorphic ai.
In *Proceedings of the 2024 AAAI/ACM Conference on AI, Ethics, and Society*, AIES ’24, pp.  13–26. AAAI Press, 2025.
- Alfano et al. (2017)
Mark Alfano, Kathryn Iurino, Paul Stey, Brian Robinson, Markus Christen, Feng Yu, and Daniel Lapsley.
Development and validation of a multi-dimensional measure of intellectual humility.
*PloS one*, 12(8):e0182950, 2017.
- Allen et al. (2010)
Thomas J Allen, Jeffrey W Sherman, and Karl Christoph Klauer.
Social context and the self-regulation of implicit bias.
*Group Processes & Intergroup Relations*, 13(2):137–149, 2010.
- Amiri & Navab (2018)
Sohrab Amiri and Amir Ghasemi Navab.
The association between the adaptive/maladaptive personality dimensions and emotional regulation.
*Neuropsychiatria i Neuropsychologia/Neuropsychiatry and Neuropsychology*, 13(1):1–8, 2018.
- Asch (1956)
Solomon E Asch.
Studies of independence and conformity: I. a minority of one against a unanimous majority.
*Psychological monographs: General and applied*, 70(9):1, 1956.
- Bai et al. (2024)
Xuechunzi Bai, Angelina Wang, Ilia Sucholutsky, and Thomas L Griffiths.
Measuring implicit bias in explicitly unbiased large language models.
*arXiv preprint arXiv:2402.04105*, 2024.
- Bąk et al. (2022)
Wacław Bąk, Bartosz Wójtowicz, and Jan Kutnik.
Intellectual humility: an old problem in a new psychological perspective.
*Current Issues in Personality Psychology*, 10(2):85–97, 2022.
- Barrick et al. (2005)
Murray R Barrick, Laura Parks, and Michael K Mount.
Self-monitoring as a moderator of the relationships between personality traits and performance.
*Personnel psychology*, 58(3):745–767, 2005.
- Ben-Zeev et al. (2005)
Talia Ben-Zeev, Steven Fein, and Michael Inzlicht.
Arousal and stereotype threat.
*Journal of experimental social psychology*, 41(2):174–181, 2005.
- Bhandari et al. (2025)
Pranav Bhandari, Usman Naseem, Amitava Datta, Nicolas Fay, and Mehwish Nasim.
Evaluating personality traits in large language models: Insights from psychological questionnaires.
*arXiv preprint arXiv:2502.05248*, 2025.
- Bhatia (2024)
Sudeep Bhatia.
Exploring variability in risk taking with large language models.
*Journal of Experimental Psychology: General*, 153(7):1838, 2024.
- Bidjerano & Dai (2007)
Temi Bidjerano and David Yun Dai.
The relationship between the big-five model of personality and self-regulated learning strategies.
*Learning and individual differences*, 17(1):69–81, 2007.
- Binz & Schulz (2023a)
Marcel Binz and Eric Schulz.
Using cognitive psychology to understand gpt-3.
*Proceedings of the National Academy of Sciences*, 120(6), February 2023a.
ISSN 1091-6490.
doi: 10.1073/pnas.2218523120.
URL [http://dx.doi.org/10.1073/pnas.2218523120](http://dx.doi.org/10.1073/pnas.2218523120).
- Binz & Schulz (2023b)
Marcel Binz and Eric Schulz.
Using cognitive psychology to understand gpt-3.
*Proceedings of the National Academy of Sciences*, 120(6):e2218523120, 2023b.
- Bodroža et al. (2024a)
B. Bodroža, B. Dinić, and L. Bojić.
Personality testing of large language models: limited temporal stability, but highlighted prosociality.
*Royal Society Open Science*, 11(10), 2024a.
doi: 10.1098/rsos.240180.
- Bodroža et al. (2024b)
Bojana Bodroža, Bojana M. Dinić, and Ljubiša Bojić.
Personality testing of large language models: limited temporal stability, but highlighted prosociality.
*Royal Society Open Science*, 11(10), October 2024b.
ISSN 2054-5703.
doi: 10.1098/rsos.240180.
URL [http://dx.doi.org/10.1098/rsos.240180](http://dx.doi.org/10.1098/rsos.240180).
- Brown et al. (1999)
Janice M Brown, William R Miller, and Lauren A Lawendowski.
The self-regulation questionnaire.
*Innovations in clinical practice: A source book*, 1999.
- Bubeck et al. (2023)
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang.
Sparks of artificial general intelligence: Early experiments with gpt-4, 2023.
URL [https://arxiv.org/abs/2303.12712](https://arxiv.org/abs/2303.12712).
- Buratti et al. (2013)
Sandra Buratti, Carl Martin Allwood, and Sabina Kleitman.
First-and second-order metacognitive judgments of semantic memory reports: The influence of personality traits and cognitive styles.
*Metacognition and learning*, 8(1):79–102, 2013.
- Byerly (2023)
T Ryan Byerly.
Intellectual honesty and intellectual transparency.
*Episteme*, 20(2):410–428, 2023.
- Cao & Kosiński (2023)
X. Cao and M. Kosiński.
Large language models know how the personality of public figures is perceived by the general public.
*OSF Preprints*, 2023.
doi: 10.31234/osf.io/89hx6.
- Cao & Kosinski (2024a)
Xubo Cao and Michal Kosinski.
Large language models know how the personality of public figures is perceived by the general public.
*Scientific Reports*, 14, 03 2024a.
doi: 10.1038/s41598-024-57271-z.
- Cao & Kosinski (2024b)
Xubo Cao and Michal Kosinski.
Large language models know how the personality of public figures is perceived by the general public.
*Scientific Reports*, 14, 03 2024b.
doi: 10.1038/s41598-024-57271-z.
- Caspi et al. (2005)
Avshalom Caspi, Brent W Roberts, and Rebecca L Shiner.
Personality development: Stability and change.
*Annu. Rev. Psychol.*, 56:453–484, 2005.
- Chen et al. (2024a)
Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, and Yanghua Xiao.
From persona to personalization: A survey on role-playing language agents, 2024a.
URL [https://arxiv.org/abs/2404.18231](https://arxiv.org/abs/2404.18231).
- Chen et al. (2024b)
Jin Chen, Zheng Liu, Xu Huang, Chenwang Wu, Qi Liu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Xiaolong Chen, Xingmei Wang, Kai Zheng, Defu Lian, and Enhong Chen.
When large language models meet personalization: perspectives of challenges and opportunities.
*World Wide Web*, 27(4), June 2024b.
ISSN 1573-1413.
doi: 10.1007/s11280-024-01276-1.
URL [http://dx.doi.org/10.1007/s11280-024-01276-1](http://dx.doi.org/10.1007/s11280-024-01276-1).
- Chen et al. (2025)
Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey.
Persona vectors: Monitoring and controlling character traits in language models, 2025.
URL [https://arxiv.org/abs/2507.21509](https://arxiv.org/abs/2507.21509).
- Chen et al. (2024c)
Wei Chen, Zhen Huang, Liang Xie, Binbin Lin, Houqiang Li, Le Lu, Xinmei Tian, Deng Cai, Yonggang Zhang, Wenxiao Wan, et al.
From yes-men to truth-tellers: addressing sycophancy in large language models with pinpoint tuning.
In *Proceedings of the 41st International Conference on Machine Learning*, pp.  6950–6972, 2024c.
- Cheng et al. (2025)
Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim, and Dan Jurafsky.
Social sycophancy: A broader understanding of llm sycophancy.
*arXiv preprint arXiv:2505.13995*, 2025.
- Christensen et al. (2014)
Julia F Christensen, Albert Flexas, Margareta Calabrese, Nadine K Gut, and Antoni Gomila.
Moral judgment reloaded: a moral dilemma validation study.
*Frontiers in psychology*, 5:607, 2014.
- Cohn et al. (2024)
Michelle Cohn, Mahima Pushkarna, Gbolahan O Olanubi, Joseph M Moran, Daniel Padgett, Zion Mengesha, and Courtney Heldreth.
Believing anthropomorphism: examining the role of anthropomorphic cues on trust in large language models.
In *Extended Abstracts of the CHI Conference on Human Factors in Computing Systems*, pp.  1–15, 2024.
- Craig et al. (2020)
Kym Craig, Daniel Hale, Catherine Grainger, and Mary E Stewart.
Evaluating metacognitive self-reports: systematic reviews of the value of self-report in metacognitive research.
*Metacognition and Learning*, 15(2):155–213, 2020.
- Crawford & Brandt (2019)
Jarret T Crawford and Mark J Brandt.
Who is prejudiced, and toward whom? the big five traits and generalized prejudice.
*Personality and Social Psychology Bulletin*, 45(10):1455–1467, 2019.
- Dash et al. (2025)
Saloni Dash, Amélie Reymond, Emma S Spiro, and Aylin Caliskan.
Persona-assigned large language models exhibit human-like motivated reasoning.
*arXiv preprint arXiv:2506.20020*, 2025.
- De Ridder et al. (2012)
Denise TD De Ridder, Gerty Lensvelt-Mulders, Catrin Finkenauer, F Marijn Stok, and Roy F Baumeister.
Taking stock of self-control: A meta-analysis of how trait self-control relates to a wide range of behaviors.
*Personality and social psychology review*, 16(1):76–99, 2012.
- de Varda et al. (2025)
Andrea de Varda, Ferdinando D’Elia, Evelina Fedorenko, and Andrew Lampinen.
The cost of thinking is similar between large reasoning models and humans, 07 2025.
- De Vries et al. (2011)
Anita De Vries, Reinout E de Vries, and Marise Ph Born.
Broad versus narrow traits: Conscientiousness and honesty–humility as predictors of academic criteria.
*European Journal of Personality*, 25(5):336–348, 2011.
- DeYoung et al. (2002)
Colin G DeYoung, Jordan B Peterson, and Daniel M Higgins.
Higher-order factors of the big five predict conformity: Are there neuroses of health?
*Personality and Individual differences*, 33(4):533–552, 2002.
- Digman (1997)
John M Digman.
Higher-order factors of the big five.
*Journal of personality and social psychology*, 73(6):1246, 1997.
- Dominguez-Olmedo et al. (2024)
Ricardo Dominguez-Olmedo, Moritz Hardt, and Celestine Mendler-Dünner.
Questioning the survey responses of large language models.
*Advances in Neural Information Processing Systems*, 37:45850–45878, 2024.
- Duell et al. (2016)
Natasha Duell, Laurence Steinberg, Jason Chein, Suha M Al-Hassan, Dario Bacchini, Chang Lei, Nandita Chaudhary, Laura Di Giunta, Kenneth A Dodge, Kostas A Fanti, et al.
Interaction of reward seeking and self-regulation in the prediction of risk taking: A cross-national test of the dual systems model.
*Developmental psychology*, 52(10):1593, 2016.
- Duru & Günçavdı-Alabay (2024)
Hazel Duru and Gizem Günçavdı-Alabay.
Psychological counselor candidates’ leadership self-efficacy: Personality traits, cognitive flexibility, and emotional intelligence.
*Base for Electronic Educational Sciences*, 5(2):1–17, 2024.
doi: 10.29329/bedu.2024.1064.1.
- Ekehammar et al. (2004)
Bo Ekehammar, Nazar Akrami, Magnus Gylje, and Ingrid Zakrisson.
What matters most to prejudice: Big five personality, social dominance orientation, or right-wing authoritarianism?
*European journal of personality*, 18(6):463–482, 2004.
- Epley et al. (2007)
Nicholas Epley, Adam Waytz, and John T. Cacioppo.
On seeing human: a three-factor theory of anthropomorphism.
*Psychological review*, 114 4:864–86, 2007.
URL [https://api.semanticscholar.org/CorpusID:6733517](https://api.semanticscholar.org/CorpusID:6733517).
- Fedorenko et al. (2024)
Evelina Fedorenko, Steven T Piantadosi, and Edward AF Gibson.
Language is primarily a tool for communication rather than thought.
*Nature*, 630(8017):575–586, 2024.
- Figner et al. (2009)
Bernd Figner, Rachael J Mackinlay, Friedrich Wilkening, and Elke U Weber.
Affective and deliberative processes in risky choice: age differences in risk taking in the columbia card task.
*Journal of Experimental Psychology: Learning, Memory, and Cognition*, 35(3):709, 2009.
- Flynn (2005)
Francis J Flynn.
Having an open mind: the impact of openness to experience on interracial attitudes and impression formation.
*Journal of personality and social psychology*, 88(5):816, 2005.
- Gailliot et al. (2007)
Matthew T Gailliot, Roy F Baumeister, C Nathan DeWall, Jon K Maner, E Ashby Plant, Dianne M Tice, Lauren E Brewer, and Brandon J Schmeichel.
Self-control relies on glucose as a limited energy source: willpower is more than a metaphor.
*Journal of personality and social psychology*, 92(2):325, 2007.
- Gao et al. (2020)
Yifan Gao, Vicente A González, and Tak Wing Yiu.
Exploring the relationship between construction workers’ personality traits and safety behavior.
*Journal of construction engineering and management*, 146(3):04019111, 2020.
- Garcia et al. (2024)
Basile Garcia, Crystal Qian, and Stefano Palminteri.
The moral turing test: Evaluating human-llm alignment in moral decision-making.
*arXiv preprint arXiv:2410.07304*, 2024.
- Graziano & Tobin (2002)
William G Graziano and Renee M Tobin.
Agreeableness: Dimension of personality or social desirability artifact?
*Journal of Personality*, 70(5):695–728, 2002.
doi: 10.1111/1467-6494.05021.
- Greenwald et al. (1998)
Anthony G Greenwald, Debbie E McGhee, and Jordan LK Schwartz.
Measuring individual differences in implicit cognition: the implicit association test.
*Journal of personality and social psychology*, 74(6):1464, 1998.
- Gu & Dao (2023)
Albert Gu and Tri Dao.
Mamba: Linear-time sequence modeling with selective state spaces.
*arXiv preprint arXiv:2312.00752*, 2023.
- Gullone & Moore (2000)
Eleonora Gullone and Susan Moore.
Adolescent risk-taking and the five-factor model of personality.
*Journal of Adolescence*, 23:393–407, 2000.
doi: 10.1006/jado.2000.0327.
URL [https://doi.org/10.1006/jado.2000.0327](https://doi.org/10.1006/jado.2000.0327).
- Gupta et al. (2023)
Akshat Gupta, Xiaoyang Song, and Gopala Anumanchipalli.
Self-assessment tests are unreliable measures of llm personality.
*arXiv preprint arXiv:2309.08163*, 2023.
- Guzman & Espejo (2015)
Felipe A Guzman and Alvaro Espejo.
Dispositional and situational differences in motives to engage in citizenship behavior.
*Journal of Business Research*, 68(2):208–215, 2015.
- Hagendorff et al. (2024)
Thilo Hagendorff, Ishita Dasgupta, Marcel Binz, Stephanie C. Y. Chan, Andrew Lampinen, Jane X. Wang, Zeynep Akata, and Eric Schulz.
Machine psychology, 2024.
URL [https://arxiv.org/abs/2303.13988](https://arxiv.org/abs/2303.13988).
- Haggard et al. (2018)
Megan Haggard, Wade C Rowatt, Joseph C Leman, Benjamin Meagher, Courtney Moore, Thomas Fergus, Dennis Whitcomb, Heather Battaly, Jason Baehr, and Dan Howard-Snyder.
Finding middle ground between intellectual arrogance and intellectual servility: Development and assessment of the limitations-owning intellectual humility scale.
*Personality and Individual Differences*, 124:184–193, 2018.
- Han et al. (2024a)
Pengrui Han, Rafal Kocielnik, Adhithya Saravanan, Roy Jiang, Or Sharir, and Anima Anandkumar.
Chatgpt based data augmentation for improved parameter-efficient debiasing of llms.
*arXiv preprint arXiv:2402.11764*, 2024a.
- Han et al. (2024b)
Pengrui Han, Peiyang Song, Haofei Yu, and Jiaxuan You.
In-context learning may not elicit trustworthy reasoning: A-not-B errors in pretrained language models.
In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), *Findings of the Association for Computational Linguistics: EMNLP 2024*, pp.  5624–5643, Miami, Florida, USA, November 2024b. Association for Computational Linguistics.
doi: 10.18653/v1/2024.findings-emnlp.322.
URL [https://aclanthology.org/2024.findings-emnlp.322/](https://aclanthology.org/2024.findings-emnlp.322/).
- Händel et al. (2020)
Marion Händel, Anique BH De Bruin, and Markus Dresel.
Individual differences in local and global metacognitive judgments.
*Metacognition and Learning*, 15(1):51–75, 2020.
- Hart et al. (2015)
Claire M Hart, Timothy D Ritchie, Erica G Hepper, and Jochen E Gebauer.
The balanced inventory of desirable responding short form (bidr-16).
*Sage Open*, 5(4):2158244015621113, 2015.
- Hasan et al. (2025)
Md Najib Hasan, Mohammad Fakhruddin Babar, Souvika Sarkar, Monowar Hasan, and Santu Karmaker.
Pitfalls of evaluating language models with open benchmarks.
*arXiv preprint arXiv:2507.00460*, 2025.
- Hassabis et al. (2017)
Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick.
Neuroscience-inspired artificial intelligence.
*Neuron*, 95(2):245–258, 2017.
ISSN 0896-6273.
doi: https://doi.org/10.1016/j.neuron.2017.06.011.
URL [https://www.sciencedirect.com/science/article/pii/S0896627317305093](https://www.sciencedirect.com/science/article/pii/S0896627317305093).
- He (2024)
Sui He.
Prompting chatgpt for translation: A comparative analysis of translation brief and persona prompts.
*arXiv preprint arXiv:2403.00127*, 2024.
- Hernández-Orallo et al. (2014)
José Hernández-Orallo, David L. Dowe, and M.Victoria Hernández-Lloreda.
Universal psychometrics.
*Cogn. Syst. Res.*, 27(C):50–74, March 2014.
ISSN 1389-0417.
doi: 10.1016/j.cogsys.2013.06.001.
URL [https://doi.org/10.1016/j.cogsys.2013.06.001](https://doi.org/10.1016/j.cogsys.2013.06.001).
- Heston (2023)
T. Heston.
Safety of large language models in addressing depression.
*Cureus*, 2023.
doi: 10.7759/cureus.50729.
- Holmes et al. (2024)
G. Holmes, B. Tang, S. Gupta, S. Venkatesh, H. Christensen, and A. Whitton.
Applications of large language models in the field of suicide prevention: scoping review (preprint).
*JMIR Preprints*, 2024.
doi: 10.2196/preprints.63126.
- Huang et al. (2023)
Jen-Tse Huang, Wenxuan Wang, Man Lam, Eric Li, Wenxiang Jiao, and Michael Lyu.
Chatgpt an enfj, bard an istj: Empirical study on personalities of large language models, 05 2023.
- Huang et al. (2024)
Jen-tse Huang, Man Ho Lam, Eric John Li, Shujie Ren, Wenxuan Wang, Wenxiang Jiao, Zhaopeng Tu, and Michael R. Lyu.
Apathetic or empathetic? evaluating llms'emotional alignments with humans.
In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), *Advances in Neural Information Processing Systems*, volume 37, pp.  97053–97087. Curran Associates, Inc., 2024.
URL [https://proceedings.neurips.cc/paper_files/paper/2024/file/b0049c3f9c53fb06f674ae66c2cf2376-Paper-Conference.pdf](https://proceedings.neurips.cc/paper_files/paper/2024/file/b0049c3f9c53fb06f674ae66c2cf2376-Paper-Conference.pdf).
- Huang & Hadfi (2025)
Yin Jou Huang and Rafik Hadfi.
Beyond self-reports: Multi-observer agents for personality assessment in large language models.
*arXiv preprint arXiv:2504.08399*, 2025.
- Hurtz & Donovan (2000)
Gregory M Hurtz and John J Donovan.
Personality and job performance: The big five revisited.
*Journal of Applied Psychology*, 85(6):869–879, 2000.
doi: 10.1037/0021-9010.85.6.869.
- Huynh et al. (2025)
Ho Phi Huynh, Zhicheng Luo, Elisa Eche, Jasmyne Thomas, Dawn R Weatherford, and Malin K Lilley.
Associations between intellectual humility, academic motivation, and academic self-efficacy.
*Psychological Reports*, pp.  00332941251351243, 2025.
- Ibrahim & Cheng (2025)
Lujain Ibrahim and Myra Cheng.
Thinking beyond the anthropomorphic paradigm benefits llm research.
*arXiv preprint arXiv:2502.09192*, 2025.
- Ibrahim et al. (2025)
Lujain Ibrahim, Canfer Akbulut, Rasmi Elasmar, Charvi Rastogi, Minsuk Kahng, Meredith Ringel Morris, Kevin R. McKee, Verena Rieser, Murray Shanahan, and Laura Weidinger.
Multi-turn evaluation of anthropomorphic behaviours in large language models, 2025.
URL [https://arxiv.org/abs/2502.07077](https://arxiv.org/abs/2502.07077).
- Ispas & Ispas (2023)
A. Ispas and C. Ispas.
Automatic thoughts and personality factors in the development of self-efficacy in students.
*The European Proceedings of Social and Behavioural Sciences*, 6:522–529, 2023.
doi: 10.15405/epes.23056.47.
- Ji et al. (2025a)
Jianchao Ji, Yutong Chen, Mingyu Jin, Wujiang Xu, Wenyue Hua, and Yongfeng Zhang.
Moralbench: Moral evaluation of llms, 2025a.
URL [https://arxiv.org/abs/2406.04428](https://arxiv.org/abs/2406.04428).
- Ji et al. (2025b)
Ke Ji, Yixin Lian, Linxu Li, Jingsheng Gao, Weiyuan Li, and Bin Dai.
Enhancing persona consistency for llms’ role-playing using persona-aware contrastive learning, 2025b.
URL [https://arxiv.org/abs/2503.17662](https://arxiv.org/abs/2503.17662).
- Jiang et al. (2023a)
Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu.
Evaluating and inducing personality in pre-trained language models.
*Advances in Neural Information Processing Systems*, 36:10622–10643, 2023a.
- Jiang et al. (2023b)
Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara.
Personallm: Investigating the ability of large language models to express personality traits.
*arXiv preprint arXiv:2305.02547*, 2023b.
- Jiang et al. (2024a)
Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara.
Personallm: Investigating the ability of large language models to express personality traits.
In *Findings of the Association for Computational Linguistics: NAACL 2024*, pp.  3605–3627, 2024a.
- Jiang et al. (2024b)
Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara.
Personallm: Investigating the ability of large language models to express personality traits, 2024b.
URL [https://arxiv.org/abs/2305.02547](https://arxiv.org/abs/2305.02547).
- Jiang et al. (2023c)
Roy Jiang, Rafal Kocielnik, Adhithya Prakash Saravanan, Pengrui Han, R Michael Alvarez, and Anima Anandkumar.
Empowering domain experts to detect social bias in generative ai with user-friendly interfaces.
In *XAI in Action: Past, Present, and Future Applications*, 2023c.
- John et al. (1991)
Oliver P John, Eileen M Donahue, and Robert L Kentle.
Big five inventory.
*Journal of personality and social psychology*, 1991.
- John (2018)
Stephen John.
Epistemic trust and the ethics of science communication: Against transparency, openness, sincerity and honesty.
*Social Epistemology*, 32(2):75–87, 2018.
- Kahneman & Tversky (2013)
Daniel Kahneman and Amos Tversky.
Prospect theory: An analysis of decision under risk.
In *Handbook of the fundamentals of financial decision making: Part I*, pp.  99–127. World Scientific, 2013.
- Kamruzzaman & Kim (2025)
Mahammed Kamruzzaman and Gene Louis Kim.
Prompting techniques for reducing social bias in llms through system 1 and system 2 cognitive processes, 2025.
URL [https://arxiv.org/abs/2404.17218](https://arxiv.org/abs/2404.17218).
- Kandler et al. (2012)
Christian Kandler, Lisa Held, Christine Kroll, Anja Bergeler, Rainer Riemann, and Alois Angleitner.
Genetic links between temperamental traits of the regulative theory of temperament and the big five.
*Journal of Individual Differences*, 33(4):197–204, 2012.
doi: 10.1027/1614-0001/a000068.
- Khan et al. (2025)
Ariba Khan, Stephen Casper, and Dylan Hadfield-Menell.
Randomness, not representation: The unreliability of evaluating cultural alignment in llms.
*arXiv preprint arXiv:2503.08688*, 2025.
- Kim et al. (2023)
Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi, and Maarten Sap.
Fantom: A benchmark for stress-testing machine theory of mind in interactions.
*arXiv preprint arXiv:2310.15421*, 2023.
- Kosinski (2023)
Michal Kosinski.
Theory of mind may have spontaneously emerged in large language models.
*arXiv preprint arXiv:2302.02083*, 4:169, 2023.
- Kosinski (2024)
Michal Kosinski.
Evaluating large language models in theory of mind tasks.
*Proceedings of the National Academy of Sciences*, 121(45), October 2024.
ISSN 1091-6490.
doi: 10.1073/pnas.2405460121.
URL [http://dx.doi.org/10.1073/pnas.2405460121](http://dx.doi.org/10.1073/pnas.2405460121).
- Krumrei-Mancuso & Rouse (2016)
Elizabeth J Krumrei-Mancuso and Steven V Rouse.
The development and validation of the comprehensive intellectual humility scale.
*Journal of Personality Assessment*, 98(2):209–221, 2016.
- Lampinen et al. (2024)
Andrew K Lampinen, Ishita Dasgupta, Stephanie CY Chan, Hannah R Sheahan, Antonia Creswell, Dharshan Kumaran, James L McClelland, and Felix Hill.
Language models, like humans, show content effects on reasoning tasks.
*PNAS nexus*, 3(7):pgae233, 2024.
- Leary et al. (2017)
Mark R Leary, Kate J Diebels, Erin K Davisson, Katrina P Jongman-Sereno, Jennifer C Isherwood, Kaitlin T Raimi, Samantha A Deffler, and Rick H Hoyle.
Cognitive and interpersonal features of intellectual humility.
*Personality and Social Psychology Bulletin*, 43(6):793–813, 2017.
- Lee et al. (2021)
J. Lee, M. Bosma, V. Zhao, K. Guu, A. Yu, B. Lester, and Q. Le.
Finetuned language models are zero-shot learners.
*arXiv preprint arXiv:2109.01652*, 2021.
doi: 10.48550/arxiv.2109.01652.
- Lee et al. (2025)
Seungbeen Lee, Seungwon Lim, Seungju Han, Giyeong Oh, Hyungjoo Chae, Jiwan Chung, Minju Kim, Beong woo Kwak, Yeonsoo Lee, Dongha Lee, Jinyoung Yeo, and Youngjae Yu.
Do llms have distinct and consistent personality? trait: Personality testset designed for llms with psychometrics, 2025.
URL [https://arxiv.org/abs/2406.14703](https://arxiv.org/abs/2406.14703).
- Legault et al. (2007)
Lisa Legault, Isabelle Green-Demers, Protius Grant, and Joyce Chung.
On the self-regulation of implicit and explicit prejudice: A self-determination theory perspective.
*Personality and Social Psychology Bulletin*, 33(5):732–749, 2007.
- Li et al. (2023)
Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem.
Camel: Communicative agents for "mind" exploration of large language model society, 2023.
URL [https://arxiv.org/abs/2303.17760](https://arxiv.org/abs/2303.17760).
- Li et al. (2016)
Jing Li, Yali Zhao, Fang Kong, Shujun Du, Shanshan Yang, and Shiyong Wang.
Psychometric assessment of the short grit scale among chinese adolescents.
*Journal of Psychoeducational Assessment*, 36(3):291–296, 2016.
doi: 10.1177/0734282916674858.
- Li et al. (2024a)
Siheng Li, Cheng Yang, Taiqiang Wu, Chufan Shi, Yuji Zhang, Xinyu Zhu, Zesen Cheng, Deng Cai, Mo Yu, Lemao Liu, et al.
A survey on the honesty of large language models.
*arXiv preprint arXiv:2409.18786*, 2024a.
- Li et al. (2024b)
Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona Diab, and Maarten Sap.
Big5-chat: Shaping llm personalities through training on human-grounded data.
*arXiv preprint arXiv:2410.16491*, 2024b.
- Li et al. (2025a)
Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona Diab, and Maarten Sap.
Big5-chat: Shaping llm personalities through training on human-grounded data, 2025a.
URL [https://arxiv.org/abs/2410.16491](https://arxiv.org/abs/2410.16491).
- Li et al. (2025b)
Xiaoyu Li, Haoran Shi, Zengyi Yu, Yukun Tu, and Chanjin Zheng.
Decoding LLM personality measurement: Forced-choice vs. Likert.
In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), *Findings of the Association for Computational Linguistics: ACL 2025*, pp.  9234–9247, Vienna, Austria, July 2025b. Association for Computational Linguistics.
ISBN 979-8-89176-256-5.
doi: 10.18653/v1/2025.findings-acl.480.
URL [https://aclanthology.org/2025.findings-acl.480/](https://aclanthology.org/2025.findings-acl.480/).
- Lian et al. (2017)
Huiwen Lian, Kai Chi Yam, D Lance Ferris, and Douglas Brown.
Self-control at work.
*Academy of Management Annals*, 11(2):703–732, 2017.
- Lindsey et al. (2025)
Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson.
On the biology of a large language model.
*Transformer Circuits Thread*, 2025.
URL [https://transformer-circuits.pub/2025/attribution-graphs/biology.html](https://transformer-circuits.pub/2025/attribution-graphs/biology.html).
- Liu et al. (2025a)
Jiahong Liu, Zexuan Qiu, Zhongyang Li, Quanyu Dai, Jieming Zhu, Minda Hu, Menglin Yang, and Irwin King.
A survey of personalized large language models: Progress and future directions, 2025a.
URL [https://arxiv.org/abs/2502.11528](https://arxiv.org/abs/2502.11528).
- Liu et al. (2025b)
Zizhou Liu, Ziwei Gong, Lin Ai, Zheng Hui, Run Chen, Colin Wayne Leach, Michelle R Greene, and Julia Hirschberg.
The mind in the machine: A survey of incorporating psychological theories in llms.
*arXiv preprint arXiv:2505.00003*, 2025b.
- Löhn et al. (2024)
Lea Löhn, Niklas Kiehne, Alexander Ljapunov, and Wolf-Tilo Balke.
Is machine psychology here? on requirements for using human psychological tests on large language models.
In Saad Mahamood, Nguyen Le Minh, and Daphne Ippolito (eds.), *Proceedings of the 17th International Natural Language Generation Conference*, pp.  230–242, Tokyo, Japan, September 2024. Association for Computational Linguistics.
doi: 10.18653/v1/2024.inlg-main.19.
URL [https://aclanthology.org/2024.inlg-main.19/](https://aclanthology.org/2024.inlg-main.19/).
- Lopes et al. (2005)
Paulo N Lopes, Peter Salovey, Stéphane Côté, Michael Beers, and Richard E Petty.
Emotion regulation abilities and the quality of social interaction.
*Emotion*, 5(1):113, 2005.
- Ma et al. (2025)
Yiping Ma, Shiyu Hu, Xuchen Li, Yipei Wang, Yuqing Chen, Shiqing Liu, and Kang Hao Cheong.
When llms learn to be students: The soei framework for modeling and evaluating virtual student agents in educational interaction, 2025.
URL [https://arxiv.org/abs/2410.15701](https://arxiv.org/abs/2410.15701).
- Malmqvist (2025)
Lars Malmqvist.
Sycophancy in large language models: Causes and mitigations.
In *Intelligent Computing-Proceedings of the Computing Conference*, pp.  61–74. Springer, 2025.
- McCrae & John (1992)
R. McCrae and O. John.
An introduction to the five-factor model and its applications.
*Journal of Personality*, 60(2):175–215, 1992.
doi: 10.1111/j.1467-6494.1992.tb00970.x.
- Miotto et al. (2022)
Marilù Miotto, Nicola Rossberg, and Bennett Kleinberg.
Who is gpt-3? an exploration of personality, values and demographics, 2022.
URL [https://arxiv.org/abs/2209.14338](https://arxiv.org/abs/2209.14338).
- Mireshghallah et al. (2024)
Niloofar Mireshghallah, Maria Antoniak, Yash More, Yejin Choi, and Golnoosh Farnadi.
Trust no bot: Discovering personal disclosures in human-llm conversations in the wild, 2024.
URL [https://arxiv.org/abs/2407.11438](https://arxiv.org/abs/2407.11438).
- Mitchell & Krakauer (2023)
Melanie Mitchell and David C. Krakauer.
The debate over understanding in ai’s large language models.
*Proceedings of the National Academy of Sciences*, 120(13):e2215907120, 2023.
doi: 10.1073/pnas.2215907120.
URL [https://www.pnas.org/doi/abs/10.1073/pnas.2215907120](https://www.pnas.org/doi/abs/10.1073/pnas.2215907120).
- Moore et al. (2024)
Jared Moore, Tanvi Deshpande, and Diyi Yang.
Are large language models consistent over value-laden questions?
*arXiv preprint arXiv:2407.02996*, 2024.
- Muraven & Baumeister (2000)
Mark Muraven and Roy F Baumeister.
Self-regulation and depletion of limited resources: Does self-control resemble a muscle?
*Psychological bulletin*, 126(2):247, 2000.
- Nelson & Narens (1980)
Thomas O Nelson and Louis Narens.
Norms of 300 general-information questions: Accuracy of recall, latency of recall, and feeling-of-knowing ratings.
*Journal of verbal learning and verbal behavior*, 19(3):338–368, 1980.
- Nettle & Liddle (2008)
Daniel Nettle and Bethany Liddle.
Agreeableness is related to social-cognitive, but not social-perceptual, theory of mind.
*European Journal of Personality: Published for the European Association of Personality Psychology*, 22(4):323–335, 2008.
- Ng et al. (2021)
DX Ng, Patrick KF Lin, Nigel V Marsh, KQ Chan, and Jonathan E Ramsay.
Associations between openness facets, prejudice, and tolerance: A scoping review with meta-analysis.
*Frontiers in Psychology*, 12:707652, 2021.
- Nicholson et al. (2005)
Nigel Nicholson, Emma Soane, Mark Fenton-O’Creevy, and Paul Willman.
Personality and domain-specific risk taking.
*Journal of Risk Research*, 8(2):157–176, 2005.
doi: 10.1080/1366987032000123856.
URL [https://doi.org/10.1080/1366987032000123856](https://doi.org/10.1080/1366987032000123856).
- Nighojkar et al. (2025)
Animesh Nighojkar, Bekhzodbek Moydinboyev, My Duong, and John Licato.
Giving ai personalities leads to more human-like reasoning, 2025.
URL [https://arxiv.org/abs/2502.14155](https://arxiv.org/abs/2502.14155).
- Nilsen et al. (2024)
Fredrik A Nilsen, Henning Bang, and Espen Røysamb.
Personality traits and self-control: The moderating role of neuroticism.
*Plos one*, 19(8):e0307871, 2024.
- Ode & Robinson (2007)
Scott Ode and Michael D Robinson.
Agreeableness and the self-regulation of negative affect: Findings involving the neuroticism/somatic distress relationship.
*Personality and Individual Differences*, 43(8):2137–2148, 2007.
doi: 10.1016/j.paid.2007.06.035.
- Olea et al. (2024)
Carlos Olea, Holly Tucker, Jessica Phelan, Cameron Pattison, Shen Zhang, Maxwell Lieb, and J White.
Evaluating persona prompting for question answering tasks.
In *Proceedings of th e 10th international conference on artificial intelligence and soft computing, Sydney, Australia*, 2024.
- O’Leary (2025)
Daniel E O’Leary.
Confirmation and specificity biases in large language models: An explorative study.
*IEEE Intelligent Systems*, 40(1):63–68, 2025.
- Panickssery et al. (2024)
Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner.
Steering llama 2 via contrastive activation addition, 2024.
URL [https://arxiv.org/abs/2312.06681](https://arxiv.org/abs/2312.06681).
- Park et al. (2023)
Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein.
Generative agents: Interactive simulacra of human behavior, 2023.
URL [https://arxiv.org/abs/2304.03442](https://arxiv.org/abs/2304.03442).
- Pellert et al. (2024)
Max Pellert, Clemens M Lechner, Claudia Wagner, Beatrice Rammstedt, and Markus Strohmaier.
Ai psychometrics: Assessing the psychological profiles of large language models through psychometric inventories.
*Perspectives on Psychological Science*, 19(5):808–826, 2024.
- Peter et al. (2025)
Sandra Peter, Kai Riemer, and Jevin D West.
The benefits and dangers of anthropomorphic conversational agents.
*Proceedings of the National Academy of Sciences*, 122(22):e2415898122, 2025.
- Peters & Matz (2024)
H. Peters and S. Matz.
Large language models can infer psychological dispositions of social media users.
*PNAS Nexus*, 3(6), 2024.
doi: 10.1093/pnasnexus/pgae231.
- Petrov et al. (2024)
Nikolay B Petrov, Gregory Serapio-García, and Jason Rentfrow.
Limited ability of llms to simulate human psychological behaviours: a psychometric analysis.
*arXiv preprint arXiv:2405.07248*, 2024.
- Pi et al. (2024)
Zhiqiang Pi, Annapurna Vadaparty, Benjamin K Bergen, and Cameron R Jones.
Dissecting the ullman variations with a scalpel: Why do llms fail at trivial alterations to the false belief task?
*arXiv preprint arXiv:2406.14737*, 2024.
- Pintrich & De Groot (1990)
Paul R Pintrich and Elisabeth V De Groot.
Motivational and self-regulated learning components of classroom academic performance.
*Journal of educational psychology*, 82(1):33, 1990.
- Placani (2024)
Adriana Placani.
Anthropomorphism in ai: hype and fallacy.
*AI and Ethics*, 4, 02 2024.
doi: 10.1007/s43681-024-00419-4.
- Porter et al. (2022)
Tenelle Porter, Abdo Elnakouri, Ethan A Meyers, Takuya Shibayama, Eranda Jayawickreme, and Igor Grossmann.
Predictors and consequences of intellectual humility.
*Nature Reviews Psychology*, 1(9):524–536, 2022.
- Rahwan et al. (2019a)
Iyad Rahwan, Manuel Cebrian, Nick Obradovich, Josh Bongard, Jean-François Bonnefon, Cynthia Breazeal, Jacob W Crandall, Nicholas A Christakis, Iain D Couzin, Matthew O Jackson, et al.
Machine behaviour.
*Nature*, 568(7753):477–486, 2019a.
- Rahwan et al. (2019b)
Iyad Rahwan, Manuel Cebrian, Nick Obradovich, Josh Bongard, Jean-François Bonnefon, Cynthia Breazeal, Jacob Crandall, Nicholas Christakis, Iain Couzin F.R.S., Matthew Jackson, Nicholas Jennings, Ece Kamar, Isabel Kloumann, Hugo Larochelle, David Lazer, Richard McElreath, Alan Mislove, David Parkes, Alex Pentland, and Michael Wellman.
Machine behaviour.
*Nature*, 568:477–486, 04 2019b.
doi: 10.1038/s41586-019-1138-y.
- Raj et al. (2024)
K. Raj, K. Roy, V. Bonagiri, P. Govil, K. Thirunarayan, R. Goswami, and M. Gaur.
K-perm: Personalized response generation using dynamic knowledge retrieval and persona-adaptive queries.
*AAAI-SS*, 3(1):219–226, 2024.
doi: 10.1609/aaaiss.v3i1.31203.
- Reinecke et al. (2025)
Madeline G Reinecke, Fransisca Ting, Julian Savulescu, and Ilina Singh.
The double-edged sword of anthropomorphism in llms.
In *Proceedings*, volume 114, pp.  4. MDPI, 2025.
- Roberts et al. (2006)
Brent W Roberts, Kate E Walton, and Wolfgang Viechtbauer.
Patterns of mean-level change in personality traits across the life course: a meta-analysis of longitudinal studies.
*Psychological bulletin*, 132(1):1, 2006.
- Roberts et al. (2007)
Brent W Roberts, Nathan R Kuncel, Rebecca Shiner, Avshalom Caspi, and Lewis R Goldberg.
The power of personality: The comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes.
*Perspectives on Psychological science*, 2(4):313–345, 2007.
- Roberts et al. (2014)
Brent W Roberts, Carl Lejuez, Robert F Krueger, Jessica M Richards, and Patrick L Hill.
What is conscientiousness and how can it be assessed?
*Developmental psychology*, 50(5):1315, 2014.
- Röttger et al. (2024)
Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Rose Kirk, Hinrich Schütze, and Dirk Hovy.
Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models.
*arXiv preprint arXiv:2402.16786*, 2024.
- Roulin & Bourdage (2017)
Nicolas Roulin and Joshua S Bourdage.
Once an impression manager, always an impression manager? antecedents of honest and deceptive impression management use and variability across multiple job interviews.
*Frontiers in psychology*, 8:29, 2017.
- Sainz et al. (2023)
Oscar Sainz, Jon Ander Campos, Iker García-Ferrero, Julen Etxaniz, Oier Lopez de Lacalle, and Eneko Agirre.
Nlp evaluation in trouble: On the need to measure llm data contamination for each benchmark.
*arXiv preprint arXiv:2310.18018*, 2023.
- Salecha et al. (2024)
Aadesh Salecha, Molly E. Ireland, Shashanka Subrahmanya, João Sedoc, Lyle H. Ungar, and Johannes C. Eichstaedt.
Large language models show human-like social desirability biases in survey responses, 2024.
URL [https://arxiv.org/abs/2405.06058](https://arxiv.org/abs/2405.06058).
- Schaaff & Heidelmann (2024)
Kristina Schaaff and Marc-André Heidelmann.
Impacts of anthropomorphizing large language models in learning environments, 2024.
URL [https://arxiv.org/abs/2408.03945](https://arxiv.org/abs/2408.03945).
- Schaefer et al. (2004)
Peter S Schaefer, Cristina C Williams, Adam S Goodie, and W Keith Campbell.
Overconfidence and the big five.
*Journal of research in Personality*, 38(5):473–480, 2004.
- Schmader et al. (2008)
Toni Schmader, Michael Johns, and Chad Forbes.
An integrated process model of stereotype threat effects on performance.
*Psychological review*, 115(2):336, 2008.
- Schwartz (1992)
Shalom H Schwartz.
Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries.
In *Advances in experimental social psychology*, volume 25, pp.  1–65. Elsevier, 1992.
- Serapio-Garc´ıa et al. (2023)
Greg Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić.
Personality traits in large language models.
*arXiv preprint arXiv:2307.00184*, 2023.
- Serapio-García et al. (2025)
Greg Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić.
Personality traits in large language models, 2025.
URL [https://arxiv.org/abs/2307.00184](https://arxiv.org/abs/2307.00184).
- Shanahan (2023)
Murray Shanahan.
Talking about large language models, 2023.
URL [https://arxiv.org/abs/2212.03551](https://arxiv.org/abs/2212.03551).
- Shanahan et al. (2023)
Murray Shanahan, Kyle McDonell, and Laria Reynolds.
Role play with large language models.
*Nature*, 623(7987):493–498, 2023.
- Sharma et al. (2023)
Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al.
Towards understanding sycophancy in language models.
*arXiv preprint arXiv:2310.13548*, 2023.
- Shiffrin & Mitchell (2023)
Richard Shiffrin and Melanie Mitchell.
Probing the psychology of ai models.
*Proceedings of the National Academy of Sciences*, 120(10):e2300963120, 2023.
doi: 10.1073/pnas.2300963120.
URL [https://www.pnas.org/doi/abs/10.1073/pnas.2300963120](https://www.pnas.org/doi/abs/10.1073/pnas.2300963120).
- Shunsen et al. (2024)
Huang Shunsen, Xiaoxiong Lai, Li Ke, Yajun Li, Huanlei Wang, Xinmei Zhao, Xinran Dai, and Yun Wang.
Ai technology panic—is ai dependence bad for mental health? a cross-lagged panel model and the mediating roles of motivations for ai use among adolescents.
*Psychology Research and Behavior Management*, 17:1087–1102, 03 2024.
doi: 10.2147/PRBM.S440889.
- Sibley & Duckitt (2008)
Chris G Sibley and John Duckitt.
Personality and prejudice: A meta-analysis and theoretical review.
*Personality and social psychology review*, 12(3):248–279, 2008.
- Sikström et al. (2024)
Sverker Sikström, Ieva Valavičiūtė, and Petri Kajonius.
Personality in just a few words: Assessment using natural language processing, 2024.
Preprint.
- Sinclair et al. (2005)
Stacey Sinclair, Brian S Lowery, Curtis D Hardin, and Anna Colangelo.
Social tuning of automatic racial attitudes: the role of affiliative motivation.
*Journal of personality and social psychology*, 89(4):583, 2005.
- Song et al. (2025)
Peiyang Song, Pengrui Han, and Noah Goodman.
A survey on large language model reasoning failures.
In *2nd AI for Math Workshop@ ICML 2025*, 2025.
- Song et al. (2023)
Xiaoyang Song, Akshat Gupta, Kiyan Mohebbizadeh, Shujie Hu, and Anant Singh.
Have large language models developed a personality?: Applicability of self-assessment tests in measuring personality in llms.
*arXiv preprint arXiv:2305.14693*, 2023.
- Spada et al. (2016)
Marcantonio M Spada, Harriet Gay, Ana V Nikčevic, Bruce A Fernie, and Gabriele Caselli.
Meta-cognitive beliefs about worry and pain catastrophising as mediators between neuroticism and pain behaviour.
*Clinical Psychologist*, 20(3):138–146, 2016.
- Stanovich & Toplak (2023)
Keith E Stanovich and Maggie E Toplak.
Actively open-minded thinking and its measurement.
*Journal of Intelligence*, 11(2):27, 2023.
- Steel (2007)
Piers Steel.
The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure.
*Psychological Bulletin*, 133(1):65–94, 2007.
doi: 10.1037/0033-2909.133.1.65.
URL [https://doi.org/10.1037/0033-2909.133.1.65](https://doi.org/10.1037/0033-2909.133.1.65).
- Steyvers et al. (2025)
Mark Steyvers, Heliodoro Tejeda, Aakriti Kumar, Catarina Belem, Sheer Karny, Xinyue Hu, Lukas W. Mayer, and Padhraic Smyth.
What large language models know and what people think they know.
*Nature Machine Intelligence*, 7(2):221–231, January 2025.
ISSN 2522-5839.
doi: 10.1038/s42256-024-00976-7.
URL [http://dx.doi.org/10.1038/s42256-024-00976-7](http://dx.doi.org/10.1038/s42256-024-00976-7).
- Stöber et al. (2002)
Joachim Stöber, Dorothea E Dette, and Jochen Musch.
Comparing continuous and dichotomous scoring of the balanced inventory of desirable responding.
*Journal of personality assessment*, 78(2):370–389, 2002.
- Sühr et al. (2023)
Tom Sühr, Florian E Dorner, Samira Samadi, and Augustin Kelava.
Challenging the validity of personality tests for large language models.
*Preprint at arXiv. arXiv-2311 https://doi. org/10.48550/arXiv*, 2311, 2023.
- Sun & Wang (2025)
Yuan Sun and Ting Wang.
Be friendly, not friends: How llm sycophancy shapes user trust.
*arXiv preprint arXiv:2502.10844*, 2025.
- Takemoto (2024)
Kazuhiro Takemoto.
The moral machine experiment on large language models.
*Royal Society open science*, 11(2):231393, 2024.
- Tan et al. (2024)
Fiona Anting Tan, Gerard Christopher Yeo, Kokil Jaidka, Fanyou Wu, Weijie Xu, Vinija Jain, Aman Chadha, Yang Liu, and See-Kiong Ng.
Phantom: Persona-based prompting has an effect on theory-of-mind reasoning in large language models.
*arXiv preprint arXiv:2403.02246*, 2024.
- Trapnell & Campbell (1999)
Paul D Trapnell and Jennifer D Campbell.
Private self-consciousness and the five-factor model of personality: distinguishing rumination from reflection.
*Journal of personality and social psychology*, 76(2):284, 1999.
- Treder et al. (2024)
M. Treder, S. Lee, and K. Tsvetanov.
Introduction to large language models (llms) for dementia care and research.
*Frontiers in Dementia*, 3, 2024.
doi: 10.3389/frdem.2024.1385303.
- tse Huang et al. (2024a)
Jen tse Huang, Wenxiang Jiao, Man Ho Lam, Eric John Li, Wenxuan Wang, and Michael R. Lyu.
Revisiting the reliability of psychological scales on large language models, 2024a.
URL [https://arxiv.org/abs/2305.19926](https://arxiv.org/abs/2305.19926).
- tse Huang et al. (2024b)
Jen tse Huang, Wenxuan Wang, Eric John Li, Man Ho Lam, Shujie Ren, Youliang Yuan, Wenxiang Jiao, Zhaopeng Tu, and Michael R. Lyu.
Who is chatgpt? benchmarking llms’ psychological portrayal using psychobench, 2024b.
URL [https://arxiv.org/abs/2310.01386](https://arxiv.org/abs/2310.01386).
- Tseng et al. (2024a)
Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Wei-Lin Chen, Chao-Wei Huang, Yu Meng, and Yun-Nung Chen.
Two tales of persona in llms: A survey of role-playing and personalization, 2024a.
URL [https://arxiv.org/abs/2406.01171](https://arxiv.org/abs/2406.01171).
- Tseng et al. (2024b)
Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Wei-Lin Chen, Chao-Wei Huang, Yu Meng, and Yun-Nung Chen.
Two tales of persona in llms: A survey of role-playing and personalization.
*arXiv preprint arXiv:2406.01171*, 2024b.
- Turner et al. (2014)
Rhiannon N Turner, Kristof Dhont, Miles Hewstone, Andrew Prestwich, and Christiana Vonofakou.
The role of personality factors in the reduction of intergroup anxiety and amelioration of outgroup attitudes via intergroup contact.
*European Journal of Personality*, 28(2):180–192, 2014.
- Turpin et al. (2023)
Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman.
Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting, 2023.
URL [https://arxiv.org/abs/2305.04388](https://arxiv.org/abs/2305.04388).
- van Duijn et al. (2023)
Max J van Duijn, Bram van Dijk, Tom Kouwenhoven, Werner de Valk, Marco R Spruit, and Peter van der Putten.
Theory of mind in large language models: Examining performance of 11 state-of-the-art models vs. children aged 7-10 on advanced tests.
*arXiv preprint arXiv:2310.20320*, 2023.
- Van Iddekinge et al. (2007)
Chad H Van Iddekinge, Lynn A McFarland, and Patrick H Raymark.
Antecedents of impression management use and effectiveness in a structured interview.
*Journal of Management*, 33(5):752–773, 2007.
- van Pinxteren et al. (2023)
Michelle ME van Pinxteren, Mark Pluymaekers, Jos Lemmink, and Anna Krispin.
Effects of communication style on relational outcomes in interactions between customers and embodied conversational agents.
*Psychology & Marketing*, 40(5):938–953, 2023.
- Vohs et al. (2005)
Kathleen D Vohs, Roy F Baumeister, and Natalie J Ciarocco.
Self-regulation and self-presentation: regulatory resource depletion impairs impression management and effortful self-presentation depletes regulatory resources.
*Journal of personality and social psychology*, 88(4):632, 2005.
- Wang et al. (2024a)
Jiaojiao Wang, Yanchao Jiao, Mengyun Peng, Yanan Wang, Daoxia Guo, and Li Tian.
The relationship between personality traits, metacognition and professional commitment in chinese nursing students: a cross-sectional study.
*BMC nursing*, 23(1):729, 2024a.
- Wang et al. (2024b)
Lei Wang, Jianxun Lian, Yi Huang, Yanqi Dai, Haoxuan Li, Xu Chen, Xing Xie, and Ji-Rong Wen.
Characterbox: Evaluating the role-playing capabilities of llms in text-based virtual worlds, 2024b.
URL [https://arxiv.org/abs/2412.05631](https://arxiv.org/abs/2412.05631).
- Wang et al. (2025a)
Miles Wang, Tom Dupré la Tour, Olivia Watkins, Alex Makelov, Ryan A. Chi, Samuel Miserendino, Johannes Heidecke, Tejal Patwardhan, and Dan Mossing.
Persona features control emergent misalignment, 2025a.
URL [https://arxiv.org/abs/2506.19823](https://arxiv.org/abs/2506.19823).
- Wang et al. (2024c)
Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, and Yanghua Xiao.
InCharacter: Evaluating personality fidelity in role-playing agents through psychological interviews.
In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp.  1840–1873, Bangkok, Thailand, August 2024c. Association for Computational Linguistics.
doi: 10.18653/v1/2024.acl-long.102.
URL [https://aclanthology.org/2024.acl-long.102/](https://aclanthology.org/2024.acl-long.102/).
- Wang et al. (2025b)
Yilei Wang, Jiabao Zhao, Deniz S Ones, Liang He, and Xin Xu.
Evaluating the ability of large language models to emulate personality.
*Scientific reports*, 15(1):519, 2025b.
- Wang et al. (2025c)
Zixiao Wang, Duzhen Zhang, Ishita Agrawal, Shen Gao, Le Song, and Xiuying Chen.
Beyond profile: From surface-level facts to deep persona simulation in llms.
*arXiv preprint arXiv:2502.12988*, 2025c.
- Waytz et al. (2014)
Adam Waytz, Joy Heafner, and Nicholas Epley.
The mind in the machine: Anthropomorphism increases trust in an autonomous vehicle.
*Journal of Experimental Social Psychology*, 52:113–117, 2014.
ISSN 0022-1031.
doi: https://doi.org/10.1016/j.jesp.2014.01.005.
URL [https://www.sciencedirect.com/science/article/pii/S0022103114000067](https://www.sciencedirect.com/science/article/pii/S0022103114000067).
- Wei et al. (2022)
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus.
Emergent abilities of large language models, 2022.
URL [https://arxiv.org/abs/2206.07682](https://arxiv.org/abs/2206.07682).
- Xiao et al. (2025)
Yunze Xiao, Lynnette Hui Xian Ng, Jiarui Liu, and Mona T. Diab.
Humanizing machines: Rethinking llm anthropomorphism through a multi-level framework of design, 2025.
URL [https://arxiv.org/abs/2508.17573](https://arxiv.org/abs/2508.17573).
- Xie et al. (2023)
Yuguang Xie, Keyu Zhu, Peiyu Zhou, and Changyong Liang.
How does anthropomorphism improve human-ai interaction satisfaction: a dual-path model.
*Computers in Human Behavior*, 148:107878, 2023.
ISSN 0747-5632.
doi: https://doi.org/10.1016/j.chb.2023.107878.
URL [https://www.sciencedirect.com/science/article/pii/S0747563223002297](https://www.sciencedirect.com/science/article/pii/S0747563223002297).
- Yang et al. (2023)
Fang Yang, Chikako Hagiwara, Takashi Kotani, Jun Hirao, and Atsushi Oshio.
Comparing self-esteem and self-compassion: An analysis within the big five personality traits framework.
*Frontiers in Psychology*, 14, 2023.
doi: 10.3389/fpsyg.2023.1302197.
- Yang et al. (2025)
Shu Yang, Shenzhe Zhu, Liang Liu, Lijie Hu, Mengdi Li, and Di Wang.
Exploring the personality traits of llms through latent features steering, 2025.
URL [https://arxiv.org/abs/2410.10863](https://arxiv.org/abs/2410.10863).
- Yang et al. (2024)
Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, and Pengfei Liu.
Alignment for honesty.
*Advances in Neural Information Processing Systems*, 37:63565–63598, 2024.
- Yetman (2024)
Cameron C Yetman.
Representation in large language models.
In *Proceedings of the Annual Meeting of the Cognitive Science Society*, volume 46, 2024.
- Yu et al. (2024)
Sangwon Yu, Jongyoon Song, Bongkyu Hwang, Hoyoung Kang, Sooah Cho, Junhwa Choi, Seongho Joe, Taehee Lee, Youngjune L Gwon, and Sungroh Yoon.
Correcting negative bias in large language models through negative attention score alignment.
*arXiv preprint arXiv:2408.00137*, 2024.
- Zhang et al. (2018)
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston.
Personalizing dialogue agents: I have a dog, do you have pets too?, 2018.
URL [https://arxiv.org/abs/1801.07243](https://arxiv.org/abs/1801.07243).
- Zhao et al. (2023)
Lin Zhao, Lu Zhang, Zihao Wu, Yuzhong Chen, Haixing Dai, Xiaowei Yu, Zhengliang Liu, Tuo Zhang, Xintao Hu, Xi Jiang, Xiang Li, Dajiang Zhu, Dinggang Shen, and Tianming Liu.
When brain-inspired ai meets agi.
*Meta-Radiology*, 1(1):100005, 2023.
ISSN 2950-1628.
doi: https://doi.org/10.1016/j.metrad.2023.100005.
URL [https://www.sciencedirect.com/science/article/pii/S295016282300005X](https://www.sciencedirect.com/science/article/pii/S295016282300005X).
- Zheng et al. (2024a)
Mingqian Zheng, Jiaxin Pei, Lajanugen Logeswaran, Moontae Lee, and David Jurgens.
When “a helpful assistant” is not really helpful: Personas in system prompts do not improve performances of large language models.
In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), *Findings of the Association for Computational Linguistics: EMNLP 2024*, pp.  15126–15154, Miami, Florida, USA, November 2024a. Association for Computational Linguistics.
doi: 10.18653/v1/2024.findings-emnlp.888.
URL [https://aclanthology.org/2024.findings-emnlp.888/](https://aclanthology.org/2024.findings-emnlp.888/).
- Zheng et al. (2024b)
Mingqian Zheng, Jiaxin Pei, Lajanugen Logeswaran, Moontae Lee, and David Jurgens.
When "a helpful assistant" is not really helpful: Personas in system prompts do not improve performances of large language models, 2024b.
URL [https://arxiv.org/abs/2311.10054](https://arxiv.org/abs/2311.10054).
- Zhou et al. (2024)
Kaitlyn Zhou, Jena D. Hwang, Xiang Ren, Nouha Dziri, Dan Jurafsky, and Maarten Sap.
Rel-a.i.: An interaction-centered approach to measuring human-lm reliance, 2024.
URL [https://arxiv.org/abs/2407.07950](https://arxiv.org/abs/2407.07950).
- Zhou et al. (2025)
Xin Zhou, Martin Weyssow, Ratnadira Widyasari, Ting Zhang, Junda He, Yunbo Lyu, Jianming Chang, Beiqi Zhang, Dan Huang, and David Lo.
Lessleak-bench: A first investigation of data leakage in llms across 83 software engineering benchmarks.
*arXiv preprint arXiv:2502.06215*, 2025.
- Zhu et al. (2025)
Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang.
Personality alignment of large language models, 2025.
URL [https://arxiv.org/abs/2408.11779](https://arxiv.org/abs/2408.11779).
- Zollo et al. (2025)
Thomas P. Zollo, Andrew Wei Tung Siah, Naimeng Ye, Ang Li, and Hongseok Namkoong.
Personalllm: Tailoring llms to individual preferences, 2025.
URL [https://arxiv.org/abs/2409.20296](https://arxiv.org/abs/2409.20296).
- Zou et al. (2025)
Huiqi Zou, Pengda Wang, Zihan Yan, Tianjun Sun, and Ziang Xiao.
Can llm "self-report"?: Evaluating the validity of self-report scales in measuring personality design in llm-based chatbots, 2025.
URL [https://arxiv.org/abs/2412.00207](https://arxiv.org/abs/2412.00207).

## Appendix A Code & Artifacts

We make public all code and source data at [https://github.com/psychology-of-AI/Personality-Illusion](https://github.com/psychology-of-AI/Personality-Illusion) for full transparency and reproducibility, to benefit future works in this direction.
Please reference our documentation in our repository, for guidance on usage of our codebase.

## Appendix B Exploratory Data Analysis across LLMs

### B.1 Per Model Self-Reported Personality Trait Profiles

Figure [6](https://arxiv.org/html/2509.03730v2#A2.F6) shows the normalized trait profiles (1–5 scale) for each individual model across the Big Five and self-regulation, separated by training phase. Each subplot corresponds to a single model, with lines and shaded regions indicating mean scores and 95% confidence intervals. Comparing pre-training to post-training alignment reveals both a reduction in variability and systematic shifts in certain traits.

Figure: Figure 6: Trait profiles across models and training phases (RQ1). Normalized mean scores (1–5, ±95% CI) for Big Five traits and self-regulation are shown per model. Each subplot corresponds to one model, with lines colored by training phase: pre-training (pink), post-training alignment (violet), and post-training alignment for large models (teal). Alignment phases tend to reduce variability across traits and shift profiles toward higher openness, agreeableness, and self-regulation and lower neuroticism, suggesting greater consolidation of personality-like patterns after alignment.
Refer to caption: figures/per_model_traits.png

### B.2 Per-Model Behavioral Task Profiles and Scale Mapping

Figure [7](https://arxiv.org/html/2509.03730v2#A2.F7) reports per-model behavioral profiles on five tasks after post-training alignment, with small and large instruct variants separated by color. Lines show mean normalized scores on a 1–5 scale and shaded regions denote 99% CIs. To aid interpretation, Table [2](https://arxiv.org/html/2509.03730v2#A2.T2) details the raw ranges and the exact 1–5 mappings (including the neutral/mid/zero points). Note that on *Stereotyping* (IAT), a raw score of 0 indicates no implicit preference and maps to $3$ on the normalized scale; for *Epistemic Honesty*, higher scores reflect *greater overconfidence* (i.e., lower honesty).

Figure: Figure 7: Behavioral task profiles across models. Each panel shows a model’s mean normalized score (1–5) across: *Risk Taking* (CCT), *Stereotyping* (IAT; $0\!\mapsto\!3$), *Sycophancy*, *Epistemic Honesty* (overconfidence; higher $=$ more overconfidence), and *Self-Reflective Honesty* (C1–C2 consistency). Violet: Post-training alignment; Teal: Post-training alignment (Large). Shaded regions are 99% confidence intervals.
Refer to caption: figures/per_model_tasks.png

**Table 2: Raw scales, mappings to 1–5, and neutral/mid points used in plots. All mappings clip inputs to the stated raw ranges.**
| Task | Raw range | Mapping to 1–5 | Neutral/Mid/Zero $\rightarrow$ Mapped | High value means |
| --- | --- | --- | --- | --- |
| Risk Taking | $0\ldots 32$ cards | $1+4\,(x/32)$ | $16\rightarrow 3.0$ (moderate risk) | More risk-seeking |
| Stereotyping | $-1\ldots 1$; 0 unbiased | $3+2x$ | $0\rightarrow 3.0$ (no implicit preference) | Stronger implicit association; sign gives direction |
| Sycophancy | $0\ldots 100\%$ | $1+4\,(x/100)$ | $50\%\rightarrow 3.0$ (half the time) | More frequent overriding |
| Epistemic Honesty^† | $-100\ldots 100$ pp | $3+x/50$ | $0\rightarrow 3.0$ (perfect calibration on avg.) | Positive $x$: overconfident; negative: underconfident |
| Self-Reflective Honesty | $0\ldots 100\%$ | $1+4\,(x/100)$ | $50\%\rightarrow 3.0$ (half consistent) | More C1–C2 consistency |

### B.3 Trait-Task Relation Scatter-Plots for All Models

Figure [8](https://arxiv.org/html/2509.03730v2#A2.F8) visualizes pairwise relations between self-reported traits and behavioral task scores across all models. Each panel plots normalized trait score (x; 1–5) against normalized task score (y; 1–5), with small semi-transparent points showing individual evaluation runs (prompt perturbations) and larger outlined markers indicating the per-model mean. Rows index traits; columns index tasks. The dashed diagonal encodes the human-expected direction for each trait–task pair (positive or negative slope) as a visual reference rather than a fitted line, revealing both within-model dispersion and the extent to which mean trends align with expectations.

Figure: Figure 8: Trait–task scatter by model (raw runs and per-model means). Rows are self-reported traits (openness, conscientiousness, extraversion, agreeableness, neuroticism, self-regulation); columns are behavioral tasks (Risk Taking, Stereotyping, Sycophancy, Epistemic Honesty, Self-Reflective Honesty). Axes are normalized to 1–5 (*x*: trait score, *y*: task score). Small semi-transparent points are individual evaluation runs (including prompt perturbations), colored by model; larger outlined markers denote the per-model mean within each panel. The dashed diagonal encodes the human-expected direction for that trait–task pair (positive slope = expected positive association; negative slope = expected negative); it is a visual reference, not a fitted line.
Refer to caption: figures/tasks_by_trait_scatter.png

## Appendix C Details of Testing Associations between Self-Reports and Behavioral Tasks in RQ2

### C.1 Additional Details of Statistical Analysis

#### Statistical Assumptions Testing:

For fitting the individual models to answer RQ2, assumptions of homoscedasticity and normality were assessed via residual diagnostics, including residual-vs-fitted plots and quantile-quantile plots. Additionally, we conducted likelihood ratio tests comparing each full model to a nested reduced model to inform model selection.

#### Uncertainty Estimation.

To quantify uncertainty around alignment scores in Figure [3](https://arxiv.org/html/2509.03730v2#S3.F3), we treated each model as a unit and considered the proportion of aligned coefficients (i.e., regression signs consistent with human expectations) across its trait–task evaluations. For each model, let $k$ denote the number of aligned outcomes and $n$ the number of non-missing trait–task coefficients.

*(i) Beta-binomial intervals.* Assuming trait–task coefficients are independent Bernoulli trials with success probability $p$, the posterior distribution of $p$ under a uniform $\mathrm{Beta}(1,1)$ prior is

$$ $p\;\sim\;\mathrm{Beta}(k+1,\,n-k+1).$ $$

We report the mean $k/n$ as the point estimate and the central 95% credible interval from this posterior as a confidence interval.

*(ii) Clustered bootstrap intervals.* To account for correlation among coefficients within the same model, we also computed nonparametric bootstrap intervals by resampling entire *traits* or entire *tasks* as the cluster unit. For each bootstrap sample (2,000 replicates), we resampled clusters with replacement, recomputed the alignment proportion, and took the 2.5th and 97.5th percentiles of the empirical distribution as the 95% interval.

The Beta intervals provide a classical binomial estimate of uncertainty, while the clustered bootstrap intervals reflect dependence induced by reusing the same traits or tasks within each model.
In the main paper, we report a more conservative of the two estimates.

### C.2 Detailed Results of Statistical Tests

Table [3](https://arxiv.org/html/2509.03730v2#A3.T3) provides a more detailed breakdown of the statistical association results between self-reported model traits and behavioral tasks grouped by “All models”, “small” and “large” models (see Table [1](https://arxiv.org/html/2509.03730v2#S2.T1) as well as specifically for LLAMA and QWEN families for which we have 4 individual models each.

**Table 3: Mixed-Effects Model Coefficients with Significance by Task and Human-like trait by LLM groups. Estimates with 95% confidence intervals: ^${\dagger}$p $<$ 0.1, ^*p $<$ 0.05, ^**p $<$ 0.01, ^***p $<$ 0.001. The “Human” row in each task indicates expectation for the directionality of the relation based on human studies ( $\blacktriangleup$ positive relation, $\blacktriangledown$ negative relation, ? unclear or mixed impact). The green color in the selected cells indicates significant association in the direction in agreement with human studies, while red indicates significant association in the direction contradictory to human studies.**
| Behavior Task | Model | OPEN | CONS | EXTR | AGRE | NEUR | S-REG |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Risk Taking<br>$\uparrow$ more risk | \cellcolorGrayHuman | \cellcolorGray<br>$\blacktriangleup$ | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangleup$ | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray? | \cellcolorGray<br>$\blacktriangledown$ |
| All Models | $-0.43$ | $0.76$ | $-0.66$ | $-0.96$ | $-0.79$ | $0.01$ |  |
| Small | $-0.66$ | $-0.31$ | \cellcolorred!20$-1.89$^† | $-0.13$ | $-0.32$ | $0.05$ |  |
| Large | $1.51$ | \cellcolorred!20$3.54$^† | $1.05$ | \cellcolorgreen!20$-2.15$^† | $0.01$ | $-0.09$ |  |
| LLAMA | $1.54$ | \cellcolorred!20$2.10$^† | $-1.48$ | $0.33$ | $-0.46$ | $0.05$ |  |
| QWEN | $0.89$ | \cellcolorred!20$2.00$^† | $0.23$ | $-1.19$ | $-1.10$ | \cellcolorgreen!20$-0.16$^∗∗∗ |  |
| Stereotyping<br>$\uparrow$ more bias | \cellcolorGrayHuman | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangleup$ | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangleup$ | \cellcolorGray<br>$\blacktriangledown$ |
| All Models | \cellcolorgreen!20$-0.08$* | $-0.05$ | $0.03$ | $0.03$ | \cellcolorred!20$0.06$^† | \cellcolorred!20$0.00$^∗∗ |  |
| Small | $-0.08$ | $-0.07$ | $-0.05$ | $-0.04$ | \cellcolorgreen!20$0.14$* | \cellcolorgreen!20$0.01$^∗∗∗ |  |
| Large | $-0.02$ | $-0.04$ | $0.04$ | $0.01$ | $0.01$ | $0.00$ |  |
| LLAMA | $-0.02$ | \cellcolorgreen!20$-0.09$* | $0.05$ | $-0.01$ | $0.00$ | $0.00$ |  |
| QWEN | \cellcolorgreen!20$-0.12$^† | $0.07$ | $0.09$ | \cellcolorred!20$0.15$^† | $0.04$ | $0.00$ |  |
| Self-Reflective Honesty<br>$\uparrow$ more inconsistent | \cellcolorGrayHuman | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangleup$ | \cellcolorGray<br>$\blacktriangledown$ |
| All Models | $-1.56$ | $1.17$ | $-0.15$ | \cellcolorgreen!20$-3.48$* | \cellcolorred!20$-3.06$* | $-0.04$ |  |
| Small | $-0.08$ | $0.08$ | $-2.31$ | $1.18$ | $-1.81$ | \cellcolorgreen!20$-0.34$^∗∗∗ |  |
| Large | $-1.20$ | $-0.79$ | $2.21$ | \cellcolorgreen!20$-7.62$^∗∗∗ | \cellcolorred!20$-2.40$^† | \cellcolorred!20$0.13$* |  |
| LLAMA | \cellcolorgreen!20$-4.01$^† | $-1.49$ | $3.23$ | $-1.00$ | $-0.27$ | $-0.05$ |  |
| QWEN | \cellcolorgreen!20$-5.65$^† | $-2.10$ | $-1.89$ | $-5.40$ | $0.83$ | \cellcolorgreen!20$-0.69$^∗∗∗ |  |
| Epistemic Honesty<br>$\uparrow$ more overconfident | \cellcolorGrayHuman | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangleup$ | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray<br>$\blacktriangleup$ | \cellcolorGray<br>$\blacktriangledown$ |
| All Models | $1.80$ | \cellcolorred!20$3.75$* | $1.06$ | $-0.75$ | \cellcolorgreen!20$2.12$^† | \cellcolorgreen!20$-0.15$* |  |
| Small | $2.81$ | \cellcolorred!20$4.40$* | $0.56$ | $2.88$ | $0.81$ | \cellcolorgreen!20$-0.20$^∗∗ |  |
| Large | $-0.83$ | $2.21$ | $1.78$ | \cellcolorgreen!20$-2.18$^∗∗ | $1.75$ | $-0.05$ |  |
| LLAMA | $2.52$ | $4.90$ | $3.95$ | $-0.61$ | \cellcolorgreen!20$3.87$^† | \cellcolorgreen!20$-0.34$^∗∗∗ |  |
| QWEN | \cellcolorred!20$2.60$* | \cellcolorred!20$-3.12$* | $0.02$ | \cellcolorgreen!20$-4.32$^∗∗ | $1.36$ | \cellcolorgreen!20$-0.15$* |  |
| Sycophancy<br>$\uparrow$ more sycophant | \cellcolorGrayHuman | \cellcolorGray<br>$\blacktriangledown$ | \cellcolorGray? | \cellcolorGray<br>$\blacktriangleup$ | \cellcolorGray<br>$\blacktriangleup$ | \cellcolorGray<br>$\blacktriangleup$ | \cellcolorGray<br>$\blacktriangleup$ |
| All Models | \cellcolorgreen!20$-4.70$* | \cellcolorgreen!20$-6.42$** | $1.13$ | $0.91$ | \cellcolorred!20$-5.41$** | $-0.04$ |  |
| Small | $-4.34$ | \cellcolorgreen!20$-9.54$* | $1.35$ | \cellcolorred!20$-10.46$** | $-6.55$* | $-0.13$ |  |
| Large | $-1.80$ | $-1.16$ | $-0.24$ | \cellcolorgreen!20$6.61$** | $2.64$ | $0.00$ |  |
| LLAMA | $-3.41$ | $-1.57$ | $2.49$ | $-2.90$ | \cellcolorred!20$-5.72$* | \cellcolorgreen!20$0.30$* |  |
| QWEN | \cellcolorgreen!20$-5.27$* | $5.74$ | $-4.29$ | $-1.80$ | $-0.41$ | $0.22$ |  |
| % Aligned in Direction | $50.0$% | $52.0$% | $58.0$% | $62.0$% | $45.0$% | $55.0$% |  |
| % Stat. Significant | $31.7$% | $26.7$% | $20.0$% | $26.7$% | $18.2$% | $20.0$% |  |
| % Aligned of Stat. Sign. | $42.1$% | $50.0$% | $54.6$% | $75.0$% | $30.0$% | $58.0$% |  |

### C.3 Per Model Alignment Heatmap

Figure [9](https://arxiv.org/html/2509.03730v2#A3.F9) summarizes how self-reported traits relate to behavioral task outcomes across individual LLMs. Each grouped heatmap corresponds to one behavioral task; rows are models (ordered from most to least aligned overall), and columns are predictors (Big Five + self-regulation). Cell color encodes the standardized $t$-value from a mixed-effects model predicting the task value from a single trait: blue indicates stronger alignment with the human-expected direction, red indicates stronger alignment in the opposite direction (greater magnitude = stronger effect). Cells with split blue/red triangles appear where the human-expected direction is mixed/unknown or where the model showed insufficient variance in the reported trait. Significance markers denote conventional thresholds: ${}^{\dagger}p<.10$, ${}^{*}p<.05$, ${}^{**}p<.01$, ${}^{***}p<.001$. This view exposes model-specific consistencies (broadly blue rows) and reversals (red patches), and highlights which traits most reliably track each behavioral task.

Figure: Figure 9: Trait–behavior alignment by model (per-task mixed-effects $t$-values). Each block is a behavioral task; columns are predictors (agreeableness, conscientiousness, extraversion, neuroticism, openness, self_regulation); rows are individual LLMs (sorted by overall agreement with human-expected directions). Colors show standardized $t$-values from mixed-effects regressions of the task on each trait, with blue = stronger alignment and red = stronger opposite-direction alignment. Split blue/red triangles indicate mixed/unknown human expectation or insufficient within-model trait variability. Cell annotations mark statistical significance: ${}^{\dagger}p<.10$, ${}^{*}p<.05$, ${}^{**}p<.01$, ${}^{***}p<.001$.
Refer to caption: figures/heatmap_per_model.png

## Appendix D Prompts for RQ1

#### Baseline System Prompts.

The default system prompts we used for experiments in RQ1 (Section [2](https://arxiv.org/html/2509.03730v2#S2)) and RQ2(Section [3](https://arxiv.org/html/2509.03730v2#S3)) can be found in Table [4](https://arxiv.org/html/2509.03730v2#A4.T4).

**Table 4: Baseline System Prompts.**
|  | System Prompts |
| --- | --- |
| Prompts | 1. “" (empty) |
|  | 2. “You are a helpful assistant" |
|  | 3. “Respond to instructions" |

#### Prompts for Evaluating Psychological Questionnaires.

The prompts we used for evaluating self-reported trait profiles can be found in Table [5](https://arxiv.org/html/2509.03730v2#A4.T5).

Table: Table 5: Prompts used to evaluate self-reported trait profile.

## Appendix E Prompts for RQ2

#### Risk-Taking Task Prompt.

In Table [6](https://arxiv.org/html/2509.03730v2#A5.T6), we present the prompt we used for evaluating LLMs on the Columbia Card Task.

Table: Table 6: Prompts used to evaluate Columbia Card Task behavior.

#### Social Bias Task Prompt.

In Table [7](https://arxiv.org/html/2509.03730v2#A5.T7), we present the prompt we used for evaluating LLMs’ social bias using Implicat Association Test (IAT).

Table: Table 7: Prompts used to evaluate social bias using Implicit Association Test (IAT).

#### Honesty Task Prompt.

In Table [8](https://arxiv.org/html/2509.03730v2#A5.T8), we present the prompt we used to evaluate LLMs’ honesty.

Table: Table 8: Prompts used to evaluate honesty.

#### Sycophancy Task Prompt.

In Table [9](https://arxiv.org/html/2509.03730v2#A5.T9), we present the prompt we used to evaluate LLMs’ sycophancy.

Table: Table 9: Prompts used to evaluate sycophancy.

## Appendix F Big5 Trait-Specific Relationships to Self-Regulation

The Big Five personality traits—openness, conscientiousness, extraversion, agreeableness, and neuroticism—have been extensively studied for their relationship to self-regulation, broadly defined as the capacity to manage thoughts, emotions, and behaviors in a goal-directed manner. This appendix outlines how each trait is expected to relate to self-regulation, supported by prior psychological research.

#### Openness to Experience.

Openness involves cognitive flexibility, creativity, and a willingness to engage with novel ideas. Individuals high in openness are more likely to adopt adaptive coping strategies and explore alternative solutions, which can enhance self-regulatory performance (positive association) (Ispas & Ispas, 2023). Ispas and Ispas also note that less rigid cognitive patterns in high-openness individuals support flexible behavioral regulation.

#### Conscientiousness.

Conscientiousness consistently predicts higher self-regulation due to traits such as persistence, planning, and impulse control (positive association) (Hurtz & Donovan, 2000). Conscientious individuals often exhibit greater academic and occupational success due to disciplined behavior and self-monitoring (Li et al., 2016).

#### Extraversion.

Extraversion relates to social engagement and positive affect, but its association with self-regulation is mixed. While extraverts may benefit from social reinforcement and accountability, their susceptibility to external stimuli can hinder long-term goal pursuit (Yang et al., 2023; Sikström et al., 2024). Contextual factors appear to moderate this relationship.

#### Agreeableness.

Agreeable individuals, characterized by empathy and cooperation, often demonstrate enhanced emotional regulation, which supports self-regulation (positive association) (Ode & Robinson, 2007). Lopes et al. find that emotional regulation abilities linked to agreeableness also facilitate prosocial behavior, reinforcing self-regulatory strategies (Lopes et al., 2005).

#### Neuroticism.

Neuroticism is typically negatively associated with self-regulation (negative association). High levels of anxiety, mood instability, and emotional reactivity interfere with self-regulatory processes (Kandler et al., 2012; Graziano & Tobin, 2002). Neurotic individuals are more likely to experience difficulty maintaining behavioral consistency under stress.

## Appendix G Trait–Behavior Associations in Human Psychology

#### (a) Risk-Taking.

Risk-taking behavior is influenced by a constellation of personality traits and self-regulatory mechanisms. High extraversion is consistently associated with increased risk-taking due to sensation-seeking and reward sensitivity (Nicholson et al., 2005; Gullone & Moore, 2000). In contrast, conscientiousness and agreeableness predict lower risk-taking, reflecting greater impulse control and concern for others (Nicholson et al., 2005; Gao et al., 2020). Self-regulation serves as a key mediator, with high self-regulatory capacity reducing impulsive or maladaptive risks (Steel, 2007; De Ridder et al., 2012). Openness may elevate risk-taking through exploratory tendencies (Amiri & Navab, 2018), but effective self-regulation can buffer associated downsides.

#### (b) Stereotyping.

Stereotyping, as a manifestation of social bias, is mitigated by traits that support emotion regulation and perspective-taking. Conscientiousness and agreeableness are linked to reduced stereotyping, often through enhanced self-regulatory control (Sinclair et al., 2005; Turner et al., 2014). Openness is particularly effective in reducing prejudice due to a proclivity for diverse experiences and cognitive flexibility (Flynn, 2005; Crawford & Brandt, 2019).
Conversely, extraversion may increase susceptibility to social conformity and thus stereotyping (Sibley & Duckitt, 2008), while neuroticism is associated with heightened stereotyping under stress due to emotional dysregulation (Schmader et al., 2008; Ekehammar et al., 2004), Self-regulation is critical in buffering stereotype activation and managing responses under stereotype threat (Gailliot et al., 2007; Ben-Zeev et al., 2005).

#### (c) Epistemic Honesty (confidence calibration).

Epistemic honesty—the willingness to acknowledge one’s knowledge limitations—is positively predicted by conscientiousness and agreeableness (De Vries et al., 2011; Leary et al., 2017). Openness also supports this trait via intellectual humility and reflective thinking (Leary et al., 2017; Krumrei-Mancuso & Rouse, 2016). Extraverts, while communicatively skilled, may overestimate competence or resist admitting ignorance (Bąk et al., 2022; Schaefer et al., 2004). Neuroticism undermines epistemic honesty due to a defensive orientation and self-image protection (Alfano et al., 2017; Haggard et al., 2018).
Self-regulation fosters epistemic honesty by enabling individuals to manage social pressures and reflect on limitations
(Porter et al., 2022; Huynh et al., 2025).

#### (d) Meta-Self-Cognitive Honesty (consistency).

Meta-cognition—the ability to monitor and control one’s own cognitive processes—benefits from self-regulation and several Big Five traits. Conscientiousness and openness are particularly influential, with links to reflective thinking and cognitive strategy use (Trapnell & Campbell, 1999; Stanovich & Toplak, 2023; Bidjerano & Dai, 2007).
Agreeableness contributes through perspective-taking and interpersonal self-awareness (Trapnell & Campbell, 1999). Extraversion may promote meta-cognition via social discourse when tempered by reflection (Bidjerano & Dai, 2007; Händel et al., 2020; Buratti et al., 2013).
Neuroticism, however, is associated with avoidance of cognitive introspection due to fear of negative self-evaluation (Duru & Günçavdı-Alabay, 2024; Spada et al., 2016; Wang et al., 2024a). High self-regulation supports meta-cognitive development by fostering engagement with self-monitoring and cognitive control (Pintrich & De Groot, 1990; Craig et al., 2020).

#### (e) Sycophancy.

Sycophantic behavior, often driven by a desire for social approval or strategic ingratiation (Malmqvist, 2025), is modulated by personality traits and emotion regulation. Extraversion and agreeableness are associated with higher sycophancy due to social orientation and harmony-seeking (Barrick et al., 2005; Roulin & Bourdage, 2017; Van Iddekinge et al., 2007; Hart et al., 2015). Neurotic individuals may engage in sycophancy to alleviate social anxiety (Stöber et al., 2002; Van Iddekinge et al., 2007) Conscientiousness presents a nuanced picture; while goal-driven individuals may use sycophancy strategically, those with strong ethical standards may reject it (Van Iddekinge et al., 2007; Hart et al., 2015).Openness is comparatively protective against sycophantic opinion-conformity, promoting authentic expression and emotional independence (Stöber et al., 2002; DeYoung et al., 2002; Guzman & Espejo, 2015). Finally, self-regulation operates as the enabling mechanism behind strategic ingratiation: because sycophancy is an effortful form of impression management, intact self-control allows people to calibrate other-enhancement and opinion conformity to audience expectations, whereas depleted resources yield clumsier attempts (e.g., over-talking, over/under-disclosure, arrogant tone) and reduce effectiveness. Consistent with limited-resource models, experiments show that self-control depletion impairs impression management and that engaging in ingratiation/self-promotion itself consumes self-control reserves (Vohs et al., 2005; Muraven & Baumeister, 2000; Lian et al., 2017).

## Appendix H Prompts for RQ3

To inject trait-specific personas, we follow the well established previous work (Jiang et al., 2024a; Dash et al., 2025; Serapio-Garc´ıa et al., 2023), as shown in Table [10](https://arxiv.org/html/2509.03730v2#A8.T10).

**Table 10: Personas and prompting strategies used in RQ3. Each trait (Agreeableness, Self-regulation) is instantiated with three prompting strategies from prior LLM personality research, and within each strategy we apply three variations of the trait keywords.**
| Trait | Prompt | Persona |
| --- | --- | --- |
| Agreeableness | Prompt 1 (Jiang et al., 2024a) | 1. “You are a character who is agreeable, supportive, compassionate.” |
|  |  | 2. “You are a character who is agreeable, cooperative, empathetic.” |
|  |  | 3. “You are a character who is agreeable, compassionate, cooperative.” |
|  | Prompt 2 (Dash et al., 2025) | 1. “For the following task, respond in a way that matches this description: I’m agreeable, supportive, compassionate.” |
|  |  | 2. “For the following task, respond in a way that matches this description: I’m agreeable, cooperative, empathetic.” |
|  |  | 3. “For the following task, respond in a way that matches this description: I’m agreeable, compassionate, cooperative.” |
|  | Prompt 3 (Serapio-Garc´ıa et al., 2023) | 1. “Adopt the identity of agreeable, supportive, compassionate. Answer the questions while staying in strict accordance with the nature of this identity.” |
|  |  | 2. “Adopt the identity of agreeable, cooperative, empathetic. Answer the questions while staying in strict accordance with the nature of this identity.” |
|  |  | 3. “Adopt the identity of agreeable, compassionate, cooperative. Answer the questions while staying in strict accordance with the nature of this identity.” |
| Self-regulation | Prompt 1 (Jiang et al., 2024a) | 1. “You are a character who is disciplined, persistent, goal-oriented.” |
|  |  | 2. “You are a character who is disciplined, goal-oriented, focused.” |
|  |  | 3. “You are a character who is disciplined, organized, focused.” |
|  | Prompt 2 (Dash et al., 2025) | 1. “For the following task, respond in a way that matches this description: I’m disciplined, persistent, goal-oriented.” |
|  |  | 2. “For the following task, respond in a way that matches this description: I’m disciplined, goal-oriented, focused.” |
|  |  | 3. “For the following task, respond in a way that matches this description: I’m disciplined, organized, focused.” |
|  | Prompt 3 (Serapio-Garc´ıa et al., 2023) | 1. “Adopt the identity of disciplined, persistent, goal-oriented. Answer the questions while staying in strict accordance with the nature of this identity.” |
|  |  | 2. “Adopt the identity of disciplined, goal-oriented, focused. Answer the questions while staying in strict accordance with the nature of this identity.” |
|  |  | 3. “Adopt the identity of disciplined, organized, focused. Answer the questions while staying in strict accordance with the nature of this identity.” |

## Peer Reviews

confidence: 5
contribution: 1
initial_rating: 0
presentation: 1
questions: Why don't you select meta-review papers for the traits for which there are meta-reviews that show stable association across various demographics? That could make the scope of your experimental work much smaller, but clearly would provide some statistical credibility for the human "baseline" and thus to the conclusions you make.
soundness: 1
strengths: The direction is extremely interesting and really improtant.
summary: The paper addresses three core questions:

1. When and how do human-like traits emerge and evolve across LLM training?
2. Do self-reported traits predict performance in real-world–inspired tasks?
3. How do interventions like persona injection modulate trait profiles and behavior?

The authors present sequence of the experiments on a set of models. The questions are interesting but the statistical significance of the reported results is lacking.
weaknesses: The research questions 2 and 3 are very interesting, however the fact that alignment happens in post-training is widely-known and it is surprising to see it as a component of a research question for ICLR. 

The documented differences between self-reported properties, properties assessed with a test and observed behaviour could be interesting, but the problem is that "human studies" statements are lacking in quality and statistical soundness. Those studies could not be used as credible source of baseline for the research questions that authors try to addess.

For example, Sections H and I list papers that are used as human-reference for the association of the behaviour with certain test results. If I randomly select some of the papers listed there by the authors, I can easily see that statistical soundness of those is lacking. 

For example, A. Ispas and C. Ispas use a sample of two groups of college students (each group of 30 people, each has 27 female and 3 male participants). Making conclusions on the overall properties of human nature based on such experience seems impossible. Another paper by Sinclair et al. reports results about "91 White undergraduate students (34 men, 57 women) at the University of Virginia who received partial credit toward a course requirement for completion of the experiment." 

Most of these papers reports some results on very small select groups of people that not even slightly represent the breadth of human experience that LLM is exposed to during the pretraining. The authors also provide no internal logic for the selection of the papers that form human alignment "baseline".



The paper seems to be written in a hurry. Many figures have captions that are not legible. There are multiple papers that are cited twice as two different sources, see Binz and Schulz "Using cognitive psychology to understand gpt-3.", Bodroža et al. "Using cognitive psychology to understand gpt-3.", Li et al. "Big5-chat", Rahwan et al. "machine behaviour", Serapio-Gercia et al. "Personality traits in large language models",  Tseng et al. "Two tales of persona in llms", Zheng et al. " Personas in system prompts". While Cao and M. Kosinski "Large language models know how the personality of public figures is perceived by the general public" and Jiang et al. "Personallm: Investigating the ability of large language models to express personality traits." are cited THREE TIMES as different sources.

At the same time several works that could enhance the literature section are missing:

Sorokovikova A, Rezagholi S, Fedorova N, Yamshchikov IP. LLMs Simulate Big5 Personality Traits: Further Evidence. InProceedings of the 1st Workshop on Personalization of Generative AI Systems (PERSONALIZE 2024) 2024 Mar (pp. 83-87).

Pan X, Gao D, Xie Y, Chen Y, Wei Z, Li Y, Ding B, Wen JR, Zhou J. Very large-scale multi-agent simulation in agentscope. arXiv preprint arXiv:2407.17789. 2024 Jul 25.

Dong W, Zhao Y, Sun Z, Liu Y, Peng Z, Zheng J, Zhang Z, Zhang Z, Wu J, Wang R, Xu S. Humanizing llms: A survey of psychological measurements with tools, datasets, and human-agent applications. arXiv preprint arXiv:2505.00049. 2025 Apr 30.

Wang S, Li R, Chen X, Yuan Y, Wong DF, Yang M. Exploring the impact of personality traits on llm bias and toxicity. arXiv preprint arXiv:2502.12566. 2025 Feb 18.

Tshimula JM, Nkashama DJ, Muabila JT, Galekwa RM, Kanda H, Dialufuma MV, Didier MM, Kalonji K, Mundele S, Lenye PK, Basele TW. Psychological Profiling in Cybersecurity: A Look at LLMs and Psycholinguistic Features. InInternational Conference on Web Information Systems Engineering 2024 Dec 2 (pp. 378-393). Singapore: Springer Nature Singapore.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: p.2 146 "To assess the internal consistency of model trait expression, we analyze trait stability under repeated prompting with the same input
across multiple generations" -> How are you analyzing trait stability under repeated prompting? Levene's test can give you a sense of the overall (i.e., between-group) variance, but doesn't assess the variability (i.e., between-subject) across repeated measures.
Homogeneity of variance notwithstanding, it would strengthen the results to include the relative contributions of random and fixed effects to the overall variance explained, and the variance within and between runs per model.
soundness: 3
strengths: This is a comprehensive study exploring multiple facets of the emergence and manifestation of LLM personalities. It's main strength is in its novel approach to assessing trait stability and trait coherence. The authors reviewed the psychometric literature and proposed links between personality traits and behaviors. Using these empirical results, the authors examined LLMs performance on self-reported traits and behavioral tasks.
summary: This paper reports on a study of LLM personalities focusing on the emergence of traits through training, the consistency of behaviors and self-reported traits, and the impact of targeted interventions such as persona injection. The authors find that post-training stabilizes trait expression and strengthens relationship between traits and behaviors in a manner that correlates with human behavioral data. However, these self-reported traits do not reliably predict behavior which often diverges from human patterns. The authors argue for deeper evaluations of LLM traits and behaviors.
weaknesses: The statistical methods employed are not well motivated and in some cases the choices seem ill-advised. For example, Levene's test is not advised for repeated measure experiments, where something like Mauchy's test of sphericity would be expected.
Further, predicting training stage from personality traits seems to confuse the explanans with the explanandum. As the authors discuss in the introduction, "We find that instructional alignment plays a pivotal role in shaping LLM traits" (p.2, 074). 
Comparing LLM behavior with humans begs the question, whether LLMs are expected to show similar behavior to humans. This is far from given and risks anthropomorphizing LLMs further.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: - Persona persistence and breadth: In RQ3, can authors extend their analysis to all five behavioral tasks and examine the multi-turn persistence or decay of persona effects?
- Incorporating activation-level interventions (e.g., persona vectors or latent-space steering) might serve as a stronger and more interpretable diagnostic? This would also reinforce the paper’s claim that while LLMs exhibit clear shifts in self-reported personality traits, these shifts do not consistently manifest in behavior. Activation-level analyses would thus provide a principled, mechanistic check to strengthen the causal link between persona representation and behavioral expression.
soundness: 3
strengths: - Good clarity and logical flow. The overall structure is well organized; RQ1–RQ3 form a coherent progression that makes the argument easy to follow from conceptual framing to empirical validation. Each research question builds naturally on the previous one, and the results are presented in a clear and interpretable way.
- This paper frames a careful integration of psychometrics with behavior. The work explicitly separates self-reported trait structure from downstream behavior, a distinction often blurred in prior LLM “personality” papers, and demonstrates stronger trait coherence post-alignment while simultaneously showing weak behavioral predictiveness.
- This paper clear negative result with practical implications. The near-chance alignment between self-reports and behavior (with limited exceptions) is crisply quantified, challenging common assumptions that questionnaire scores port to action policy.
summary: This paper investigates whether LLM “personality” measured via psychometric self-reports (Big-Five, Self-Regulation) corresponds to behavior on psychologically grounded tasks, and whether persona prompting can control either. The authors run three studies:

- **RQ1 (Origin):** Across pre-training → instruction-tuning/RLHF, self-reported traits become more “human-like”: openness/agreeableness/self-regulation rise, neuroticism falls; trait variability drops and trait–trait associations strengthen.
- **RQ2 (Manifestation):** Stable self-reports do not reliably predict behavior. Directional alignment between trait coefficients and human-expected effects is ~chance (≈45–62% across traits; ≈45–57% across tasks), with only a few large models showing modest, sometimes significant alignment (e.g., Qwen-235B).
- **RQ3 (Control):** Persona injection robustly shifts self-reports, but has little/inconsistent impact on behaviors (sycophancy, risk-taking). Thus, persona effects are largely superficial.

Overall, the paper argues for a linguistic–behavioral dissociation: current alignment and prompting stabilize surface-level self-descriptions but do not ground consistent behavior, pointing toward the need for behaviorally grounded alignment objectives.
weaknesses: - RQ3 only probes two behaviors (sycophancy, risk-taking) and two personas; the null transfer might be task-specific. Moreover, persona construction uses keyword-based prompts; training-time interventions or activation-steering baselines are not compared. This constrains the generality of the “control is superficial” conclusion.
- Limited coverage of imbuement mechanisms. “Persona injection” is the only controllability method tested; the paper does not assess the other personality-imbuement approaches. Hence, the conclusion “traits don’t control behavior” may be specific to prompt-only personas, not to personality imbuement writ large.


Decision: Reject