From Five Dimensions to Many: Large Language Models as Precise and Interpretable Psychological Profilers
Authors: Yi-Fei Liu
Yi-Long Lu
Di He
Hang Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Psychological Profiling
Human Simulation
Zero-Shot Prediction
Reasoning Trace Analysis
Primary area: applications to neuroscience & cognitive science

Abstract: Psychological constructs within individuals are widely believed to be interconnected. We investigated whether and how Large Language Models (LLMs) can model the correlational structure of human psychological traits from minimal quantitative inputs. We prompted various LLMs with Big Five Personality Scale responses from 816 human individuals to role-play their responses on nine other psychological scales. LLMs demonstrated remarkable accuracy in capturing human psychological structure, with the inter-scale correlation patterns from LLM-generated responses strongly aligning with those from human data (R² > 0.88). This zero-shot performance substantially exceeded predictions based on semantic similarity and approached the accuracy of machine learning algorithms trained directly on the dataset. Analysis of reasoning traces revealed that LLMs use a systematic two-stage process: First, they transform raw Big Five responses into natural language personality summaries through information selection and compression, analogous to generating sufficient statistics. Second, they generate target scale responses based on reasoning from these summaries. For information selection, LLMs identify the same key personality factors as trained algorithms, though they fail to differentiate item importance within factors. The resulting compressed summaries are not merely redundant representations but capture synergistic information—adding them to original scores enhances prediction alignment, suggesting they encode emergent, second-order patterns of trait interplay. Our findings demonstrate that LLMs can precisely predict individual participants' psychological traits from minimal data through a process of abstraction and reasoning, offering both a powerful tool for psychological simulation and valuable insights into their emergent reasoning capabilities.

Published as a conference paper at ICLR 2026
FROM FIVE DIMENSIONS TO MANY: LARGE LAN- GUAGE MODELS AS PRECISE AND INTERPRETABLE PSYCHOLOGICAL PROFILERS
Yi-Fei Liu', Yi-Long Lu², Di He & Hang Zhang
ABSTRACT
Psychological constructs within individuals are widely believed to be intercon- nected. We investigated whether and how Large Language Models (LLMs) can model the correlational structure of human psychological traits from minimal quantitative inputs. We prompted various LLMs with Big Five Personality Scale responses from 816 human individuals to role-play their responses on nine other psychological scales. LLMs demonstrated remarkable accuracy in capturing hu- man psychological structure, with the inter-scale correlation patterns from L.M-- generated responses strongly aligning with those from human data (R² > 0.88). This zero-shot performance substantially exceeded predictions based on seman- tie similarity and approached the accuracy of machine learning algorithms trained directly the dataset. Analysis of reasoning traces revealed that LLMs use systematic two-stage process: First, they transform raw Big Five responses into natural language personality summaries through information selection and com- pression. Second, they generate target scale responses based on reasoning from these summaries. For information selection, L.MMs identify the same key person- ality factors as trained algorithms, though they fail to differentiate item importance within factors. The resulting compressed summaries are not merely redundant rep- resentations but capture synergistic information information-aadding them to original scores enhances prediction aligmment, suggesting they encode emergent, second-order patterns of trait interplay. Our findings demonstrate that LLMs can precisely pre- dict individual participants psychological traits from minimal data through a pro- cess of abstraction and reasoning, offering both powerful tool for psychological simulation and valuable insights into their emergent reasoning capabilities.
INTRODUCTION
Understanding the human nomothetic network-the complex web of how psychological traits cor- relate (Cronbach & Meehl 1955) is a central ambition of psychology and a key frontier in compu- tational social science (Karvelis et al., 2023] Ziems et al., 2024). Capturing this structure accurately thus serves as profound benchmark for Al's ability to reason about human nature. Large Lan- guage Models (LLMs) have emerged as promising candidates (Serapio-García et al.] 2023), demon- strating powerful capabilities for group-level simulation in social and economic scenarios (Aher cet al. 2023 Dillion et al., 2023). Concurrently, advancements in individual-level simulation have enabled models to simulate specific personality traits (Jiang al. 2024) using techniques like in- context learning (Choi & [i] (024). Furthermore, these capabilities have been integrated into large- scale agent frameworks (Pan al.) 2024), with landmark studies demonstrating L.MSs' potential for mimicking real-world individuals with high fidelity (Park et al. 2024).. Despite these successes in mimicking behaviors, the source of this capability remains critical open question (Dong al.
"Peking-Tsinghua Center for Life Sciences, Peking University: State Key Laboratory of General Arti- ficial Intelligence. BIGAI, Beijing. China; 'State Key Laboratory of General Artificial Intelligence, Peking University: 'School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Hcalth, Peking University; SSehool of Intelligence Science and Technology, Peking University: *PKU- IDGMMGGvvemm Institute for Brain Research, Peking University; Key Laboratory of Machine Perception (Min- istry of Education), Poking University, Beijing, China. Corresponding authors. Emails; dihe @ @ pku.edu.cn hang @ pku.edu.cn.

---

Published as a conference paper at ICLR 2026
2023): does it stem from genuine psychological reasoning, from sophisticated pattern matching on inputs with strong semantic overlap (Messeri & Crockett 2024)?
However, designing convincing experiment to address this question is not easy. We believe it should meet the following key criteria. First, the task should, as far as possible, rule out the possi- bility that LLMs exploit semantic pattern-matching strategies (Messeri & Crockett 2024 [Suh et al.) 2024) by using controlled, low-overlap inputs and requiring item-level predictions for individuals. Second, the data analysis should move beyond evaluating LLMs on isolated trait-to-trait predictions, and instead provide a comprehensive account of the correlation structure among psychological traits. Third, the test should be evaluated across different models (Samuel et al. 2024 Tuuajja et al. 2024), ensuring that the conclusion is widely applicable rather than model-specific artifacts. Finally, the test should enable a degree of process-level interpretability, moving beyond performance metrics to distinguish between genuine psychological reasoning and sophisticated pattern matching that could yield similar performance scores. Given that the faithfulness of Chain-of- of-Thought rationales re-
mains contested (Wei et al. 2022 Turpin et al. 2023), this necessitates complementary approaches to reveal the underlying cognitive processes driving LLM predictions.
To fulfill these criteria, here we designed new psychometric paradigm and data analysis pipeline. We tasked a diverse suite of LLMs with a challenging, zero-shot prediction task: given only the 20 item-level answers (on a 5-point response scale from strongly disagree to strongly agree) from an individual's Big Five personality inventory-a widely-used measure of five core personality dimensions- LLMS were required to predict the individual's answers each item of nine other, distinct psychological scales (questionnaires). To evaluate LLM predictions, we compared them against human participants' actual responses on these psychological scales using a dataset where each participant completed all measures. Different from prevalent methodologies that focus on first-order prediction accuracy assessing how well models directly replicate specific behavioral in- stances or trait scores (Jiang et al., [224] Park et al. 2024] Zhu et al. (002)) -w argue that simply correlating model predictions with actual scores would be methodologically flawed this context, as such correlations are constrained by the ground-truth relationships between the Big Five and tar- get scales. Instead, we developed a second-order morphism measure that compares the inter-scale correlation patterns in LLM predictions versus human data, using the regression coefficient between these patterns as our alignment metric. To further achieve interpretability, we analyzed the reasoning traces of LLMs with reasoning capabilities using reta-ppoompt approach, where annotation mod- els parsed the reasoning outputs to identify which specific Big Five items influenced each prediction and the underlying iiformation-procesette
stages.
Our experiments on LLMs yield two interconnected findings characterizing both LLM performance and the underlying process. The inter-scale correlations elicited from L.M predictions showed strong linear alignment with human data (1t2 > 0.88). This zero-shot performance substantially ex- ceeded predictions based on semantic similarity and approached the accuracy of machine learning algorithms trained directly on the dataset, indicating that LLMs can accurately reconstruct the nomo- thetic network of how individuals' different psychological traits covary. Our analysis of the models' reasoning traces reveals this capability is driven by two-stage process: First, LLMs transform raw Big Five responses into natural language personality summaries through information selection and compression; second, they generate target scale responses by reasoning from these summaries. For information selection, LLMs identify the same key personality factors as trained algorithms, though they fail to differentiate item importance within factors. The resulting compressed summaries prove sufficient for prediction, achieving similar alignment performance when used as input in place of the original Big Five responses, and even superior alignment when used as addition.
Our work demonstrates that LLMs possess genuine psychological reasoning capabilities rather than mere semantic pattern matching, as evidenced by their ability to accurately reconstruct the correla- tional structure of psychological traits from minimal personality data through systematic abstraction processes. By developing novel evaluation methods that bypass semantic overlap and revealing the mechanistic underpinnings of LLM psychological reasoning, we provide both a powerful tool for psychological simulation and insights into the interpretability of emergent AI reasoning capabilities.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
LLMs Human Simulators Many studies on LLMs involve personality tests, but for different purposes. Some measure psychological traits exhibited by the models themselves (Pellert et al,
2024 Sorokovikova et al., 2024 Tjuatja et al.] 2024 Dong et al., (025), while others use LLMs to role-play human participants, either to demonstrate their capability to replicate aggregate human
behaviors (Aher et al. 2023 Argyle et al., [023] Dillion al., 2023 Horton 2023 [Santurkar al.
2023) or to generate individualized agents with specified personality profiles (Choi & Lij 2024 2024). Like our work, some of these studies use Big Five per-
Jiang al. 2024, Petrov et al.,
sonality traits as input (Vu et al.,
2024: Li ct al. 2025 Wang et al. (025b). Notably, Park et al.'s
"Generative Agent Simulations of 1,000 People" (Park al. (024) demonstrates the possibility of
simulating individual- agentic behavior that aligns with humans across multiple dimensions of psychological traits and behavioral attitudes. Our work extends this line of work in three key aspects. First, whereas prior works primarily evaluate first-order prediction accuracy (i.e., how accurately the model mimics a specific trait), we perform a second-order analysis investigating whether LLMs can accurately reconstruct the entire psychological correlational network between traits. Second, our use of sparse inputs isolates the model's inferential capability from the memory retrieval common in data-rich, open-world settings (Park al. 2024). Finally, we analyze reasoning traces reveal the cognitive processes driving this capability, rather than merely evaluating outcomes.
Deconstructing Reasoning Mechanisms in LLMs The advent of Chain-of- (CoT) prompting (Kojima et al.] 2022 Wang et al.] [022] Wei et al., [2022), particularly in reasoning- enhanced models (Bi et al.] 2024). suggests that models could externalize their reasoning steps. However, key debate surrounds the faithfulness of these rationales: whether they truly guide the model's 's reasoning or are merely plausible post-hoc justifications (Lanham et al. 2023] Turpin al., 2023 Paul et al. (Acusschn et al. (2025] Our methodology offers insights into this debate. By analyzing LLMMs self-generated reasoning trace, find that LLMS spontaneously generate in- termediate representation based on high-level personality factors before predicting specific item re- sponses. This suggests that LLMs' reasoning is not passive retrieval process, in Self-RAG (Asai iet al. 2024), but an active process of information compression. LLMs synthesize verbose item data into dense, low-dimensional summary, representation our analysis confirms is instrumental in shaping the final prediction. The functional utility of this summary is further supported by provid- ing back to the model as additional input, which enhances prediction performance.
Evidence for a Cognitive Hierarchy in LLMs A central question in AI research concerns the quality and depth of cogmition in LLMs, with debates often polarized between claims of genuine understanding (Ichien et al., 2024) and the "stochastic parrot" hypothesis (Bender et al.. 2021). A more productive framework, inspired by cognitive science, is to investigate the existence of a cog-
nitive hierarchy (Landauer & Dumais [197]] Huber & Niklaus 2025 Wang et al., 2025a), which posits that cognition comprises tiered processes of increasing sophistication, from simple associ- ation to abstraction and rule-based generalization (Eppe et al. 2022 Graham & Granger 2025). Addressing the methodological gap how to dissect these strata, our work introduces novel psy- chometric paradigm to move the conversation from whether LLMss reason to how and at what level of abstraction they operate. Our findings chart a path up this hierarchy, demonstrating that LLMs' cog- nitive process (1) transcends surface-level statistical association, (2) prioritizes abstract conceptual structure over specific item-level details, and (3) performs a novel form of theoretical idealization, actively refining noisy inputs into theory-comsistent representations.
3 EXPERIMENT 1: LLMS' RECONSTRUCTION OF HUMAN PSYCHOLOGICAL. STRUCTURES
The aim of Experiment I is to test whether LLMs can reconstruct the entire nomothetic network from sparse, quantitative personality inputs such as Big Five personality scores, distinguishing be- tween genuine reasoning and pattern matching. A dataset comprising the psychological test results of 816 Chinese participants, collected online during the COVID-19 pandemic as part of longitu- dinal diary study on epidemic perceptions (Lu et al. (003)) served as the ground truth. The dataset consists of each participant's responses to the Big Five personality scale (Topolewska et al. 2014).
3

---

Published as a conference paper at ICLR 2026
which provided the input for the LLMs. alongside scores from nine other psychological scales: the Perceived Stress Scale (Cohen et al.) 1983; Leung et al. 2010), the Simplified Coping Style Questionnaire (Xiel 1998). the State-Trait Anxiety Inventory (Spielberger et al. 1971). the Self- Compassion Scale (Neff 2003), the Psychological Resilience Scale (CD-RISC) (Connor & David- son 2003), the Intolerance of Uncertainty Scale (Buhr & Dugas 2002), the Emotion Regulation Questionnaire (Gross & John 2003), the Risk Perception & Behavior Questionnaire, and the Future Time Perspective Scale (Carstensen & Lang 1996). Collectively, these instruments assess wide spectrum of constructs related to emotional well-being, adaptive functioning, and cognitive styles.
We evaluated a comprehensive suite of state-of-the-ae LLMs, including their "Chat" and reasoning- enhanced "Thinking" variants where applicable: DeepSeek's ' DeepSeck-Vi V3.1 (Liu et al., (024), OpenAT's GPT-5 (OpenAl| 2025), Anthropic's 8 Claude 3.7 Sonnet (Anthropic 2025), Google's Gem-
ini 2.5 Flash (Comanici et al. 2025), Zhipu Al's GL-5-.55 (Zeng et al. 22025). Moonshot Al's Kimi K2 (Team et al.J 2025), and Alibaba's Qwen3-235B (Yang et al.) 2025). We benchmarked their performance against traditional machine learning models (c.g., K-Nearest Neighbors, SVM, Linear
Regression) and a Semantic Similarity model based on BAAI's bge-reranker-larg mode [BAA
2024
cross-encoder for relevance scoring (details in Appendix B.4).
3.1 TESTING PROCEDURES FOR LLMS
The core procedure of our experiment, as illustrated in Figure 1, consists of two phases: (1) Predic- tion Generation (Per-Individual Task). For each of the 816 individuals in our dataset, we tasked the LLM with role-playing prediction. Each task involved providing the model with the individ- ual's s 20 item-level scores from the Big Five inventory. This served the sole information source for the model to predict that same person's 's responses on all items across the nine other psycho- logical scales. (2) Structural Comparison Analysis (Dataset-Levvl Analysis). After the model generated the complete dataset of predictions, we performed the following structural analysis. We computed the Pearson correlation matrix for every pair of psychological scale sub-factors (i.c., the individual dimensions that make up broader psychological measures) within the data. This matrix was then compared against a benchmark correlation matrix, which was calculated using the same method on the human ground-truth data, to evaluate the overall structural fidelity of the model's psychological inferences.
Phase Shot Role Playing Prediction
Individual its Profile
Prompt
Big Five Scures (20 Nlet lolall
SYSTEM
"inagie person with these trali: Itig Flus Proflie Inserned here]"
Repeat for all N-816 individuals
Aff the ((ivouree). huve frequent mood swingss 3 flgrrel...
LIM-Gnnerated Predictions (lur Individual))
"Nooi plocuc how you would with
USER:
Target Scale Quasstons
tnings aftos
tollowing
LLM
kusp feeling yoursuff
Questions
hare)""
Phase 2: Dataset Structural Comparison
LIM Correiation
LLM Correlation Mabrix
Human Correlation Matrix
K
Big
Big Five
Five
Model Prediction
Ground Truth
Human Conrelation
Finding: Linear Amplification
Figure 1: Experiment I's procedure. Phase (Per-Individual): The LLM role-played each partici- pant based on the individual's Big Five scores to predict responses on nine other target scales. Phase 2 (Dataset Level): We computed correlation matrices between all psychological scale components for both LLM predictions and human ground truth data, and then compared the resulting correlation matrices, which reveals a structural amplification in LLMs reconstructed psychological structures.
'The model was accessed via Alibaba Cloud's OpenSearch platform, where it is identified as "ops-bge- largnr"r-largerr. We refer it by its original BAAI name for clarity.
4

---

Published as a conference paper at ICLR 2026
Our primary analysis focuses comparing these correlation matrices to assess whether the model captures the underlying structure of human psychological trait relationships, rather than simply eval- uating individual prediction accuracy. As noted in the Introduction, this approach provides a more robust test of whether the model has internalized the human psychological network, independent of the actual strength of relationships in the ground truth data.
3.2 LLMS RECONSTRUCT A LINEARLY AMPLIFIED PSYCHOLOGICAL STRUCTURE
This structural comparison reveals that LLMs not only accurately replicate the correlational structure of human psychological traits, but also reconstruct an idealized, amplified version of it. First, when the correlation matrix from the model-generated scores compared against that of human data, the model-generated correlations (Gemini 2.5) consistently match the sign of human data but are more saturated (further from zero) than their human counterparts (see Figure top-left panel).
Second, to quantify this observation, we regressed the LLM's inter-scale correlations against the hu- man data. As shown in Figure 2 (top-right), Gemini 2.5 demonstrates an exceptionally strong linear relationship ( R 0.92, p .001) with regression slope (k) of 1.42, significantly greater than 1.0. This indicates that LLMs systematically overestimate the strength of correlations between psycho- logical traits-a phenomenon we term structural amplification. This should be distinguished from "bias amplification" ( (Fernández et al. 2023 Taori & Hashimoto 2023 Wang et al. (024), which typically describes a first-order effect where models exaggerate specific, pre-existing societal biases from training data (e.g.. gender stereotypes). In contrast, the structural amplification we identify is a second-order effect concerning the entire relational network of traits, rather than the strengthening of a specific biased association. To confirm the model constructs a coherent internal psychological network rather than just input-output mapping. we analyzed the correlational structure among pre- dicted target scales themselves (excluding Big Five inputs). The amplification effect persists with remarkable strength (k: = 1.41, R²2 = 0.91, p < ,001), confirming the model builds a coherent and internally amplified representation of the entire psychological structure.
Third, we verified this a general property of LLMs. All tested LLMs (Figure 2 bottom-left) exhibit an amplification coefficient & > 1.0 (see Appendix B.2 for their scatter plots). Crucially, all LLMs amplification coefficients surpassed that of baseline models like a k-Nearest Neighbors (KNN) model and, notably, the Semantic Similarity model. This suggests the models' performance stems from a process more sophisticated than simple retrieval or surface-level semantics. To in- vestigate the functional implications of this phenomenon, we correlated each model's amplification coefficient (k) with its predictive performance, a metric derived from the mean Pearson correla- tion between its predictions and the ground-truth scores across factors. The scatter plot in the bottom-right panel reveals a near-perfect positive linear relationship between these two variables (R2 0.95, p < .001). The greater a model amplifies the underlying structure, the better predicts human traits, suggesting this amplification is not an artifact but a core functional mechanism.
3.3 VALIDATING STRUCTURAL AMPLIFICATION: STATISTICAL SIGNIFICANCE AND ROBUSTNESS
We performed two further analyses to validate the observed linear structural amplification is genuine and robust. First, we established statistical significance using a 1,000-trial permutation test. By shuffling the model-generated correlation vectors, we created an empirical null distribution for our key statistics (/2² and Kendall's 7). The originally observed statisties were extreme outliers relative to this null distribution (p .001), unlikely to arise from pure chance.
Second, ensure the structural amplification arises from genuine psychological reasoning and not merely an artifact of the prompt's 's specific phrasing or structure, we examined the model's 's sensitivity (1) our original "Standard Order" setup, where the Big Five items were presented in a fixed sequence; (2) a "Random Order" condition, where the sequence of the 20 Big Five items was shuffled for each trial to disrupt potential order effects and (3) "Single Question" condition, where each target item was predicted individually to rule out artifacts from batch match-prompting. The amplification coefficient for Gemini 2.5 remained exceptionally stable across these conditions (k 1.42, 1.41, and 1.42, respectively). demonstrating that it is a robust feature of the model's 's reasoning process. Detailed results and figures for these validation tests are provided in Appendix C.1
to task framing and input organization (Oren et al. 2023). We tested three conditions:
5

---

Published as a conference paper at ICLR 2026
Human vs. Geminiz 5 Ocean Correlation Nathi Heatmap
Humer on Carren Ocien Cordittio
Colyong soon
X.M
KIN
506
430
AM
0.000.02
1/2
8:11
6:31
KMM
We prafie
8:1
198
K19
4/1
009
Coer arr
Set Homon
4.73
Linear Amplificaion Effect Across All Modele
Cooffident Mean Committion Across Moder
Modil Catsgerie kenorrc sinaero Hoee
Gologeing
1.54
R 25
Medils
1.42
1:45
1.0
TIME
1.53
1.54
Large Lang anggagee
Vacine Lein hi-s4s
p 00001
1.3
1.0
1.00
60
OLM Che
Unoe Rogeose
Da Todaai
Deepdook The
cartee
Owan
GPF
Cenia
Figure I.LMs' structural amplification of the correlational structure of human psychological traits. Top-left panel: heatmap comparing the correlations from human data with those predicted by Gemini 2.5 (paired rows), separately for each of the Big Five personality factors (rows) and the tar- get psychological scales their sub-scales (columns). Top-right panel: Gemini 2.5's correlations against human data correlations, revealing a strong linear relationship (R² = 0.92) with an ampli- fication slope (k: 1.42). Each dot denotes pair of correlations in top-left panel (color codes different target scales, see Appendix D for complete list). Bottom-left panel: All tested LLMs ex- hibit amplification coefficient & > T.O, consistently outperforming retrieval (KNN) and semantic similarity models. Bottom-right panel: a near-perfect linear relationship (R² 0.95) between model's 's amplification coefficient (k) and its predictive performance. Each dot denotes one model.
3.4 EXPLAINING STRUCTURAL AMPLIPICATION THE IDEALIZATION HYPOTHESIS
To understand the source of the structural amplification, we tested the hypothesis that it stems from LLMs acting "idealized participants" by filtering the random measurement error inherent in hu- man self-reports. According to classical test theory, such random noise attenuates the observed correlations between psychological scales (Spearman) 1961 Nunnally 1975). Our investigation into this hypothesis proceeds in two stages: first laying the thcoretical foundation with reliability analysis, and then providing empirical validation through convergent experiments.
Our analysis begins with the theoretical support for the idealization hypothesis, grounded in reliabil- ity metrics. As detailed in Appendix C.4.I we first compared the internal consistency (Cronbach's data. The LLM data exhibited significantly higher reli-
Alpha) of both human- and A.MM-generade ability (mean aLLM 0.87) than the human data (OHumma 0.75), suggesting that LLMs produce less noisy responses. Furthermore, we found that L.MM reliability profiles showed strong inter-model convergence (mean inter-I.I. MSE = 0.0060 vs. mean L.t---nuumnnn MSE = 0.0357). This sug- gests that different LLMs converge common, idcalized response model that differs from human patterns, positioning structural amplification as direct consequence of higher data fidelity.
We then sought empirical validation for this noise-filterng hypothesis through two convergent perimental analyses. From the human side, we isolated a more "attentive" subgroup (N = 309) by filtering out participants with fast response times. This less- human data produced an in- herently stronger correlation structure that was significantly closer to the LLM's output (k = 1.08 when compared to the full sample). Conversely, from the model side, established near-causal

---

Published as a conference paper at ICLR 2026
link through intervention. By systematically injecting increasing levels of Gaussian noise into baseline model's predictions, we observed a clear dose-response relationship: as noise increased, the structural amplification effect was progressively attenuated (k decreased from 1.55 to 1.12). This convergence of evidence-wie where removing noise from human data and adding it to model pre- dictions produce opposite, predictable effects- provides robust empirical support for interpreting structural amplification as process of idcalized abstraction (see Appendix CC..2 for details).
4 EXPERIMENT 2: DECONSTRUCTING THE REASONING MECHANISM
Experiment revealed that LLMs exhibit systematic "structural amplification" when simulating personality networks. However, this finding describes a macroscopic outcome, leaving the internal process black box. Experiment 2 is designed to open this black box. By analyzing the models' reasoning traces, we aim to uncover the cognitive processes driving this phenomenon, thereby en- hancing the explainability of their behavior. Specifically, a quantitative, testable framework that addresses two key questions: (1) How do models perform information selection from sparse inputs, and does this process align with psychological-concepe
move from qualitative observation to
structures? (2) Is the natural language summary generated during reasoning merely byproduct, or a predictively potent form of information compression?
4.1
INFORMATION SELECTION: A CONCEPT-DRIVER STRATEGY
Our analyses reveals that LLMs what may call "concept-driven" reasoning strategy, prior- itizing forming high-level conceptual understandings (e.g., personality factors) from the raw data to guide their predictions, rather than operating on isolated input items directly.
Methodology To parse the complex reasoning traces collected from Experiment 1, we used the methodology illustrated in Figure 3 For each specific reasoning trace accompanying a prediction, the I.LM that generated is referred to as "Reasoning Model" (e.g.. Soaude3.--Thinking GL.M4.5-
Thinking). To deconstruct the Reasoning Model's decision-making process while controlling for the interpretive biases of any single annotator, we parsed each trace using a separate, diverse suite of "Annotation Models", which included SeepScek-V3.1. Qwen3-235B, GLM-4.5, and Claude 3.7 Sonnet. Each Annotation Model generated 20 20-dimensional attribution distribution that quantified the perceived importance of each Big Five input item. These individual distributions were then av- eraged to produce a single, robust, and de-biased attribution vector for each reasoning trace. Finally, we benchmarked these averaged distributions against the feature importance weights derived from an outperforming Bayesian Ridge Regression model trained on ground truth.
Role: Performe the task and
Input Scarres (tees))
Reasoning
Yhi person "can empasiiec with others" ensotiiss and undertten ethoo lenor "zomewhst pittes
persos aggiffos ercod swings and gnoersly
Lnstructure Brasoning Tracs:
Task: Parse trace for item-llvel attr butions
Annotation Model (o- DeepSeekV3. 1-thon]
Directly extract holistic summary from trace by Summary Extraction Model (e.g. SeepdeskVS.1-Tinnnin
cill
Siructuned Aitritution Vicctor (20 dim)
Netural langunge Summary
Ready Quentitative Analysia
Roady for Efficacy Test
Figure 3: Flowchart of the "Reasoning- to- "oo-Annttttion" analyses in Experiment 2. Each reasoning trace comes from Experiment 1, where "Reasoning Models" use the 20 input scores to generate predictions along with reasoning traces. These traces are processed in two parallel analyses: an "Annotation Model" parses each trace to create structured attribution vector (addressing our first research question), while the summary within that same trace is used to predict the outcome, testing its predictive potency (addressing our second research question).
7

---

Published as a conference paper at ICLR 2026
Convergent Attribution Strategy Our analysis reveals remarkable cross-model consensus in attribution strategies. We performed a pairwise analysis across all combinations of Reasoning and Annotation models, and the resulting attribution distributions showed remarkable wooss-moeel con- sensus. As detailed in the Appendix C.2.1 heatmaps of cross-model Pearson correlation coefficients and Kullback-Leible (KL) divergences confirm this alignment. With seff-comparisons excluded, the average Pearson correlation between the attribution vectors of different Reasoning Models was 0.934, and the average KL divergence was exceptionally low at 0.0475. This strong consensus indicates that different reasoning, models independently reach the same strategy for mapping psy- chological traits, and this finding not an artifact of the annotation process.
Factor-Level Accuracy Despite Item-L.evel Confusion To dissect this convergent strategy, we benchmarked the models' averaged attribution profiles against two distinct baselines at two levels of granularity (Figure 4 These baselines were derived by fitting a Bayesian Ridge model to two different targets: the Human Ground Truth baseline was fitted to actual human responses, while the Semantic Similarity baseline was fitted to predictions from our semantic similarity model (detailed in Appendix [B.4) This three-way way comparison allows us to test whether LLM reasoning aligns more with human psychological patterns or with semantic associations.
At the fine-grained item level, the analysis reveals diffuse and inconsistent picture. The average Pearson correlation between the LLM attributions and the Human Ground Truth baseline is poor (r = 0.207). Similarly, the correlations between LLM attributions and the Semantic Similarity baseline (r = 0.087), and between the Human and Semantic baselines themselves (r = 0.201), are also weak. This indicates that at the level of individual items, no clear, consistent attribution strategy emerges across any of the models or baselines.
The distinction becomes stark at the coarse- grained factor level. Here, the LLM attribution profiles align almost perfectly with the Human Ground Truth bascline, achieving an average Pearson corre- lation of r 0.981. In sharp contrast, the correlation between L.M attributions and the Semantic Similarity baseline is substantially lower (r 0.790). value nearly identical to the correlation between the Human and Semantic baselines (r 0.787).
This clear dissociation uncovers key insight into the models' reasoning: LLMs robustly identify the correct high-level personality factor (e.g., Neuroticism) in a way that mirrors human-like con- ceptual importance, significantly diverging from strategy based on mere semantic similarity. While they struggle to differentiate the importance of specific items within that factor, their high-level ab- straction process supports the hypothesis of a top-down, concept-driven inference, rather than one guided by surface-level word associations.
DeepSeek Item Mapzing Ditritution (bng Across Ancortt Modelsi
Bemm Mapping Mean Distrbbttoon Models
DeepSeok Factor Napping Distribution (Avg Across Annotation Modalal
facto Mapping Mean Datt bution of Al Models
Ter Vanallyy
Figure 4: A concept-driven information selection strategy. The plots compare the models' averaged attributions (solid lines) against a Human Ground Truth (dashed blue) and a Semantic Similarity (dashed purple) baseline. The top row shows item-level attributions, where each bar represents subscale, revealing weak and inconsistent alignment. The bottom row shows factor-level attribu- tions, revealing that LLM attributions align almost perfectly with the Human Ground Truth, but starkly diverge from the Semantic Similarity baseline. This evidence supports that models identify high-level factors (e.g., Neuroticism) based on human-like conceptual understanding, rather than on surface-leve semantics. Each colored bar denotes one subscale, with color codes following that of the scatter plot in the top-right panel of Figure 2
8

---

Published as a conference paper at ICLR 2026
4.2 INFORMATION COMPRESSION: THE POWER OF ABSTRACT SUMMARIES
Methodology Beyond selecting items, reasoning-enabled models also generate holistic, natural- language summaries of the personality profile. To quantify the predictive power of these abstract summaries (e.g.. concluding that the provided scores reflect the risk perception of "an individual who is sensitive, prone to worry, imaginative, and relatively pessimistic, often overestimating the probability of negative rvents"), we designed an experiment with three information conditions:
ScoreOnly: The baseline condition from Experiment 1, where the model receives only the 20 Big Five numerical scores.
2. SummaryOnly: The model receives only the generated natural-language summary.
3. Suomary+Score: The model receives both the Big Five scores and the summary, and is instructed to use all available information.
Summaries as Potent and Sufficient Information Vehicle This analysis was aimed to under- stand the role of the natural-language summary-the brief, narrative description the model synthe- sizes to guide its final predictions. The central finding, as seen in Figure is the remarkable efficacy of this summary alone. When comparing the SummaryOnly condition to the ScoreOnly condition, we found that the structural amplification effect remained remarkably robust. This demonstrates that summary is not just a useful aid, but a highly potent and sufficient compression
the natural-languae of the original 20-item numerical input. The model is able to reconstruct the vast majority of the personality structure from this compressed linguistic representation alone.
Furthermore, we observed an unexpected synergistic effect. The Summary+Score condition consis- tently yielded the highest amplification multiplier for every model (right panel of Figure The fact that performance improves when adding a summary derived from the scores themselves indicates that the summary is not merely a redundant compression. Instead, it appears to contain emergent, during the model's 's reasoning process. Crucially, this enhancement in the amplification coefficient (k) is functionally significant, as it corre- sponds with increase in predictive performance. For all models, the mean predictive performance consistently followed the order: Summary+ > ScoreOnly > SummaryOnly. An extended anal- ysis across all 15 conditions models x 3 information types) revealed strong positive correlation between k and predictive performance (n²² = 0.93, < .001), confirming that greater structural amplification systematically predicts better outcomes (see Appendix C.3 for details).
second-order information conceptual cestalt-syntheett
C(SummaryOriy. Comrlation
Amplfficaion Ac Wodels Information
infommction Type
140
Summarp--oore
0.0
Uniu Mettllutttnn Ma Roloo
1.0
1.57
1.54
3.19
US44
1.5
223
1.44
1.44
3.26
3.39
2.2
3.50
1.00
1.0
Geniin
DeepSoek
Harran Oceen Oorelation
Mudel Rome
Figure 5: Analysis of the efficacy and synergy of L.LM-Meneraae summaries. The left panel shows the amplification effect persists even when using only the abstract summary (R² 0.91). The right panel compares the amplification multiplier for all models across the three information conditions, demonstrating the synergistic value of adding the summary.
5 DISCUSSION
We investigated the capacity of Large Language Models to reason about psychological trait correla- tions from sparse, quantitative data. Our finding of good alignment between L.M predictions and

---

Published as a conference paper at ICLR 2026
human psychological structure seemingly contrasts with Zhu et al. 2025 who found poor align- ment when inferring personality from qualitative interviews. However, Zhu et al. assessed first-order prediction of specific traits, whose performance depends ground truth correlation strength. We instead perform second-order structural analysis, comparing inter-scale correlation patterns between LLM predictions and human data. This approach reveals how L.MS preserve the entire correlational structure of psychological traits- capabiiapa that first-order prediction accuracy alone would miss.
Why do LLMs systematically "purify" the correlational structure of psychological traits? We con- jecture that this stems from the model acting as an idealized participant, capable of bypassing the statistical noise inherent in human self-reports. Human responses are known to be contaminated by at least two distinct sources of noise: systematic biases from idiosyncratic response styles (Grim-) mond et al. 2025), and random measurement error arising from factors like inattention, fluctuating emotional states, or momentary misinterpretation of items (Nunnally, 1975). As detailed in Sec- tion 3.4. our investigation supports this moise-filtering hypothesis with convergent evidence. The higher internal consistency of LLM-generated data provides theoretical support, further substan- tiated by two empirical analyses: filtering noise from human data and injecting noise into model predictions (see Appendix [C4.2] for details). This evidence supports interpreting structural amplifi- cation as idealized abstraction driven by the model's ability to filter random statistical noise.
Having established that LLMs perform this abstraction, our analysis of reasoning traces reveals a two-stage process. First, during information selection, models prioritize high-level psychologi- cal factors over specific item details, strategy findings distinguish from reliance surface- level semantics. This prioritization of abstract concepts over concrete data is consistent with both a cognitive Concept-Driven Strategy (Jackson) 2015) and the computational Information Bottleneck principle (Tishby et al. 2000), as the model generates a compressed representation by discarding less critical, item-level information. Meanwhile, information compression constructs predictively potent natural-language summary that contains emergent, second- orderr information. Together, this integrated process of concept selection and information compression provides a clear account of how LLMSs move beyond mere data replication to operate at higher, more abstract level of reasoning.
Our work has several limitations that should be addressed in future research. First, the human dataset that our findings are based is from a single cultural context that may lack broader demographic representativeness. Future work may apply our framework to diverse datasets to explore how cul- tural biases in training data might affect the idealized structure (Jakesch et al. 2023). Second, while we establish structural amplification as a general property of modern LLMs, we do not deconstruct the variance between them. Future work could investigate why certain models architectures hibit a stronger amplification effect, potentially linking this capacity to factors like model scale or fine-tuning methods. Third, the rich semantics of the synergistic summaries warrant further inves- tigation. Finally, as our analyses are primarily observational, future studies should move towards interventional approaches,such actively modulating the amplification effect establish definitive causality.
6 CONCLUSION
This paper investigates the capacity of Large Language Models to reason about human individu- als' psychological traits from sparse data. We identify robust and counter-intuitive phenomenon we term structural amplification, where LLMs do not merely replicate but systematically idcalize the correlational structure of human personality. We provide empirical evidence that this effect represents idealized abstraction, driven by the model's ability to filter statistical noise inherent in human self-reports. Our mechanistic analysis reveals that this abstraction involves concept-driven information selection and synergistic information compression that synthesizes predictively potent linguistic summaries. This work provides a mechanistic account of how LLMs transcend passive data replication and engage in active, abstract model construction.
ETHICS STATEMENT
This research conducted with strict adherence ethical principles, prioritizing the privacy of human participants. Our study used a fully anonymized dataset from a previous study, which had been approved by the Institutional Review Boards of School of Psychological and Cognitive Sci-
10

---

Published as a conference paper at ICLR 2026
ences Peking University and Faculty of Psychology Beijing Normal University. In that original data collection, all participants provided informed consent online. No personally identifiable infor- mation was accessed used at any stage of our research, ensuring the confidentiality of all 816 participants.
REPRODUCIBILITY STATEMENT
Core experimental details are provided in the Appendix. Specifically:
Data and Materials: To facilitate reproducibility, the complete dataset, including anonymized participant responses, LLM-generated outputs, and the psychological scales used in our experiments, available at our project repository
Prompts:
All prompt templates used for the main tasks (Experiment & 2) and robustness checks are detailed in Appendix A and Appendix C.I
Models: A comprehensive list of all Large Language Models used in our experiments, including their specific model identifiers, is provided in Appendix[B.]
Baselines: The implementation details for the semantic similarity baseline are described in Appendix B.4
ACKNOWLEDGMENNTS
HZ was partly supported by National Natural Science Foundation of China (32471152), Clini- cal Medicine Plus X-Yuung Scholars Project of Peking University, the Fundamental Research Funds for the Central Universities (PKU202M_CXQ02)
and PKU2024I.CXQ046) and funding from Pcking-Tsinghus Center for Life Sciences. DH was supported by National Science Foun- dation of China (NSFC62376007), National Science Foundation of China (under Key Project No. 92570203), Beijing Natural Science Foundation (Z250001) and Beijing Major Science and Tech- nology Project under Contract no. Z25 125100000842004.
REFERENCES
Gati V Aher, Rosa Arriaga, and Adam Tauman Kalai. Using large language models simulate multiple humans and replicate human subject studies. In International conference on machine learning, pp. 337-371. PMLR, 2023.
Anthropic.
Claude 3.7 sonnet system card. Technical report, Anthropic, June 2025. com/ 2651///B22216666033bbbboo/r pdf
URL
hetps: http:///sssttt..tttaass. -laude-3- --se--------noee---tttteeeeee
Iván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, and Arthur Commy. Cfain-of-thought reasoning in the wild not always faithful. arXiv preprint DrXiv:2503.08679, 2025.
Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. Out of one, many: Using language models to simulate human samples. Political Anal- ysis, 31(3):337-351, 2023.
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. 2024.
BAAL BGE Reranker Model. httt:s///ogemmmaaa..cc//oooott// htwpa//////eeaaainggniiggn 2. 2.html 2024.
Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big?. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp. 610-623, 2021.

---

Published as a conference paper at ICLR 2026
Xiao Bi. Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint rrXh:2241.002954 2024.
Kristine Buhr and Michael Dugas. The intolerance of uncertainty scale: Psychometric properties of the english version. Behaviour research and therapy, 40(8):931945,, 2002.
LL Carstensen and FR Lang. Future orientation scale. Unpublished manussiii Stanford University, 1996.
Hyeong Kyu Choi and Yixuan Li. Picle: Eliciting diverse behaviors from large language models with persona in-context learning. arXiv preprint arXiv:2405.02501, 2024.
Sheldon Cohen, Tom Kamarck, and Robin Mermelstein. A global measure of perceived stress. Journal of health and social behavior, pp. 385-396, 1983.
Gheorghe Comanici, Eric Bieber, Mike Schackermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arxiv preprint arXiv:2507.06261, 2025.
Kathryn M Connor and Jonathan RT Davidson.
Development of a new resilience scale: The connor- davidson resilience scale (cd-risc). Depression and anxiety, 18(2)76-82, 2003.
Lee Cronbach and Paul E Meehl. Construct validity in psychological tests. Psychological bulletin, 52(4):281, 1955.
Danica Dillion, Niket Tandon, Yuling Gu, and Kurt Gray. Can ai language models replace human participants? Trends in Cognitive Sciences, 27(7):997-600, 2023.
Wenhan Dong, Yuemeng Zhao, Zhen Sun, Yule Liu, Zifan Peng, Jingyi Zheng, Zongmin Zhang, Ziyi Zhang, Jun Wu, Ruiming Wang, et al. Humanizing Ilms: A survey of psychological measurements with tools, datasets, and human-agent applications. arXiv preprint arXiv:2505. 00049, 2025.
Manfred Eppe, Christian Gumbsch, Matthias Kerzel, Phuong DH Nguyen, Martin V Butz, and Stefan Wermter. Intelligent problem-solving as integrated hierarchical reinforcement learning. Nature Machine Intelligence,
2022.
Simón Peña Fernández, Koldobika Meso Ayerdi, Ainara Larrondo Ureta, and Javier Díaz Noci. Without journalists, there is no journalism: The social dimension of generative artificial intelli- gence in the media. Profesional de la información, 32(2):2, 2023.
Graham and R Granger. Three tiers of computation in transformers and in brain architectures. arXiv preprint arXiv:2503.04845 2025.
Jess Grimmond, Scott D Brown, and Guy E Hawkins. A solution to the pervasive problem of response bias in self-reports. Proceedings of the National Academy of Sciences, 122(3): e2412807122, 2025.
James J Gross and Oliver John. Individual differences in two emotion regulation processes: im- plications for affect, relationships, and well-being, Journal of personality and social psychology, 85(2))348, 2003.
John J Horton. Large language models as simulated economic agents:
What can we learn from homo silicus? Technical report, National Bureau of Economic Research, 2023.
Thomas Huber and Christina Niklaus. Lims meet bloom's 's taxonomy: A cognitive view on large lan- guage model evaluations. In Proceedings of the 31 International Conference Computational Linguistics, pp. 5211-5246, 2025.
Nicholas Ichien, Sudeep Bhatia, Anna Ivanova, Taylor Webb, Tom Griffiths, and Marcel Binz. Higher cognition in large language models. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 46, 2024.
12

---

Published as a conference paper at ICLR 2026
Daniel Jackson. Towards theory of conceptual design for software. In 2015 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (On- ward!), pp. 282-282222 296, 2015.
Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman. Co-writing with opinionated language models affects users' views. In Proceedings of the 2023 CHI conference on human factors in computing systems, pp. 1-15, 2023.
Hang Jiang, Xiajic Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. Personallm: Investigating the ability of large language models to express personality traits. In Findings of the association for computational linguistics: NAACL 2024. pp. 3605 6605-3627, 2024.
Povilas Karvelis, Martin P Paulus, and Andreea Diaconescu. Individual differences in computa- tional psychiatry: A review of current challenges. Neuroscience & Biobehavioral Reviews, 148: 105137, 2023.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-22211 2022.
Thomas K Landauer and Susan T Dumais. solution to plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological review, 104(2):211, 1997.
Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Her- nandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, al. Measuring faithfulness in chain-of-thought reasoning. arXiv preprini vrXiv:2307.13702 2023.
Doris YP Leung Tai-hing Lam, and Sophia SC Chan. Three versions of perceived stress scale: validation in a sample of chinese cardiac patients who smoke. BMC public health, 10(1):513, 2010.
Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona Diab, and Maarten Sap. Big5-chat: Shaping IIm personalities through training on human-grounded data. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 20434- 20471, 2025.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
Yi-Long Lu, Yang-Fan Lu, Zhuo Rachel Han, Shaozheng Qin, Xin Zhang, Li Yi, and Hang Zhang. Prediction from minimal experience: How people predict the duration of ongoing epidemic. Cognitive Science, 47(5))e13294, 2023.
Lisa Messeri and Molly J Crockett. Artificial intelligence and illusions of understanding in scientific research. Nature, 69788002): 2024.
Kristin Neff. The development and validation of scale to measure self-compassion. Self and identity, 2(3):223-250, 2003.
Jum C Nunnally. Psychometric theory -2 years ago and now. Educational Researcher, 4(10): 7-21, 1975.
OpenAl. Gpt-5 system card. Technical report, OpenAI, 2025. URL httphtttpp::/ch.opeaaa
Com/ 5-pt-5-system card. pd
Yonatan Oren, Nicole Meister, Niladri S Chatterji, Faisal Ladhak, and Tatsunori Hashimoto. Proving test set contamination in black-box language models. In The Twelfth International Conference on Learning Representations, 2023.
Xuchen Pan, Dawei Gao, Yuexiang Xie, Yushuo Chen, Zhewei Wei, Yaliang Li. Bolin Ding, Ji-Rong Wen, and Jingren Zhou. Very large-scale multi-agent simulation in agentscope. arXiv preprint arXiv:2407. 17789, 2024.
13

---

Published as a conference paper at ICLR 2026
Joon Sung Park, Carolyn Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S Bernstein. Generative agent simulations of 1,000 people. arXiv preprint arXiv:2411.10109 2024.
Debjit Paul, Robert West, Antoine Bosselut, and Boi Faltings. Making reasoning matter: Measur- ing and improving faithfulness of Mhain-of-thought reasoning, arXiv preprint arXiv:2402.139.. 2024.
Max Pellert, Clemens M Lechner, Claudia Wagner, Beatrice Rammstedt, and Markus Strohmaier. Ai psychometrics: Assessing the psychological profiles of large language models through psy- chometric inventories. Perspectives on Psychological Science, 19(5):808-88) 826, 2024.
Nikolay Petrov, Gregory Serapio-Garche and Jason Rentfrow. Limited ability of Ilms simulate human psychological behaviours: a psychometric analysis. arXiv preprint wrXiv:2405.07244 2024.
Vinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari, Ashwin Kalyan, Tanmay Rajpuro- hit, Ameet Deshpande, Karthik Narasimhan, and Vishvak Murahari. Personagym: Evaluating persona agents and llms. arXiv preprint arXiv:2407. 18416, 2024.
Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? In International Conference on Machine Learning. pp. 29971-30004. PMLR, 2023.
Gregory Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Marwa Ab- dulhai, Aleksandra Faust, and Maja Matarić. Personality traits in large language models. 2023.
Aleksandra Sorokovikova, Natalia Fedorova, Sharwin Rezagholi, and Ivan P Yamshchikov. Llms simulate big five personality traits: Further evidence. arXiv preprint arXiv:2402.01665, 2024.
Charles Spearman. The proof and measurement of association between things. 1961.
Charles D Spielberger, Fernando Konzalez-Reigose Angel Martinez-Urutia, Luiz FS Natalicio, and Diana S Natalicio. The state-trait trait anxiety inventory. Revista Interamericana de Psicolo- gia/Interamerican journal of psychology, 5(3 & 4), 1971.
Joseph Suh, Suhong Moon, Minwoo Kang, and David M Chan. Rediscovering the latent dimensions of personality with large language models as trait descriptors. arXiv preprint arXiv:2409.09905, 2024.
Rohan Taori and Tatsunori Hashimoto. Data feedback loops: Model-driven amplification of dataset biases. In International Conference on Machine Learning, pp. 33883-33920. PMLR, 2023,
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534 2025.
Naftali Tishby, Fernando Percira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004057, 2000.
Lindia Tjuatja, Valerie Chen, Tongshuang Wu, Ameet Talwalkwar, and Graham Neubig. Do Ilms exhibit human-like response biases? case study in survey design. Transactions of the Association for Computational Linguistics, 12:1011-1026 2024.
Ewa Topolewska, Ewa Skimina, Wlodzimierz Strus, Jan Cieciuch, and Tomasz Rowinski. The short ipip-bfm-20 questionnaire for measuring the big five. Roezniki Psychologiczne, 17(2):385-409 2014.
Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don't always say what they think: Unfaithful explanations in chain-of-than of-thouggtt prompting. Advances Neural Information Processing Systems, 744.74952279955 2023.
14

---

Published as a conference paper at ICLR 2026
Huy Vu, Huy Anh Nguyen, Adithya V Ganesan, Swanie Juhng, Oscar NE Kjell, Joao Sedoc, Mar- garet L Kern, Ryan L Boyd, Lyle Ungar, H Andrew Schwartz, et al. Psychadapter: Adapting Ilm transformers to reflect traits, personality and mental health. arXiv preprint arXiv:2412.16882, 2024.
Xinglin Wang, Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Boyuan Pan, Heda Wang, Yao Hu, and Kan Li. Coglm: Tracking cognitive development of large language models. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics:
Human Language Technologies (Volume I: Long Papers); pp. 73-87, 2025a.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11111 2022.
Yilei Wang, Jiabao Zhao, Deniz S Ones, Liang He, and Xin Xu. Evaluating the ability of large language models emulate personality. Scientific reports, 15(1):519, 2025b.
Ze Wang, Zekun Wu, Jeremy Zhang, Xin Guan, Navya Jain, Skylar Lu, Saloni Gupta, and Adriano Koshiyama. Bias amplification: Large language models as increasingly biased media. arXiv preprint arXiv:2410. 15234, 2024.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of-thhuugle prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 21124824-24837, 2022.
Yaning Xie. Reliability and validity of the simplified coping style questionnaire. Chinese Journal of Clinical Psychology, 1998.
Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.
Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, al. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models. arXiv preprint arXiv:2508.0647/, 2025.
Jianfeng Zhu, Julina Maharjan, Xinyu Li, Karin G Coifman, and Ruoming Jin. Evaluating Ilm align- ment on personality inference from real-world interview data. arXiv preprint arXiv:2509.132 2025.
Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. Can large language models transform computational social science? Computational Linguistics, 50(1): 237-291, 2024.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
A EXPERIMENTAL PROMPTS
This section provides the English translations of the core prompt templates used in our experiments. Placeholders like *[Personality Profile]' and '[Target Scale Items]' are used denote the parts of the prompt that were dynamically populated for each participant and task.
A.1 MAIN TASK PROMPTS (EXPERIMENT 1 & 2)
A.1.1 SCOREONLY CONDITION PROMPT
This is the standard prompt used to elicit predictions based only on the Big Five personality scores. The model is given a personality profile and asked role-play that individual to answer questions from a target psychological scale.
"role": "system",
"content" "Please imagine you are role-playing a specific person. + The follewing are some descriptions of your personality. Each
4 descriptien has one of five levels of appiicability: 'Strongly Disagree', 'Disagree', 'Neutral', 'Agree', 'Strongly Agree'.
[20 Big Five personality items are inserted here, for example:)
You are the life of the party: Disagree
You sympathize with others' feelings: Agree
You get chores done right away: Neutral
You have frequent mood swings: Agree
You have a vivid imagination: Agree
Please rate the extent to which the person you are role-playing
would agree with the follewing descriptions. If you Stronngyy
4 Disagree', output 1. If (Disagree', output 2. If 'Somewhat
4 Disagree', output 3. If 'Neither Agree nor Disagres', output
If Scmewhat Agree', output 5. If 'Agree', output 6. If SStrengly Agree', , output 7.
Please use the format DDscriptioo 1: (integer from 1 to 7) Descrippio 2: (integer from to 7) ...'' ' for your output."
"role": "user",
"content": "[Target scale items are inserted here, for example:] positive emotion (such
Description 1: when I want to feel
4 joy or amuscment), I change what I'm thinking about.
Description 2: keep my emoticns to myself.
Description 3: When I want te feel less negative emotion (such as sadness anger), I change what I'm thinking ng about.
A.1.2 SUMMARYADDED CONDITION PROMPT
This prompt is identical the ScoreOnly condition, but includes the model-generater summary as additional context. The model explicitly instructed to use all available information.
"role": "system", "content": "please imaging you are role-playing speci fic person.
16

---

Published as a conference paper at ICLR 2026
The following
descriptions of your personality...
[The same 20 Big Five personality items as in ScoreOnly]
The fellowing is supplementary summary generated by model, which you may refer to as you see fit:
[The LiM-geneeated abstract summary is inserted here, for example:] Key Points Summary:
This person is likely pessimistic and prone to worry, being
emoticnally volatile and easily disheartened... these estimationn reflect the risk perception of individual who is sensitive, prone to worry, imaginative, and relatively pessimistic, often overestimating the probability of negative events
Flease rate the extent to which the person you are role-playing 4 would agree with the follewing descriptions.... [The rest of the prompt is identical to the ScorcOnly condition
"role": "user", "content": "[Target scale items are inserted here!"
A.1.3 SUMMARYONIY CONDITION PROMPT
In this condition, the model receives only the abstract summary, without the original numerical scores.
"role": "system",
"content"; "please imagine you are role-playing a specific person.
+ The fellewing is a summary of your personality:
[The LLM-generated abstract summary is inserted here]
Please rate the extent to which the person you are role-playing
4 would agree with the follewing descriptions....
[The rest of the output instructions are identical to the ScoreOnly conditicn]"
"role": "user", "content"; "[Target scale items inserted here!"
A.2 ROBUSTNESS CHECK PROMPTS
A.2.1 RANDOM ORDER CONDITION PROMPT
The prompt for this condition was identical the ScoreOnly prompt, with the sole exception that the 20 Big Five personality items presented in the system message were randomly shuffled for each trial. The user message containing the target scale items remained unchanged.
A.2.2 SINGLE QUESTION CONDITION PROMPT
This prompt was modified to test for artifacts from asking multiple questions at once. The model was prompted for each target item individually.
17

---

Published as a conference paper at ICLR 2026
"role": "system", "content"; "Please imagine you are role-playing specific person....
[The Big Five perscnality items in ScoreOnly]
Flease rate the extent to which the person you are role-playing
+ would agree with the tollewing description... Output only single integer from to 7, with other text."
"role": "user",
"content": "Description: [A single target scale item is inserted here]"
]
A.3 REASONING MECHANISM ANALYSIS PROMPTS (EXPERIMENT
A.3.1 ATTRIBUTION MAPPING PROMPT (FOR INFORMATION SELECTION ANALYSIS)
This meta- peetapprompt was given to "Annotation Model" to parse the reasoning trace of a "Reasoning Model" and identify which input items were used for a prediction.
"role": "user",
"content"; "we have provided the Big Five personality scores of
participant to large language model and asked it to predict
+ the participant's s scores on the ERQ questionnaie items. Based
4 the model's reasoning content below, please identify which
Big Five items the model relied when giving its score for each ERQ description.
Strictly follow the format 'Description 1: Item 2, Item 3;
4 Description 2: Item 7, Item 15, Item 20...' Do not output any other content.
The following are the Big Five scores input to the model: 120 Big Five personality items with their scores]
The following
model's reasoning content: [The full reasoning trace from the "Thinking Model' is inserted ( here]"
]
A.3.2 SUMMARY EXTRACTION PROMPT (FOR INFORMATION COMPRESSION ANALYSIS)
This prompt was used to have a model read its own reasoning trace and extract only the parts that constitute holistic summary of the persona.
"role"; "user",
"content":
"We have provided the Big Five personality scores of + participant to model and asked it to predict scores the relevant questionnaire. The following is the model's reasoning content.
Please determine if the reasening content includes a summary 4 description of the person (pay attention to summary words like 'in summary', 'overall', 'synthesizing"). If it does, please
18

---

Published as a conference paper at ICLR 2026
output these summary descriptions exactly they appear,
without cmitting any, If there are multiple summaries, include
them all. Do not output any other content. If no summary is found, cutput 'None'.
Reasoning Content:
[The full reasoning trace from the 'Reasoning Model' is inserted
here]"
B MODELS AND PERFORMANCE
B.1 LIST OF LANGUAGE MODELS
For clarity and reproducibility, this section lists the large language models used. All models were accessed via API calls to the paaukooutrpplafoomn and their identifiers follow its specific conventions.
Our study utilized model conditions: standard "Chat" mode and "Thinking/Col" (Chain- of-TThought mode. Experiment benchmarked all listed models for predictive performance, with most models being evaluated under both conditions. Experiment then focused exclusively on analyzing the reasoning traces from the "Thinking/CoT"" modes. This advanced mode was enabled either by calling a dedicated model identifier (e.g., with :thinking suffix) or via an API parameter, specified in the table.
Table I: Large Language Models Evaluated in the Study.
Developer
Short Name
Condition/Notes
Model Identifier
DeepSeek Al
DeepSeek V3.1
Standard Chat Thinking/Co'
deepseekdeepsckkcchtt.... ddepsseekdeppseeccchtt-33
DeepSeek AI
DeepSeek V3.1
OpenAl
GPT-5
Standard Chat
openai/gpt-5-chat
Anthropic
Claude 3.7 Sonnet
Standard Chat Thinking/Co'T
muutropic/claude-.7-ssommmm
Anthropic
Claude 3.7 Sonnet
Google Google
Gemini 2.5 Flash
Standrd Chat Thinking/CoT
google/gemiin--2.5-lass
Gemini 2.5 Flash
google/gemini-2.5--22
Zhipu AI
GLM-4.5
Standard Chat Thinking/CoT
z-ai/glm-455
Zhipu AI
GLM-4.5
z-ai/glm-4.5
Moonshot AI
Kimi K2
Standard Chat
muonshotai/kimi-k2-999
Alibaba Alibaba
Qwen3-235B
Standard Chat
wwen/quen3-235b-a222
Qwen3-235B
Thinking/CoT
qwen/qwen3-2355-
B.2 INDIVIDUAL MODEL SCATTER PLOTS
To provide a comprehensive visualization of the structural amplification phenomenon across all evaluated large language models, this section presents scatter plots for each of the 12 large language models tested in our study. Each plot displays the linear relationship between the inter-scale corre- lations derived from model predictions and those from human ground-truth data, analogous to the Gemini 2.5 example shown in Figure 2 (top-right panel) of the main text.
The consistent pattern observed across all models reinforces our central finding: LLMs system- atically reconstruct an amplified version of the human psychological correlational structure, with regression slopes (k) consistently exceeding 1.0. This cross-mous cross-model consistency provides robust evi- dence that structural amplification is a general property of modern large language models rather than model-specific artifact.
19

---

Published as a conference paper at ICLR 2026
Human DeepSeek-Chat Ocean Correaation
Human DeepSoek-Thinking Ocean Correlation
k 1.38
k 1.41
R+= 0.91
= 0.91
1.00
p 1001
DespSeek: Thining Uoean Conelation
p.
0.70
0.75
0.50
0.50
0.20
0.25
0.00
0.00
DaspSesi
0.20
0.25
1.29
1.25
-1500
-1.53
1.75 1.50
1.25
1.0 9.75 0.50 0.25 0.00 0.29 0.50
0.75
1.00
1.20
1.00
1.75
-1.7
1,2 1.00 9:75 (350 0:25 0.00 6.25 0300 0.75 1.00
1,25
150 1.75
Human Ocean Correlation
Human Ocean Correaation
Human Claude Chat Ocean Corretation
Human Claude Claude-Thinking Ocean Correlation
1.50
K 1.58
1.50
k 1.56
1.5
0.90
1.25
R(= 0.90
.001
.000
Condation
Claude Thintin Ocean Corrlation
0.76
0.50
0.90
0.00
0.00
0.25
0.20
0.50
0.50
-1.00
-1.00
-1.25
-129
1.775
250 0.25 0.75
1.00 125 7.50 175
-1.7
1.75-1.90-1 355.:000......:8:0000
0.00 325 3.60 3.75
1.00
125
1.00 1.75
Haman Ocean Correlation
Human Ocean Correlation
Gernini-Thinking
1.50
1.50
1.29
0.92 p
1.42 k
= 0.89
1.48
p 001
Gammi thrking Ocean Corsation
0.75
0.75
0.50
0.50
0.25
0.25
0.00
0.00
GentniChett
6.25
0.25
0.50
0.69
-.25
75-7..5011225555 2125-100 40.-000000000--0 117-5-5---3--- 0.0 032 020 0.75 1.00 1225 120 1.75
-175 2.00 41775-5000000 0.00 6.6 030 0.75 1.0 122 175 -..0
Human Ocean Corretation
Human Ocean Corretation
Figure 6: Scatter plots demonstrating the structural amplification effect for the first six evaluated large language models. Each plot shows the linear relationship between model-predicted inter- scale correlations and human ground-truth correlations, with regression slope k 1.0 indicating systematic amplification of the psychological structure.
20

---

Published as a conference paper at ICLR 2026
Human GLM-Chat Ocean Correlation
Human GLM-Thinking Ocean Correlation
k 1.51
k 1.46
0091
R 0.91
1.00
P0..001
0.75
GLM Tapkinn Ocean Correatton
0.75
0.50
0.50
0.20
0.25
0.00
0.00
0.25
0.25
1.29
-125
-1.00
-1.53
-175
1,20
1.25
1.0 9.75 0.50 0.25 0.00 0.29 0.50 0.75
1,00
1.25
1.00
1.75
-1.7
-1.70-1.00
1.00-100-00
1,2 1.00 9:75
(350 0:25 0.00 6.25 0350 0.75
1.00
1,25
150
1.75
Human Ocean Correaation
Human Ocean Correaation
Human Owen-Chat Ocean Correlation
Human Owen-Thinking Ocean Correlation
1.50
1.54
k=
1.25
R== 0:91
0 0xx001
x .001
1.00
1.00
Owen: Thinting Ocean Correlatin
Owen-Chat Condatto
0.70
0.76
0.50
0.50
0.20
0.26
0,70
0.75
-1.00
-1.09
-1.20
.1.25
-1
-1.5
1.75.-1.50.120
1.200.... 0.76
0.60-0222 0.00 0.25 0.50 0.76
1.00
1,25
1.50 1.76
-1.7..1.00-111 1.000-000077 0.76 0.00 0.00 0.26 0.50 0.75
1.25 1.60 1.76
Human Comelation
Human Ocean Commiation
Human GPT-Chat Oosan Correlation
Human vs Kimi-Chat Ocean Correlation
1.50
1.50
k 1.63
1.25
R" 0.92
1.2
R²= 0.88
0000
1.00
p.
CIPT-Chul Dosan Condattin
0.75
Kima Chat Dosan Corrsation
0.50
0.00
0.00
0.25
4.25
0.50
0.50
0.75
.1.75
-1.00
.1.00
-.25
-.25
4.50
1.75
1.76
1.76
1.26
1.00 0.76 0.60 0,00000 0.26 0.60 0.76 1.00 1.25 1.50 1.75
Human Ocean Correlaton
Human Correlation
Figure 7: Scatter plots demonstrating the structural amplification effect for the remaining six evalu- ated large language models. Each plot shows the linear relationship between model-predicted inter- scale correlations and human ground-truth correlations, with regression slope k 1.0 indicating systematic amplification of the psychological structure. The consistent pattern across all 12 models provides robust evidence for structural amplification as general property of modern LLMs.
21

---

Published as a conference paper at ICLR 2026
B.3 DETAILED PREDICTIVE PERFORMANCE METRICS (HEATMAP)
To supplement the analysis of the structural amplification effect, this section provides detailed breakdown of each model's predictive performance. Figure & visualizes the Pearson correlation co- efficient (r) between each model's generated scores and the human ground-truth scores for every target psychological subscale. This heatmap offers a granular view of model capabilities, reveal- ing which models excel at predicting specific psychological constructs and providing the detailed evidence that supports the aggregate performance rankings discussed in the main text.
Comparison of Pearson Correlation Coefficients Across Models
4
0.6
1.90
0:10
0,22
3.29
0.2
0,20
0:11
022
0.1"
0.10
0.2
0.10
Tul, Jowely
100m
1:9m
125m
322
102m
103m
0.2
006
000
047
8:11
1.2
643
Set Conpasion
30
0.47
0.35
133"
139
043
1.2"
046
0.35
1.30
0.8
0.5-
0,67
1.47
290.4
0.13
3.99
0.01
0.11
Dng 035
8330
0357
3.22
0:37
0
8:07
038
0,25
1377
038
628T
0,907
03
0.55
0.11
0:341
1.99
Meam
D08
2:37
000
(000000)
Figure 8: Detailed heatmap of predictive performance across all models and target psychological scales. The metric shown is the Pearson correlation coefficient (r).
B.4 SEMANTIC SIMILARITY BASELINE
To test if structural amplification stems from surface-level semantics, we built a non-reasoning base- line. We used a re-ranking model, specifically the ops-bge leoge-rerable seerrnner-largee BGE-based cross-encoder model served through Alibaba Cloud's OpenSearch platform for docu- ment relevance scoring. This model generated semantic similarity scores that served as fixed weights in linear model to predict target scale scores from participant's 's Big Five input. Crucially, predic- tions for siverse-scored items were inverted to ensure directional correctness.
which is a
As shown in Figure this baseline yielded an amplification coefficient of only k = 0.99 with a weak fit (R² = 0.52). A coefficient near 1.0 indicates a failure to amplify. merely replicating the input data's structure. This finding provides strong evidence that the LLM's amplification phenomenon not a byproduct of simple semantic matching but stems from more sophisticated, abstract reasoning process.
Hurran Semontic Similarty Ocean Comrlation
2:90.9
p. 10.001
3.75
3.00
035
4.25
Hamann www.nnoaann.coo Comaaddr
Figure 9: Analysis of the Semantic Similarity baseline. The regression slope (A: 0.99) is nearly 1.0, indicating a failure to produce the structural amplification effect. The poor linear fit (R² 0.52) further confirms that LLM performance transcends surface-level semantics.
22

---

Published as a conference paper at ICLR 2026
C SUPPLEMENTARY ANALYSES FOR MAIN FINDINGS
C.I VALIDATING STRUCTURAL AMPLIFICATION: SIGNIFICANCE AND ROBUSTNESS
To ensure the observed structural amplification genuine and robust feature of the models' rea- soning, we conducted two further validation analyses.
Statistical Significance. We conducted a 1,000-trial permutation test, shuffling the LLM- generated correlation vectors to construct empirical null distribution for key statistics (R² and Kendall's T). As shown in Figure [T0] the originally observed statistics were extreme outliers, falling far outside the range of values expected under the null hypothesis (p .001). This confirms the structural amplification effect is a highly significant phenomenon and not product of random chance.
Gemini2.5 5 Permutation Analysis (Ocean Correlation)
Gemini2.5 Permutation Analysis (Ocean Excluded)
Distibution
Crigen
Orgiial
Original
Cog
300
600
(0
400
200
0.35
0101
00205
6.00
0.75
1.03
0.2
3.00
-.25
8.20
3.70
325
330
0.25
0.53
0.70
1.00
0.05
0100
3:25 0150
0.75
Figure 10: Permutation tests for Gemini 2.5 confirm statistical significance (p < .001). The ob- served statistics (red dashed line) for R2 and Kendall's are extreme outliers relative to the empiri- cal null distribution.
Robustness to Task Framing. To ensure the structural amplification effect arises from genuine psychological inference rather than being an artifact of the experimental setup, we tested the model's invariance across three distinct conditions: our original "Standard Order" setup, a "Random Order" setup and "Single Question" setup.
As shown in Figure II the amplification coefficient (k) for Gemini 2.5 remained stable across these conditions (k: T.42, 1.41, and 1.42, respectively). This consistency significant as these conditions systematically varied the task structure. The "Random Order" condition, which shuffled the sequence of the 20 input Big Five items, demonstrates that the model's reasoning is robust against variations in the input information's structure. The "Single Question" condition, which altered the task from predicting an entire questionnaire to predicting each item individually, shows that the effect is also robust against changes in the output task's 's format. The remarkable stability across these manipulations confirms that structural amplification a core feature of the model's 's reasoning process, not byproduct of specific input-output format.
Htrraa Geriing Swweean Oetered Downs Commerionn
Jummon et Demiiy 2 S00wen Random) Oceen Consration
Harree ve. GerreO TUSEncl Quedd Ooeer Comeesoo
1-1.41
K--1-42
5.82
Ri-
R
(01
4.0
0.22
0,2
0.25
4.2
-125
450
Figure 11: Robustness of the structural amplification effect in Gemini 2.5. The amplification mul-
tiplier (k) remains highly stable across Standard Order, Random Order and Single Question condi- tions.
23

---

Published as a conference paper at ICLR 2026
C.2 DECONSTRUCTING THE REASONING MECHANISM
C.2.1 CROSS-MODEL CONSENSUS IN ATTRIBUTION STRATEGY
To quantitatively assess the consistency of the information selection strategy across different models, we performed pairwise comparison of all attribution vectors generated. This analysis covered every combination of Reasoning Model and Annotation Model.
As shown Figure 12 the resulting attribution distributions exhibited remarkable alignment. Ex- cluding the trivial diagonal values, the average Pearson correlation coefficient was exceptionally high at p 0.9343, while the average Kullback- DKL 0.0475. Together, these metries confirm that all models independently converge on near- identical, fundamental attribution strategy.
(KL) divergence was extremely low at
Cormlation
Combinations
Combinations
Junc
Join oue
Jook prn uvsa 0:D1 Karo
pond
Desplee -hath
Oven-Oves
Oven -hhe
Oven odaa
Oten Cenni Lite
GM cls
C.l Creine
Figure 12: Pairwise comparison of attribution strategies across model combinations. Left: Heatmap of the Pearson correlation (p). Right: Heatmap of the Kullback- (DKL) divergence. Each axis label follows the format Reasoning Model -> Annotation Model. Together, the high correlations and low divergences demonstrate a strong cross-model consensus.
C.3 EFFICACY AND SYNERGY OF ABSTRACT SUMMARIES
To empirically validate our amplification coefficient (k), we tested its association with predictive performance, measured by the mean correlation (r). The analysis used data from 15 experimental conditions, generated by crossing five large language models (DeepSeek, GL.M, Qwen, Claude and Gemini) with three information input scenarios (SummaryOnly. ScoreOnly, and Summary+Score). The raw numerical results for each condition are listed in [
As shown in scatter plot of these data points (Figure 13 there is strong, positive linear rela- tionship between the two metrics. A high coefficient of determination (R² = 0.93) confirms that variations in amplification account for most of the variance in predictive correlation. This result strongly supports our claim that structural amplification is a key mechanism corresponding directly to enhanced predictive power.
24

---

Published as a conference paper at ICLR 2026
Table Amplification coefficients (k) and mean predictive correlations (r) for each model across the three information type conditions.
Model
Information Type
Mean Correlation (r)
Amplification Coefficient (k)
DeepSeck
SummaryOnly
0.361
1.36
ScoreOnly
0.376
1.38
Summary+Score
0,402
1.44
GLM
SummaryOnly
0.395
1,44
ScoreOnly
0.418
1.52
Summary+Score
0.421
1.54
Qwen
SummaryOnly
0.399
1,51
ScoreOnly
0.432
1.56
Summary+Score
0.435
1.61
Claude
SummaryOnly
0.403
1.54
ScoreOnly
0.442
1.62
Summary+Score
0,445
1.65
Gemini
SummaryOnly
0.345
1.33
ScoreOnly
0.374
1,41
Summary+Score
0.389
1.44
Coefficient vs. Mean Correlation Across Information Types
Information Types
0.93
Sumimaryynly
ScoreOnly
p .001
1.61
Summary Score
1.57
1.53
Coefficlent (K)
1.49
1.41
1.37
1.33
1.29
0.330
0.345 0.360
0.375 0.380 0.405 0.420 0.435 0.450 0.465
Mean Correlation
Figure 13: Correlation between the amplification coefficient (k) and mean predictive correlation (r) across 15 conditions (5 models 3 information types). The plot shows a strong, significant positive
linear relationship (R² 0.93. p < .001), demonstrating that higher structural amplification is associated with improved predictive performance.
C.4 IDEALIZATION HYPOTHESIS: RELIABILITY AND NOISE ANALYSIS
C.4.1 DATA RELIABILITY AND ATTENUATION CORRECTION
This section provides the detailed results supporting the reliability analysis discussed the main text. We demonstrate that (1) LLM-generann data has higher internal consistency than human data, and their reliability profiles converge, and (2) correcting for the attenuation human data brings the amplification coefficient (k) close 1.0.
25

---

Published as a conference paper at ICLR 2026
Figure 14 shows that LLM-generated data exhibits significantly higher internal consistency (Cron- bach's s Alpha) than human data across most psychological constructs (specifically, in 16 out of 18 cases; mean dLLM 0.87 vs. Human 0.75). This provides direct evidence that LLMs bypass a significant source of statistical noise inherent in human self-reports, which can stem from factors such inattention, fluctuating emotional states, momentary misinterpretation of items. Further- more, our analysis of the reliability profiles confirms strong convergence among LLMs towards shared, idcalized response model. The mean Mean Squared Error (MSE) between any two LLM profiles was substantially lower than between any LLM and the human profile (mean inter-LLM MuMatn--ummnn similar cognitive pattern that is systematically distinct from human.
MSE = 0.0060 vs. mean LLM-to- Human MSE 0.0357), highlighting that various LLMs share
Reliability Comparison: Human Al Models Across Constructs
Human Morees dMooe SDs
C.M
30
CM
Cronbach's 's Alpha
BR0
E30
15
IT, (100
me, Optimim
FR Strength
Tenscny
Self Humanlly
Self isoladion
Sett Minduuiness
ciltiiim
Kindress
coping N
Copinu P
The Anc cty
Pros
Psychological Constructs
Figure 14: Reliability Comparison: Human vs. AI Models Across Psychological Constructs. This bar chart displays the Cronbach's Alpha coefficients for data generated by humans (blue) and the mean across all AI models (purple, with eтror bars indicating standard deviation). In the majority of cases, LLM -eM-eeeerrtedd data exhibits higher internal consistency. supporting the hypothesis that LLMss filter random measurement error.
To directly test whether the structural amplification phenomenon is an artifact of measurement error, we performed a unilateral correction for attenuation on the human correlation data, as recommended by classical test theory (Nunnally) 1975). This standard psychometric procedure uses the reliabil- ity coefficients (Cronbach's Alpha) of two scales to estimate the "true score" correlation between them, effectively removing the attenuating effect of measurement error. We applied this correction to the entire human correlation matrix and then re-calculated the amplification coefficients (k) by regressing the original LLM correlation matrices against this newly disattenuated human matrix. As shown in Figure 15 after this correction, the amplification coefficients for all Large Language Models dropped significantly, clustering around value of 1.0. This result demonstrates that the amplification effect is largely accounted for by the unreliability in numan-generaan data, and that L.MMs effectively reconstructing psychological structure akin to the "true score" correlations.
26

---

Published as a conference paper at ICLR 2026
Linear Amplificatio Across Models (Corrected Human Unilateral Attenuation)
Model Calegories
Machine Loaminn Mode's
Large Langnage
Coefidient (k)
1.23
1.24
1.2
1.1
1.05
1.09
1.11
1.51
1.13
1.14
1:15
1.16
1.17
1.18
1.02
1.03
1.0
0.9
0.84
Decisin Tree
KNN
DespSook Thinking
Genri Chat
Trinkngg
Qwen Thinking
Cemini Thinking
GLM Chel
Qwen Chal
Claude Chai
Cluude Thinking
XGBoodi
GPT Chat
Random Forest
SVM
Regression
Bayesia Rudge
DespSook
Kimi
Model
Figure 15: Amplification Coefficients After Unilateral Attenuation Correction. The amplification coefficients (k) for all LLMs now cluster around 1.0 after correcting for measurement error in human data. This suggests they are modeling the disattenuated, "true score" psychological structure, rather than simply amplifying noisy, observed correlations.
C.4.2 EMPIRICAL SUPPORT FOR THE IDEALIZATION HYPOTHESIS
In Section 4.1, we posit that the LLM may function as an "idealized participant" by abstracting away the noise inherent in human responses. To provide empirical support for this hypothesis, conducted two complementary analyses on both human and model side.
Analysis of Attentive Human Participants. To isolate the effect of human inattention key source of data noise we identified a subgroup of attentive participants (Natemiiv 309) from the full dataset (N = 816). This was operationalized by excluding individuals based on their response times. Specifically, the attentive subgroup consists of participants whose response times on all ques- tionnaires were consistently above a lower-bound threshold, defined the mean of inter-response time differences minus half a standard deviation. As shown in Figure [6 this low-noise subgroup exhibited stronger internal correlations. quantified by amplification multiplier of 1.08 when compared to the full sample. This provides strong correlational evidence that as human-gent -enae-egeeeaaed noise decreases, empirical data converges toward the idealized structure captured by the L.M.
All vs Filtered Human Subjects Ocean Correlation
1.25
Fitered Hurre Subjects Orawan Conebation
1.00
R²
0.98
0010
0.50
0.00
0.25
0.75
-1.00
-1.55
-1.00
4.78
.0.50
-025
0.00
0.25
0.60
0.75
1.00
1.25
1.50
All Human Subjects Oosan Correlstion
Figure 16: Scatter plot comparing the OCEAN intra-correlation coefficients derived from the full hu- man sample (All Subjects, N=816) against the more attentive subgroup (Filtered Subjects, N=309).
27

---

Published as a conference paper at ICLR 2026
The Noise Injection Experiment. To establish near-causal link, intervened baseline Linear Regression model by systematically adding Gaussian noise (a E (0.25, 0.5, 0. 0.75, 1.0)) to its predictions. As visualized in Figure [7] this revealed a clear dose-response relationship: as injected noise increased, the amplification coefficient k systematically decreased from 1.55 to 1.12. This inverse relationship between noise and amplification supports hypothesis that LLMss achieve this effect through process of noise filtering.
Juman Linear Model Ocean Carrelation (Noise Level: 0,25)
150
Human vs Linear Model Ocean Correlation (Noise Level: 0.5)
1.55
1.5
R .696
K 1.38
1.00
RP 0.96
.000
Linear Modal Doean Corsalinn
0.75
2.73
0.50
0.00
0.25
00.00
0.50
33.50
4.25
-1.25
-1.50
1.50
-.85
1.00
0.75 0.50 0.25 0.03 0.25
0.75 1.00
1.25
1.50
-1.50
1,50
1.25 -1.00 0.75 0.50 0.25 0.50 0.25 0.50
0.75
1.00
125
1.0
Human Ossan Committion
Human Olesnn Comsalion
Human Linear Model Ocean Correlation (Noise Level: 0.
Human Linear Model Ocean Correlation (Noise Level:
1.29
k
1.00
1.00
P :001
0.70
=
Unner Model Orcaan Corelatin
gtc .000
0.50
0.50
0.25
0.29
0.00
0.00
4.25
00.55
0.50
0.50
-1.0
-1.00
1.00 1,20
1.30
0.75
0.30 0.2b 0.0 0.25 0200
0.75
1.00
125
1.50
1.00 1.26
-1.00
0.70
0000 1125 000 028 0.30
0.75
1.00 125 1.0
Human Ocean Correcttion
Human Ocean Corretation
Figure 17: Results of the noise injection experiment. As the level of Gaussian noise added to the linear model's predictions increases from 0.25 1.0, the amplification coefficient (k) systematically decreases, demonstrating clear dose-response relationship.
D TARGET PSYCHOLOGICAL SCALES
For clarity and reproducibility. the following table lists the full names and abbreviations of the psychological scales used as prediction targets in our experiments. Detailed descriptions of the scales and their items available in GitHub repository.
28

---

Published as a conference paper at ICLR 2026
Table 3: List of Sub-scale Abbreviations, Their Parent Scales, and Color Mappings.
Parent Scale
Sub-scale Abbreviation
Color
Perceived Stress Scale (Cohen al. 1983 et al.
Leung
PSS
black
2010
Coping P
darkblue
Simplified Coping Style Questionnaire Xie
1998
Coping N
State- Trait Anxiety Inventory (Trait, Spielberger
Anxiety
darkgreen
al.
1971
_ef__Critiiism
darkviolet
_ef__Isolation
Over ID
Self- Ceff-Compassion Scale Neff 2003
Self Kindness
Self Humanity
Self Mif_MMindulless
Self Compassion
PR PR_ IR_Tenacity
darkorange
Psychological Resilience Scale (CD-RISC) (Connor & Davidson 2003)
PR Strength
PR_ PR_Optimism
Intolerance of Uncertainty Scale Buhr & Dugas 2002)
Pros_Anx
darkred
Inhib_Anx
Emotion Regulation Questionnaire Gross & John 2003)
ERQ R
darkgray
ERQ S
Risk Unrel
darkcyan
Risk Perception & Behavior Questionnaire
Obj Risk
Future Time Perspective Scale (Carstensen &
FT Persp
darkgoldenrod
Lang 1996)
LLM USAGE STATEMENT
In this work, large language models served an assistive tool the research and writing process. The model's contributions included assisting with drafting and iteratively revising the manuscript- such as generating initial text and rephrasing sentences to improve clarity-and helping to write and debug Python code for the data analysis pipeline. For manuscript presentation, the LLM's role was specifically limited to generating 18TEX code for several icons used schematic figures and assisting with the aesthetic formatting of tables. The authors retained full intellectual control and bear complete responsibility for the final content, having critically reviewed, edited, and validated all Al-generated contributions. The model is therefore not eligible for authorship. and its role is acknowledged here in the spirit of transparency.
29


Reviews:
[{"review_id": "ps2bIGgpC4", "reviewer": "Reviewer_GtxT", "summary": "The authors show good alignment between LLM predictions and human psychological structure comparing inter-scale correlation LLMs patterns between LLM predictions and human data. This is stark contrast with recent results by Zhu et al. (2025) showing poor LLM alignment when inferring personality of a narrator from a given text.\n\nThe authors claim that the reason of this contrast is that though LLMs struggle to predict specific traits they tend to amplify the correlations between scores on different scales that human subjects show.\n\nThis contradiction is particularly interesting, in my opinion, and might span a very fruitful line of further research.", "questions": "Could the correlation amplification phenomenon you find be the result of post-training in the alignment phase, when models are trained to simulate an agreeable personality?\n\n\n\nHere are some further works that could be informative for the scope of the work:\n\nSorokovikova A, Rezagholi S, Fedorova N, Yamshchikov IP. LLMs Simulate Big5 Personality Traits: Further Evidence. InProceedings of the 1st Workshop on Personalization of Generative AI Systems (PERSONALIZE 2024) 2024 Mar (pp. 83-87).\n\nPan X, Gao D, Xie Y, Chen Y, Wei Z, Li Y, Ding B, Wen JR, Zhou J. Very large-scale multi-agent simulation in agentscope. arXiv preprint arXiv:2407.17789. 2024 Jul 25.\n\nDong W, Zhao Y, Sun Z, Liu Y, Peng Z, Zheng J, Zhang Z, Zhang Z, Wu J, Wang R, Xu S. Humanizing llms: A survey of psychological measurements with tools, datasets, and human-agent applications. arXiv preprint arXiv:2505.00049. 2025 Apr 30.\n\nTshimula JM, Nkashama DJ, Muabila JT, Galekwa RM, Kanda H, Dialufuma MV, Didier MM, Kalonji K, Mundele S, Lenye PK, Basele TW. Psychological Profiling in Cybersecurity: A Look at LLMs and Psycholinguistic Features. InInternational Conference on Web Information Systems Engineering 2024 Dec 2 (pp. 378-393). Singapore: Springer Nature Singapore.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The authors suggest a great research question, find an answer to it and demonstrate the validity of the answer in a convincing way.", "weaknesses": "I am not an expert in classical psychology literature, but some of the human-score correlations should be taken with a grain of salt due to the fact that they are often reported on small selection of subjects that are not representing the scale of human experience that an LLM is exposed to. In this context the stronger correlation that the authors report is even more surprizing to me, especially in the context of the later remark on more \"attentive\" subjects.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T48Bov42ar", "reviewer": "Reviewer_Pih4", "summary": "The paper studies whether LLMs can recover the latent structure of personality. (1) Given only each person’s Big Five item responses, models ‘role-play’ the person on other 9 psychological scales, and the resulting correlation structure closely mirrors human data but with a consistent, stronger signal. (2) Analysis of generated reasoning traces suggest a two-stage process: compressing human responses into summaries, then reasoning from those summaries to answer new items. Controls rule out simple semantic matching and indicate the effect isn’t due to chance. The authors argue the amplification reflects “idealized version” of human participants, with LLMs filtering human noise.", "questions": "There are multiple vague terminologies that I ask authors to clarify.\n\n- Line 58 “the test should enable mechanistic interpretability”, line 75 “to further achieve interpretability”, line line 464 “using advanced mechanistic interpretability” -> I don’t get why authors mention mechanistic interpretability. This paper is completely irrelevant to mechanistic interpretability. Could authors please describe how their test enables mechanistic interpretability, e.g. is it able to establish casual relationships through interventions on activations?\n\n- Line 25: “analogous to generating sufficient statistics” -> How is it analogous to sufficient statistics? In the paper, there is no description why and how generating natural language personality summaries is relevant to sufficient statistics..\n\n- Line 193: “linear amplification” -> is linear amplification a terminology used in the community? I think the analysis done here is effectively summarized as calculating correlation coefficients between entries in LLM-prediction correlation matrix and human correlation matrix, getting the coefficient > 1.\n\n- Line 151: “ops‑bge‑reranker‑larger” -> there is no model named “ ops-bge-reranker-larger” exists, looks like a hallucinated model label.\n\nI would like to ask authors about representativeness, availability, and ethical approval detail of 816 human subjects.\n\n- Representativeness: is it a representative sample of the entire population?\n\n- Availability: It “was collected during COVID-19 as a part of a larger study” (Line 143), is it published somewhere as a publicly accessible dataset? If so, please mention the original data source rather than just a description of it. If not, please mention the future availability of it for reproducibility.\n\n- Ethical approval detail: following ICLR code of ethics, I would like to ask authors for more details of human subject recruitments, including IRB process, documentation, etc.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Moving from the first‑order prediction to the second‑order correlation‑structure analysis is an interesting reframing of the psychology questions with LLMs. The heatmaps and regression plots in Figure 2 (page 5) compellingly visualize the phenomenon.\n\n- An attempt to mechanism‑seeking analysis: factor‑level and item‑level attribution comparison in Figure 4, and amplification depending on information type in Figure 5 are good steps toward understanding the model's decisions rather than reporting raw scores alone.", "weaknesses": "- Related Work (Section 2) significantly lacks depth to situate the paper within existing literature.\n\nIn particular, it overlooks a growing body of research suggesting that LLMs exhibit latent psychological structure of humans. Few papers I found by search are “Personallm: Investigating the ability of gpt-3.5 to express personality traits and gender differences” (Jiang et al.), “Personality traits in large language models” (Serapio-Garcia et al.), “Rediscovering the Latent Dimensions of Personality with Large Language Models as Trait Descriptors” (Suh et al.), to name a few. Currently the related work is a bland listing of a few disconnected sets of papers, and must be strengthened to understand the value of this paper.\n\n- LLMs, especially reasoning models, would have utilized its ‘knowledge’ of correlation between traits rather than role-playing.\n\nAlthough inputs to LLM are just Big-Five item results, LLM still reads what are the target items it will predict (e.g., “ERQ questionnaire items” in Line 782). Their well‑known semantics (i.e., LLMs know what is ERQ, and possibly how it is related to Big-Five items from pretraining corpus and past psychological research results) could let the model rely on learned associations between Big-Five traits and those constructs rather than inferring a structure unique to this dataset. I think this is especially prevalent when using reasoning models. Reasoning models tend to produce reasoning trajectories in terms from a third-person perspective rather than actual role-playing a given persona / traits, try to come up with logical explanations, far from what the paper claims LLMs do to make a prediction. However, from the current paper I cannot see how the model actually works nor can run reproducible experiments; it would be great to see the real example of Figure 3 (“reasoning-to-annotation” analyses).\n\n- Is ‘linear amplification’ an ideal thing, and isn’t it confounded with unreliability (Section 3)?\n\nThe slope k>1 is interpreted as “idealization”. However, when the human correlation matrix is attenuated by measurement variance and inherent variability during the human's choice making process, while the LLM prediction correlation matrix is less noisy, regressing the latter on the former will naturally produce a slope k>1. Also, internal consistency (measured in terms of Cronbach’s alpha, for example) of too high values is not necessarily desirable. Without disattenuating human correlations for scale reliabilities (e.g., Cronbach’s alpha per subscale) or applying classical correction for attenuation, the amplification claim risks restating a reliability artifact.\n\n- Reproducibility.\n\nI checked the repo, and there are only .pdf files of questionnaires and no code available for analysis and experiment, limiting the reproducibility of the experiment since authors have brought multiple ML models for evaluation (Figure 2).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "36RdVsOflP", "reviewer": "Reviewer_UCxr", "summary": "The paper presents a study of the ability of LLMs using chain-of-thought prompting to reconstruct personality profiles using psychometric data and use this information to reconstruct scores on other psychometric measures in a coherent and consistent manner compared to the original human behavioral scores. By analyzing reasoning traces, the authors find that LLMs are reasoning from representations and not simply performing semantic pattern matching.", "questions": "P30L130: \" Our findings chart a path up this hierarchy, demonstrating that LLMs’ cognitive process (1) transcends surface-level statistical association, (2) prioritizes abstract conceptual structure over specific item-level details, and (3) performs a novel form of theoretical idealization, actively refining noisy inputs into theory-consistent representations\"\n=> It's not immediately clear how a correlational study, even a sophisticated one, can demonstrate that LLMs prioritize abstract conceptual structure.The experimental setup prompts chain-of-thought reasoning. Hence, how is any evidence of abstract reasoning not simply an artifact of the prompt structure?\n\nP5-6L264-276: \"We performed two further analyses to validate the observed linear structural amplification is genuine and robust. First, we established statistical significance using a 1,000-trial permutation test. [...] Second, to verify the effect arises from genuine understanding rather than semantic mapping, we examined the model’s sensitivity to task framing or input ordering (Oren et al., 2023) by testing three conditions: our original setup (Standard Order), a setup with shuffled input items (Random Order), and a setup where each prediction was made individually (Single Question\"\n=> This is good for excluding context effects like chat history, but I fail to see how this validates that the amplification is not a result of chain-of-thought prompting.\n\nP7L344-247: \"This clear dissociation uncovers a key insight into the models’ reasoning: LLMs robustly identify the correct personality factor (e.g., Neuroticism) but struggle to differentiate the importance of specific items within that factor. This supports the hypothesis of a top-down, concept-driven process where high-level abstract knowledge guides the inference, even if low-level execution is imprecise.\"\n=> Or the abstraction is belied by the lack of internal representation. This confusion could equally result from semantic pattern matching. How do you exclude this possibility? The high levels of noise suggest that the regression models are incomplete. Did you prompt the models to explain their choices?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper is well-argued and well-structured. By comparing scores across a range of psychometric tests, the experimental design address a fundamental gap in the literature, namely with regard to the coherence and stability of LLM personalities. Further, the experimental setup allows for comparisons across models and across learning algorithms, which strengthens the findings.\nThe paper offers an insightful analysis and discussion of LLMs tendency towards idealized representations.", "weaknesses": "The correlational analysis of Experiment 1 presented in 3.2 is interesting from an exploratory perspective but it is not very convincing and slightly problematic: Multiple comparisons are discouraged and should be replaced with more robust multivariate tests. Indeed, experiment 2 and the subsequent analysis in 4.2 is much more conclusive.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KPvapAyHWS", "reviewer": "Reviewer_GETG", "summary": "This paper proposes a controlled paradigm to test whether large language models can infer psychological structures from the big-5 personality traits of a user. Given individual responses from 20 elements in a big-5 questionnaire, models are prompted to role-play that individual across 9 other, distinct, psychological scales. In the first experiment, the paper compares the inter-scale correlation matrices between. model-generated scores and human ground-truths, showing that LLMs often closely reproduce humans' correlation patterns (Gemini, R=0.92), and that they display an amplification effect in the correlations. In a second experiment, the paper investigates how models achieve this correlation, by focusing on the reasoning traces generated during thinking, showing that models are consistent (there is high correlation between the attribution vectors of different reasoning models), but do not have fine-grained attribution ability (instead focusing on modre global patterns), which is confirmed by ablation (using a summary in addition to the original numeric elements).", "questions": "- How does this change for non instruction-tuned models? it seems like many of these artifacts may come from the optimization process during RLHF or similar tuning processes. \n- Is there a correlation between length of the thinking trace, and some of the \"summary\" behavior? It feels like in some cases, the thinking trace performs as a summary for the model, allowing it to reason/perform test-time compute instead of directly producing an output.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 2, "strengths": "This is an interesting paper, which introduces a fairly novel way of evaluating how well LLMs can capture and express personalities. The experiments are well thought out, and they show some interesting artifacts, particularly the results on attribution and COT reasoning/summarization. The methodology is strong, and the results are convincing (and likely reproducible). The work is reasonably clear in its core motivation and experimental pipeline, and the finding that LLMs can reconstruct and amplify latent psychological structure from minimal inputs has potential implications for downstream applications (particularly in behavioral simulation)", "weaknesses": "- The related work section is somewhat thin on detail, and largely ignores a rich set of works designed to approximate specific behaviors, even up to the Big-5 personality traits. Some relevant related work beyond the park study of LLMs include [1,2,3,4,5,6,7,8,9]. It's really not clear to me that the paper significantly differentiates itself solely by \" introducing a well-controlled psychometric paradigm\", since many of these other paper do the same (either in the domain of behavior, or opinion). There are also papers which use Big-5 as conditioning, such as [13,14], and the ability of LLMs to emulate personality was explored in [15] (which is quite similar to this work). \n- There's no ablation of the experimental prompt in A.1. It's likely that using a system paradigm, with the response in the user, compared to several back to back user turns will affect the performance of the model (and the strength of the correlations). Similarly, it's likely that the wording of the system prompt has impacts on the output. It would be good to perform some analysis accounting for that. In addition, it's interesting that all models here are instruction-tuned models: how does this change if a base model is used instead?\n- Some of the results seem a bit cherry-picked per-model/experiment. For example, Figure 2 only discusses Gemini, but how does this change for other models? \n\nMore minor issues:\n- The scales used are never discussed in the main text, and only included in Appendix D.\n- Language models are only discussed in Appendix E (which makes it really hard to follow what the overall experiments are). \n- I'm not entirely sure how novel the \"amplifcation\" idea is. Several papers [10,11,12] have explored how LLMs amplify biases (and much of this tracks with how LLMs perform maximum likelihood prediction). \n- The presentation of the paper is a bit dense, and wanders a lot, making it hard to understand exactly what is being tested, what the research questions are, and what experiments are being performed on which models. Restructuring the paper around clear RQs and hypotheses would significantly improve the readability of the paper. \n- There's no pipeline ablation in this paper (ie. the annotation model in Figure 3), which makes it hard to understand if the underlying results are from biases in the annotation models, or biases in the data itself. \n\n[1] Ziems, Caleb, et al. \"Can large language models transform computational social science?.\" Computational Linguistics 50.1 (2024): 237-291.\n[2] Dillion, Danica, et al. \"Can AI language models replace human participants?.\" Trends in Cognitive Sciences 27.7 (2023): 597-600.\n[3] Aher, Gati V., Rosa I. Arriaga, and Adam Tauman Kalai. \"Using large language models to simulate multiple humans and replicate human subject studies.\" International conference on machine learning. PMLR, 2023.\n[4] Tjuatja, Lindia, et al. \"Do llms exhibit human-like response biases? a case study in survey design.\" Transactions of the Association for Computational Linguistics 12 (2024): 1011-1026.\n[5] Choi, Hyeong Kyu, and Yixuan Li. \"Picle: Eliciting diverse behaviors from large language models with persona in-context learning.\" arXiv preprint arXiv:2405.02501 (2024).\n[6] Hilliard, Airlie, et al. \"Eliciting personality traits in large language models.\" arXiv preprint arXiv:2402.08341 (2024).\n[7] Santurkar, Shibani, et al. \"Whose opinions do language models reflect?.\" International Conference on Machine Learning. PMLR, 2023.\n[8] Kang, Minwoo, et al. \"Deep Binding of Language Model Virtual Personas: a Study on Approximating Political Partisan Misperceptions.\" Second Conference on Language Modeling.\n[9] Suh, Joseph, et al. \"Rediscovering the Latent Dimensions of Personality with Large Language Models as Trait Descriptors.\" NeurIPS 2024 Workshop on Behavioral Machine Learning.\n[10] Taori, Rohan, and Tatsunori Hashimoto. \"Data feedback loops: Model-driven amplification of dataset biases.\" International Conference on Machine Learning. PMLR, 2023.\n[11] Wang, Ze, et al. \"Bias Amplification: Large Language Models as Increasingly Biased Media.\" arXiv preprint arXiv:2410.15234 (2024).\n[12] Peña Fernández, Simón, et al. \"Without journalists, there is no journalism: the social dimension of generative artificial intelligence in the media.\" Profesional de la información 32.2 (2023).\n[13] Li, Wenkai, et al. \"Big5-chat: Shaping llm personalities through training on human-grounded data.\" arXiv preprint arXiv:2410.16491 (2024).\n[14] Vu, Huy, et al. \"Psychadapter: Adapting llm transformers to reflect traits, personality and mental health.\" arXiv preprint arXiv:2412.16882 (2024).\n[15] Wang, Yilei, et al. \"Evaluating the ability of large language models to emulate personality.\" Scientific reports 15.1 (2025): 519.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)