PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits
Authors: Loka Li
Wong Yu Kang
Minghao Fu
Guangyi Chen
Zhenhao Chen
Gongxu Luo
Yuewen Sun
Salman Khan
Peter Spirtes
Kun Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Datasets
LLM-Inferred Behavior Traits
Causality
Primary area: datasets and benchmarks
TL;DR: We present PersonaX, a meticulously curated collection of multimodal datasets that analyze the relationships among human behavior traits, facial attributes, and other biographical features.

Abstract: Understanding human behavior traits is central to applications in human-computer interaction, computational social science, and personalized AI systems. Such understanding often requires integrating multiple modalities to capture nuanced patterns and relationships. However, existing resources rarely provide datasets that combine behavioral descriptors with complementary modalities such as facial attributes and biographical information. To address this gap, we present PersonaX, a curated collection of multimodal datasets designed to enable comprehensive analysis of public traits across modalities. PersonaX consists of (1) CelebPersona, featuring 9444 public figures from diverse occupations, and (2) AthlePersona, covering 4181 professional athletes across 7 major sports leagues. Each dataset includes behavioral trait assessments inferred by three high-performing large language models, alongside facial imagery and structured biographical features.
We analyze PersonaX at two complementary levels. First, we abstract high-level trait scores from text descriptions and apply five statistical independence tests to examine their relationships with other modalities. Second, we introduce a novel causal representation learning (CRL) framework tailored to multimodal and multi-measurement data, providing theoretical identifiability guarantees. Experiments on both synthetic and real-world data demonstrate the effectiveness of our approach. By unifying structured and unstructured analysis, PersonaX  establishes a foundation for studying LLM-inferred behavioral traits in conjunction with visual and biographical attributes, advancing multimodal trait analysis and causal reasoning. 
The code is available at https://github.com/lokali/PersonaX.

Published as a conference paper at ICLR 2026
PERSONAX: MULTIMODAL DATASETS WITH LLM- INFERRED BEHAVIOR TRAITS
Loka Lif", Wong Yu Kangl Minghao Ful.3, Guangyi Chen 1.2, Zhenhao Chen', Gongxu Luo', Yuewen Sun Salman Khan Peter Spirtes", Kun Zhangl. Mohamedd bin Zayed University of Artificial Intelligence, 2 Carnegie Mellon University 3 University of California San Diego, 4 Australian National University
ABSTRACT
Understanding human behavior traits is central to applications in human-computer interaction, computational social science, and personalized Al systems. Such under- standing often requires integrating multiple modalities to capture nuanced patterns and relationships. However, existing resources rarely provide datasets that combine behavioral descriptors with complementary modalities such as facial attributes and biographical information. To address this gap, we present Persona curated collection of multimodal datasets designed to enable comprehensive analysis of public traits across modalities. Pers onaX consists of (1) CelebPersona, fea- turing 9444 public figures from diverse occupations, and (2) AthlePersona, covering 4181 professional athletes across major sports leagues. Each dataset includes behavioral trait assessments inferred by three high- segh-perforinng large lan- guage models, alongside facial imagery and structured biographical features.
We analyze Person versona at two complementary levels. First, we abstract high-level trait scores from text descriptions and apply five statistical independence tests to examine their relationships with other modalities. Second, we introduce a novel causal representation learning (CRL) framework tailored to multimodal and multi- measurement data, providing theoretical identifiability guarantees. Experiments both synthetic and real-world data demonstrate the effectiveness of our approach. By unifying structured and unstructured analysis, PersonaX establishes a foun- dation for studying LLM-inferred behavioral traits in conjunction with visual and biographical attributes, advancing multimodal trait analysis and causal reasoning. The code is available at tt thub. com
CelebPersona: aegpinnaaeeoollaaaee///eeeeeeeeo
AthlePersona:
INTRODUCTION
Human behavior traits (or behavioral summaries) refer to outwardly observable patterns of conduct inferred from public information such as spoken or written language, facial expressions, and biograph- ical records (Rothe 2017 Johnson 1997 DeNeve Cooper 1998 Briggs Cheek 1986). These traits differ from psychological personality, which concerns internal dispositions typically measured through self-reports or expert evaluation (Cattell al.) 1970 Eysenck & Eysenck 1975] Myers et al.)
1998. Goldberg 1993). Unlike clinical diagnoses, behavior traits can be inferred ethically and at scale from non- intrunirusive signals, offering reproducible, population-level insights that complement personality research without medicalizing individuals. Advances in large language models (LLMs) (Achiam et al.] [023]] Floridi & Chiriatti| [2020) Jiiang et al.) 2024: Liu et al.) (2024) have further expanded this teasibility. Several studies demonstrate that L.I. assessments of behavior traits aligned with the Big Five framework can be reliable under carefully designed prompting strategies Serapio-Gareis al. 2023 Jiang et al., [2023] Tseng et al. 2024 Zou al. 2024). These approaches enable large-scale, automated analysis while mitigating some biases inherent in self-reports.
Related Work. Research on human attributes spans two complementary directions: internal personal- ity and external behavior traits. Psychologically internal personality has traditionally been measured
Equul contributions.

---

Published as a conference paper at ICLR 2026
Data processing pipelines of AthlePersona (Left) and CelebPersona (Right) datasets. (1) AthlePersona was constructed by collecting player rosters and publicly available data (ineluding facial images and basic features) from the official websites of major sports leagues. These data were then processed with LLMs for inferring behavior traits. (2) CelebPersone
Figure 1:
was derived from the CelebA dataset (Liu et al.) 2015). Celebrity face identities were linked to their corresponding Wikidata entities, enabling the retrieval of additional biographical details and physical characteristics, which were similarly processed with LLMs for inferring behavior traits.
AthlePersona
CelebPersona
Use fiee Aloray accaa Wildar cidee
GaRA Duastt
Doceppion
Thacter anyy thu heleer
Backs
"Auupa Nu lig Niviia volo
C0
(400)
CaltPrinns (3id4 onttertoo
Rosie Festares Hi
Reharier Treis
Rosic Festeres
Betooker Trais
Face Featuers
Imigr
A
Leaple
Clapet Aosepion CARDAC Jocie
Keyblt
Chags Matrye
Soatet Frlherd Fig Vac. Rarm New
Pay
dicha
Calbill Loggra Payk Hogg
Lai Accminn
Ragel Mhipl
Narre Gurther
with self-report instruments such as the 16PF (Cattell et al. 1970), EPQ (Eysenck & Eysenek 1975). MBTI (Myers et al.] (198), and the Big Five framework (Goldberg 1993). In contrast, behavior traits emphasize outwardly observable patterns, inferred from signals such as text, facial expressions, physiology, or digital traces. Several datasets target this perspective, including SALSA for group interactions (Alameda-Pinen) et al., 2015), nonsocial-conter
datasets for daily activities (Dotti et al. 2018), driving and physiological data for trait prediction (Evin et al. 2022). and lifelog corpora capturing multimodal daily behavior (Chung et al.) 2022). Digital records such as Facebook Likes have also been shown to predict sensitive traits, including personality dimensions (Kosinski et al. 2013). Other multimodal resources, such as YouTu--VVlog (Biel & Gatica-Perez (0022). FI-V2 (Escalante al., 2020), MuPTA (Ryumina et al.. 2023), and MDPE (Cai et al., 2024), combine video, audio, or oogical signals for prediction tasks like impression analysis or deception detection, but they typically lack explicit textual trait descriptions or frameworks for cross-modal interpretation. A comparison table of different datasets is in Tab. A1 Beyond datasets, empirical studies show that observable features in one modality can signal traits in another. For instance, facial structure has been linked to health and aggression cues (Kramer & Ward 2010 Carré & McCormick 2008), body images to personality judgments (Naumann et al./ 2009), and facial behavior to Big Five traits (Cail & Liu 2022). Together, these works underscore the promise of behavior trait analysis, but existing resources are limited for systematic cross-modal and causal study. See App. A2 for more details.
To address these gaps, introduce PersonaX, a curated collection of multimodal datasets that contain LLM --iferrrred behavior traits. The assessments are derived from public information, including direct quotes from interviews, observed behaviors, career trajectories, and biographical details. For consistency, we follow the Big Five framework (Goldberg) 1993), providing trait scores across its five dimensions. PersonaX includes (i) CelebPersona, comprising 9444 public figures from the CelebA dataset (Liu et al. 2015), and (ii) AthlePersona, covering 4181 professional athletes across seven major sports leagues. Each record integrates (1) textual trait descriptions and Big Five scores inferred by three high- peegfomifonnngg LLMs, (2) facial images, and (3) structured biographical metadata. To safeguard privacy. we release only transformed embeddings rather than raw images or text. The proposed dataset provides a unique foundation for cross-moda and causal analysis.
Our contributions are mainly twofold. (i) We release PersonaX, a set of multimodal datasets that combine LLM--ifferee behavior traits, facial embeddings, and biographical metadata for large populations of public figures. (ii) We introduce a two-level analysis framework: at the structured level, applying diverse independence tests to uncover behavior-trail dependencies; and at the unstructured level, proposing a causal representation learning approach with identifiability guarantees tailored to multimodal, multi-measuremme demonstrate the practical effectiveness of this framework. By unifying structured and unstructured perspectives, PersonaX enables systematic study of L.LM-inferred traits alongside visual and
settings. Experiments on both synthetic and real-world data
2

---

Published as a conference paper at ICLR 2026
Table 1:
Evaluations on LLM selection with CelebPersona and AthlePersona subsets. Metrics consist of generation time (GT), missing rate (MR), indecisive rate (IR), privacy preservation (PP), output formatting (OF), context consistency (CC), factual accuracy (FA), and an overall score
(OS), Please refer to App. A4.1 for more details about models and the definition of metries.
Model (LLMss
CelebPersona
AthlePersona
GT
MR.
IR]
PP
OF
CC2
FA
OS
GTL
MR.
IR]
PP1
OF
CCI
FAt
OS?
3 ChatGPT-4o
4.19
0.03
0.17
0.59
1,00
1.00
1,00
0.96
3.92
0.27
0.17
1.00
1.00
0.99
1.00
0.93
Gemini2,5-Po
23.48
0.06
0.19
0.99
1.00
1.00
1.00
0.96
21.31
0.29
0.22
0.99
1,00
1.00
1.00
0.91
Owen2.5-Max
0.24
0.29
0.99
0.99
0.91
0.32
0.99
0.99
0.88
0 Grok-3-Beta
5.92
0.34
0.17
1.00
1.00
0.99
1.00
0.91
4.96
0.66
0.10
1.00
1.00
1.00
1,00
0.87
Laama
3.73
0:25
0.29
1.00
1.00
0.97
1.00
0.90
3.99
0.30
0.43
1.00
1.00
0.95
1.00
0.87
Gemini2.0-FT
0.28
0.38
1.00
1.00
0.89
0.48
1.00
0.87
* DeepSeck-RI
39.13
0.40
0,11
0.98
0.90
1.00
1,00
0.89
26.61
0.64
0.10
1.00
0.81
1.00
0.98
0.84
QwQ-32B
9.22
0.24
0.46
1.00
0.99
1.00
1.00
0.88
8.87
0.30
0.48
1.00
0.98
1.00
1.00
0.87
DeepSeek-VV
0.47
0.24
1.00
1.00
0.88
0.64
0.20
1,00
1.00
0.86
Gemini2.0.F
2.53
0.43
0,32
1,00
1.00
0.99
1.00
0.87
2.29
0.69
0.18
1.00
1.00
1.00
1,00
0.86
biographical attributes, opening new pathways for deeper multimodal interpretation and causal reasoning. Our long-term vision is to leverage such resources to uncover invariant causal patterns across populations, thereby advancing diversity, equality, and mutual respect for all human beings.
PERSONAX DATASET
In this section, we introduce PersonaX. collection of two complementary multimodal datasets: AthlePer eonnsona and Ce CelebFersona: Together, they provide acgeesccal resources for studying LLM-inferred behavior traits in conjunction with visual and biographical attributes. We first describe the construction of each dataset below, then detail the selection of LLMs and prompts for trait generation in $ 2. and finally discuss consent, privacy, and bias considerations in $ $2.2
AthlePersona. Built from scratch, this dataset documents 4181 male professional athletes across seven major sports leagues worldwide, including the NBA, NFL, NHL, ATP, PGA, Premier League, and Bundesliga" From official league sources, collected biographical information (c.g., name, birth date, nationality), physical attributes (e.g.. height, weight), and facial images. Nationalitics were geocoded into continuous spatial coordinates (latitude and longitude) to support geographic analyses.
CelebPersona. This dataset builds on the established CelebA dataset (Liu et al. 2015). which contains rich facial attribute annotations. We linked each celebrity's name to its corresponding WikiData entity, enabling retrieval of additional biographical details and physical characteristics. From the original 40 CelebA attributes, we manually retained 10 (e.g., .., Big Nose, High Cheekbones) that reflect more stable, inherent appearance properties, while discarding attributes subject to short- term variation (c.g.. Heavy Makeup). CelebPerse sona totally contains 9444 public figures.
Multimodality. Each record integrates three components: (1) textual behavior-trait descriptions and Big Five scores inferred by LLMs, (2) facial images or embeddings with attribute annotations, and (3) structured biographical metadata. The full feature lists of AthlePersona and Ce WelebPersore are shown in T2b and Tab respectively. Dataset distributions for each feature are presented Terms-of-use compliance across all sports leagues for AthlePersons is
in Fig [lAandd and Fig. A2
summarized in Tab A4
The complete prompt for inferring behavior traits is provided in Prompt]
2.1 LLM SELECTION AND PROMPT DESIGN
We systematically evaluated ten sfte-offtheeart LLM across both AthlePersona and Ce CelebPersona. A summary of those model performances is shown in Tab. [ Full model details,
We also examined four additional leagues (MLB, La Liga, Serie A, and Ligue 1), but could not include them in the current release due to pending written consent requirements for academic research. A complete summary of terms-or terms-of-use compliance including the original statements provided in Tab Tab.A4 See App A3.4 for details.
IIiitillly, we considered the Top 10 models from the Arena leaderboard Chiang et al. 2024 on April
10, 2025, supplemented with Qwen2.5-MM Max and QwQ-32B (Bai et al. (023) for diversity, GPT445 G.T.,5--revveew (Achiam et al. 2223) was excluded due high API costs, and Gemini-2.00 2.0-ii--.-PPPPPP-- P12--2 02-05 (Team et al. 2023) was merged into the later Gemini-2.5 Pr5- release. Thus, ten models were ultimately retained for evaluation.
Preview
3

---

Published as a conference paper at ICLR 2026
Figure 2: Evaluation on LLM consistency for prompt design. Top:
Radar plots show the standard
deviation (std) of Big Five trait scores actoss repeated runs under different prompt formats, for each model (by column) and dataset (by row). Middle: Box plots summarize the average of std across Big Five behavior traits, highlighting intru-prompi variability. Bottom: Manhattan distances between two
prompt pairs quantify inter-prompl variability, Refer to $2.1 for more setup and result analysis.
Llame---Maverick
Geminl-2.5-Pro
ChatGPT-4o-Latest
@wen2.5-Max
Open
Open,
Open
Open
CalabParsona
Neuço.
Cors. Neuró
Cons. Nougo
Cons. Neujo
Cons. Neuro
Cors.
Agree....
Extra.
Agrab
Extra
Agrée
Extra.
Open
Open
Open
Open
AthlePersona
Nour
Cons.
Cons.
Agre
Extra.
Agreb
Extra
Agroe
Extra.
Agrét
ExIra
Agreb
Extra
Number-L3-inc
NumberL5-inc
NumberL3-Dec
Number-L5-Det
Text-13-nnx
Text-LS-Inc
CelebPersona
AthlePersona
Statadd Deviation
Lome Morranc
Demin P. CIRRSPT Latek
Movero
CHAIO Uitest
Ore Seda
Owerd Max
Ang.
Gemini 2.5
1.13
0.72
0.72
0.76
1,65
1.75
1,56
1.25
1.17
1.22
1.45
Cha
1.86
1.92
2.49
Qwee2.5
2.52
Avg.
1.29
Number-L3-
Numbeer-L5
Number-L3 L3-eer-----Du 3-Dec
Number-L5-Doc
Text-L3-Inc
Text-LS-Inc
including Arena score and API pricing, are given in Tab, A5 Models were assessed eight criteria, including generation time, missing and indecisive rates, privacy preservation, output formatting, factual accuracy, and context consistency. See App. A4.1 full metric definitions and model details.
Prompts were carefully designed to balance interpretability and consistency. We experimented with numeric vs. textual outputs (e.g.. {1.2.3} vs. {disagree, neutral, agree}), 3-level vs. 5-level scoring scales (e.g., {1,2,3) vs. (1,2,3,4.5)), and different ordering directions (e.g., [1,2,3] vs. {3,2,1)) running controlled trials across all candidate models. Results showed that 3-level scales with numeric outputs minimized variability, whereas 5-level scales increased inconsistency. A detailed comparison of scoring scales and complete prompts provided Prompt 3 See App. A4.2 for more details. Based these evaluations, selected three consistently best pest-performing LLMs (i.e., ChatGPT-4o- Latest, Gemini-2, P.o-Pro. and Maama-4-Maveriche to generate both textual descriptions and Big Five scores for building datasets. Detailed results and analysis of the experiments in App. A4.2
2.2 ETHICAL CONSIDERATIONS: CONSENT, PRIVACY, BIAS, AND USAGE
We emphasize four aspects to address ethical and technical concerns: (i) Consent and legality: both datasets are derived entirely from legally accessible, consent- resources, including official sports league websites (non-commercial use), CelebA (non-commercial use), and WikiData (free license). (ii) Privacy protection; no raw images or trait texts are released. Each facial image is replaced with a 1024-dimensional embedding, and each textual description with a dine-simmlli embedding, both further obfuscated through an invertible transformation. Categorical variables are converted into indices. (iii) Bias: AthlePerson. currently includes only male athletes, while Ce mellepersse focuses wealthy, high-visibility individuals. Although findings should be
4

---

Published as a conference paper at ICLR 2026
Figure 3: Independence test (IT) results and distributions of trait scores. (a) and (b) present heatmaps of significant IT results between Big Five behavior traits and other structured features for CelebPersona and Athles atchlePersee Pereesona, respectively. Each cell reports "z/y." where is the number of methods that reject the null hypothesis (p < 0.05) and y is the total number of applied methods. Lighter shades indicate stronger evidence of dependence. (C) shows the overall distribution of Big Five behavior scores across both datasets. Refer to Tab. A3 and Tab. A4 for complete y- y-values.
Open.
Cors
Extra
Agres.
Neuro.
Open
Cond
Agres.
Birth Year-
5/5
5/5
5/5
5/5
Birto Year
5/5
5/5
5/5
Birth
05
9/3
W/3
Birth, Manch-
0/5
0/5
2/5
4/5
05
Brth Day-
0/5
0/5
0/5
05
1/
Bithh Day-
8/5
0/5
0/5
8/5
375
Leaque-
5/5
5/5
5/5
5/5
55
Gender-
5/5
5/5
Latitude
3/3
3/3
3/3
213
30
Occupation
5/5
515
Longitude-
3/3
3/3
3/3
3/3
Latibue-
3/3
3/3
3/3
2/3
3/3
Height-
1/3
3/3
Longitue-
3/3
M/
Weight-
3/3
3/3
Height-
3/3
313
-ca
(b) Heatmap for AthlePersona
Weight-
3/3
3/3
3/3
1/3
2/3
Nosee
Opwn
Coms
Extra Aurow. Neuro.
Open. Coms. Exta, Agrow, Neuro
Big Lios-
5/5
5/5
3/5
Score
Ova Face-
6/5
5/5
$/5
5/5
3/5
Double_Chin
Pointy
5/5
5/5
515
Narrow Eyes-
6/5
0/5
4/5
915
2/5
High checkeeeksnnsss
Eyebrows.
5/5
5/5
515
Eyebroess
6/5
0/5
4/5
5/5
0.5
Receding Harrine
CelebPersona
AthlePersona
(a) Heatmap for CelebPersona
(c) Distribution of Trait Score
interpreted as population-specific rather than universal, these focused cohorts provide consistency, and the diversity across domains (e.g., different sports leagues) creates valuable opportunities to study invariant causal patterns. (iv) Usage restrictions: a mandatory usage guideline limits the dataset to non-commercial use and prohibits applications in high-stakes contexts (e.g.. insurance lending).
3 ANALYSIS LEVEL I: INFERRING STATISTICAL DEPENDENCE FROM STRUCTURED DATA
We analyze both datasets at two levels, starting with structured tabular data. For each individual, trait scores are derived by prompting three LLMs to generate text descriptions, which are then mapped to Big Five trait scores. To ensure robustness, we remove "0" scores (denoting insufficient information) and aggregate the remaining values using a median-based voting rule that minimizes sensitivity to outliers. For Ce l eeperrona, facial attributes across multiple images are also aggregated into a single stable value per person through majority voting. More implementation details on the voting and aggregation procedures for trait scores and facial attributes are provided in App. A5.I
To examine dependencies between trait scores and other structured features, we apply five different independence test methods: three non-paran parametric approaches (KCI (Zhang et al.) 2012)), RCIT (Strobl al. 2019), HSIC (Gretton et al. 2005)) and two tests designed for discrete variables (Chi-square (Tallarida et al., 1987) and G-square ( sammardins et al., 2006)). The detailed descriptions about these methods are summarized in Tab. A66 A dependency is deemed significant p < 0.05. Fig 3 shows how many of the five methods found significant dependence, and plots the score distributions. Complete p-value results for each independent test method are reported Tab. and Tab. A4
The heatmaps in Fig reveal clear and interpretable dependency patterns across celebrities and athletes. In CelebPersone demographic attributes such as gender and occupation exhibit strong dependence with nearly all trait scores, whereas in Athleeerson with birth year and league affiliation. Physical attributes display divergent effects: celebrities show significant associations between facial features (e.g., pointy nose, arched eyebrows) and trait scores, while athletes exhibit more consistent yet moderate associations with height and weight. Geographic variables (latinudeloongitude) demonstrate comparable moderate dependence in both
stronger dependencies arise
5

---

Published as a conference paper at ICLR 2026
datasets, suggesting stable spatial influences. Taken together, these findings highlight systematic differences in information transfer mechanisms across persona types: celebrity representations are more heavily influenced
more strongly shaped by appearance cues, whereas athlete representations by organizational affiliation. This provides novel insights into the structure of human behavioral traits across varying social contexts. More detailed analyses are provided in App and App. A5.3
ANALYSIS LEVEL II: LEARNING CAUSAL RELATIONS FROM UNSTRUCTURED DATA
Instead of using well-built tabular data, here we aim to directly learn the latent variables and their underly- ing causal mechanisms from unstructured data, such as text and images. This task has been widely studied causal representation learning (CRL) (Xu et al.
Latent Space
F21
20e4nZZheng et al. 2022; Yao et al. 2023; Daun-
E22
hawer et al.] 2023] Sturma al.) 2023; [Sun al.
2025). Our persona datasets inherently contain both multi-measurement and multi- modal information capturing rich observations across diverse formats. Inspired by that, we therefore design a correspond- ing multi-modality multi-measurement CRL method. F shows the causal model of our unique prob- lem setting. Our framework unifies and extends prior work (Yao et al.) 2023 Sun et al. 2025), supported by new identifiability theory specifically tailored to the multi-modality, multi-meeaurement setting. The overall structure of this section is follows: we first formulate the causal model in $ [4. then
Fa.1
Figure 4: Multi-modality multi- measuemmas causal model. Latent space is in grey. is shared latent variables across different modal- ities, z is modalitt-speciiff latent variables. Xm.y denotes the m-th modality i-th observed measurement. is the independent noise term.
establish the identifiability theory results in 4.2 followed by details on network training in & 4.3 synthetic experiments in 4.4 and real-world analysis on the curated PersonaX dataset in $ 4.5
4.1 CAUSAL MODEL FORMULATION
-ata-generating processes. Let x := x XM]] be a set of observations/mmassurennnss from M modalities, where Xm E RR represents the observation from modality m with dimensionality dn- Let [z.....; [zy] be the set of causally related latent variables underlying m-th modalities. Specifically, the data generation process (see Fig. 4 can be formulated as
Zm,i = Izm.: (Pa(Sm,i)s S, S,C,),, (latent causal relations)
Xm == Ix... (Z.p mm))
(generating functions)
where we denote the parents of a variable with Pa(-). Since we allow for general causal relations within each modality and across multiple modalities, Pa(-) potentially returns latent variables across multiple modalities. Additionally, we allow the shared latent variable s generally governing the modality-specift latent variables Zm The differentiable function Ia encodes the latent causal graph connecting latent components and its Jacobian matrix Jaz. can be permuted into strictly triangular matrix. We use Gn.i to denote the exogenous variable for 2m,; and exogenous variables are mutually independent. We use nm to denote -ppelit--specific information independent of other components.
Definition of Identifiability. As mentioned previously, our aim was to learn the latent vari- ables underlying each modality and their causal relations. Formally, for two specifications : {Ixm Iam , p(s), p(.pp(())))))))))( (1e)).p(p. (m))) 1x-1 | and Ô : (0xm: 9rm. femm:(()) p(s), ()))))(m)))) (((m)))))))))))))))))))))))))( ) m== of the data-generating process Eq. [and Eq. that fit the marginal distribution p(x), we would like to show that: given the same x value, each latent component ém, is equivalent to its counterpart zm,5 up to an invertible map hm, i.c., & hmm,) This property is known as identifiability.
'Mulii modality (a.k..,, mulli--iew) refers to different types of data formats, such as facial images and textual descriptions. fulti-mmasurement usually denotes the different instantiations of the same modality, celebrity photos captured at different locations or an individual's 's trait description generated by different L.L.s..
for example,

---

Published as a conference paper at ICLR 2026
Figure 5: Synthetic experiments. (a) The synthetic dataset consists of modalities, colored MNIST and fashion MNIST (LeCun 1998 Xiao et al. (017). (b) The underlying true causal graph shown here. For colored MNIST we generated three different measurements, (c) Experimental results show that our method outperforms the other baselines in terms of both I8 and MCC.
126
6
L
6
Horizontal Position
Vertical Pasition
D/9
6
6
6
6
6
6
6
0.93
6
Lo
6
6
6
6
Image Transparency
Grayscale Intensity
Colored MINIS
Fashlon HNIST
(e) Synthetic Dataset
(b) Latent Variables
(c) Experiment
4.2 IDENTIHIABLITY THEORY
A central challenge in causal representation learning is ensuring that the learned representations correspond to the latent variables up well-defined transformations. We establish theoretical guarantees for the identifiability of our model under specific conditions. The proofs are in App. A6
Theorem (ldentifiability of Subspace) Under the causal model described above, if the estimated observations matches the true joint distribution of any A: Km,cc (they are exchangable) which are three meusurements draw from one modality, and:
(Well-Posed Probability): The joint, marginal, and conditional distributions of (x,,B 2m) are all bounded and continuous.
i (Modality Variability): The operators Lxm.clam and Alx, Alxm are injective.
in (Measurement Changes): For any 2 (222 E Ze where z, + zi: we have p(xm, zi t p(xm.Blzz (z,
iv (Differentiability)): There exists a functional M such that M [pxm, mlmmas Zm, s)] h(zm, 8)
for all zm 2m and S, where is differentiable.
Then we have [2m, S] h1Zzm, s), where h is an invertible and differentiable function.
Discussions. Assumption i is moderate condition that ensures the probability distributions well-defined and computable. Assumption i informally requires that distinct input distributions correspond distinct output distributions. In similar spirit, Assumption iu guarantees that different values of Zn, induce different conditional distributions p(xm, Zm). e.g., heteroskedastic noise. Notably, this condition is significantly weaker than monotonicity. Finally, Assumption iv imposes the differentiability of the mapping from [Z., s) to Px.. R.... which can be explicitly enforced through use of differentiable models, i.e., variational autoencoders (VAEs).
Theorem 2. (Identifiability of Shared Subspace) Suppose assumptions are hold true for all the modality and the whole latent space, and we further assume
(Entropy Regularization): 9-m) represent a set of shared latent variable encoders that minimizes
EnE(M] H (0xl (xx)).
Then we have the & (s),,,, where he an invertible function.
Discussions. After identifying the entire latent space and each latent subspace underlying modality observations: [|z s) ... [ZM, s|}. the shared component
be isolated by leveraging the prelim- inary result that each is block-wise identifiable. This enables the application of existing techniques for isolating shared latent spaces, e.g., developed in (Yao et al. 2023 Von Kügelgen et al. 2021)
7

---

Published as a conference paper at ICLR 2026
Theorem 3. (Component-wise Identifiability) Suppose the assumptions (a lot abuse) in Theorem Theorem are satisfied, suppose we have
i (Sufficient Variability); Denote MMa the number of edges in Markov network Mzm Let
w(m) = 88 p(mm(s)
8 logplzzmls
02m, Asd.
02m Osa.
(3)
(8 log p(Zmmss)
8 log pl 1s)
@ logg(zn(s))
+ 0c,, Oc,, (sS)EE(MM..)
Ozm, 0Ss,
dzm.dm Osa.
where @ denotes concatenation operation and (i,j) E ((M2...) denotes all pairwise indice such that zm..i 3m. are adjacent in Mz.. For m E [I.... , there exist 4n + +M different values of Sa,, such that the 4n + [Mz.. m values of vector independent.
vector functions w(m) are linearly
ii (Sparsity Regularization); Let G E 10,1)dxxd: denote the true adjaceney matrix of the latent causal graph, and G E {0, l]d.xx xd, be the estimated adjacency matrix. assume that the estimated graph is most dense the true graph:
||G||o S ||G||u.
where || llo denotes the elementwise &a norm, i.e. the number of nonzero entries. Then we have am,i his (4 (5))), where h2 is an invertible and differentiable function.
Discussions. The core idea to exploit the rich multi-modal information present in behavior trait datasets disentangle shared latent variables from modality-specine ones. Shared latent factors, e.g.,
genetic traits, act as confounders and induce sufficient variability across This motivates the adoption of nonlinear ICA (Hyvarinen al., (019).. To achieve identifiability, we impose structural constraints derived from the ground-truth Markov network Ma,,, onto the estimated network M leveraging the connection between conditional independence and vanishing
components.
cross-partial derivatives (Lin 1997;: if Zm,i These established conditions shed light on the design of training network below.
{s, \ {=m,i, Em.j) H. then Dem,
log p(am) 0.
4.3
NETWORK TRAINING
Embedding extraction. For images, adopt ImageBind (Girdhar et al. (023) to obtain 1024- dimensional embeddings, leveraging its strength in multimodal representation learning. For text, -tt--wee2-77BB i00-instruch (Bai et al., 2023), foundation model optimized for long- sentence embeddings, yielding 5884-dimensional vectors. Importantly, the released dataset does not contain raw images text. Instead, all images and texts are converted into embeddings and further transformed through an additional invertible transformation, ensuring privacy while preserving utility.
Encoders and decoders. Each modality has its own encoder, which estimates modality-speeifff latents 2m. exogenous variables fm.. and shared latents s. To maintain conditional independence across measurements, decoders reconstruct each observation separately. This reconstruction optimized via mean squared enror: LReson = E.. Σx ||_mmkk -mm,
Independence constraints. To enforce theoretical assumptions, we require independence among latents and exogenous variables. This is implemented by aligning their joint distribution ř with an isotropic Gaussian prior using KL divergence: Lind KL(p(r) [N I)).
Sparsity regularization. Causal relations are captured through a Iearnable adjacency matrix Á.
implemented via normalizing flows (Papamakarios al. 2021 A sparsity penalty encourages minimal yet sufficient causal graphs: Lsp |||||]
Final objective. The overall training loss combines all three components:
L O'Recon LRecon Chnnhttd + CSpp
8

---

Published as a conference paper at ICLR 2026
This framework enforces reconstruction fidelity, independence constraints, and causal sparsity simul- taneously, enabling the recovery of identifiable and interpretable latent variables across modalities. More details about the loss functions and network design are shown in A7
4.4 SYNTHETIC EXPERIMENTS ON VARIANT MNIST
We first evaluate our method on synthetic data derived from MNIST (LeCun 1998), benchmarking against ofate-of-the-art baselines including Beta VAE (Higgins et al. 2017). Multimodal Contrastive Learning (MCL: (Daunhawer et al., 2023)), ) and Multimodal Causal Representation Learning (MM- CRL: (Sun et al. 2025)). We construct two modalities using Colored MNIST (Arjovsky al, 2019) and Fashion MNIST (Xiao al. 2017), where cross-modal causal dependencies are explicitly de-
signed: the horizontal position of digits affects image transparency, which in turn causally influences the vertical placement of fashion items and consequently their grayscale intensity. Fig 5 (a illustrates some generated images, while Fig shows the underlying causal graph. This setup provides a structured yet non-deterministic mapping across modalities. Refer App. A8 for more details.
As shown in Fig. c), our method achieves an Rº of 0.96 and an MCC of 0.92, clearly outperform- ing MMCRL which reaches R² of 0.90 and MCC of 0.85, and also consistently surpassing both BetaVAE and MCL. These results highlight the advantage of explicitly modeling cross-modal causal dependencies with multiple measurements, setting where existing approaches remain constrained.
4.5 REAL-WORLD TRAIT ANALYSIS ON PERSONA
After validating method synthetic data with strong re- sults, we next apply it to the PersonaX datasets, training net- works to learn latent representa- tions and then applying causal discovery to obtain meaning- ful causal graph. Fig. 6 shows the causal graph obtained from AthlePersone while results for CelebFersona are pro- vided in App. A9 The discovered latents naturally group into three categories: shared latents (Sx), image-bases basee latents (Z,,)), and trait-based latents (Z2,a). From AthlePersone, we identify two shared factors (S1, S2), five image-based latents (Z1,-Z11ss. and five trait-based latents (Z2,1 -Z2.s). Importantly, the estimated latents may correlate with, but not necessarily identical to, the well-defined Big Five traits. After obtaining the causal graph, we assign each variable a concrete interpretation, guided by the independence
[S,] o.g. mindset
[Sp] u.g. culture
12 skin tone
[Z ag confidence
12, 21 grooming
2 ag achievament
12, 1 1,3 attractiveness
[2 eg emotional stabiity
[Z og. tocial expressions
[Z ag. self-marrnnas
1 og facial
12 ag. positivity
Figure The causal graph with latent variables learned from AthlePersona dataset. Red, blue, and green nodes corre- spond to shared latents, image latents, and behavior trait latents.
results reported in Tab A7 to facilitate clearer analysis. The undirected edge between S₁ and S2 suggests bidirectional relation between mindset and culture. mindset (S1) influences self-awareness (Z2,1) Moreover, cross-modal links reveal that confidence (Z2,1) affects facial expressions (Z1,4). and emotional stability (Z2.3) impacts grooming (Z1,2). sequential pathway emerges among image-basee based latents: Z1,1 (skin tone) + Z1,3 (attractiveness) + Zi,4 (facial expressions), highlighting appearance factors in athletes.
Together, these findings validate our framework: synthetic experiments confirm identifiability and quantitative performance, while real-world analysis demonstrates its ability to uncover meaningful and interpretable mrosl causal structures in human trait data. See App. A9 for more analysis.
DISCUSSIONS AND CONCLUSION
Discussions. (i) Cohort-specifie scope: AthlePersona currently covers only male athletes, while Ce CelebPersona only includes wealthy and high-visibility celebrities. These cohorts are not univer- sally representative, but they provide controlled populations for analysis across different domains such sports and entertainment. Looking ahead, will expand PersonaX by continuously collecting and incorporating data from additional sources, enabling broader coverage and greater inclusivity over time. Specifically, will introduce version-controlleed update framework with transparent

---

Published as a conference paper at ICLR 2026
changelogs ensure stability, inclusivity, and responsible evolution across dataset versions. (ii) Lack of temporal stabiliry: Behavioral traits are subjective and dynamic, yet our traits are inferred by LLMs from static public data without longitudinal tracking. This complicates validation but points to future work on temporally rich datasets. We will further borrow the merits of recent advances in lightweight temporal modeling and semantic understanding Ye et al. (2025) Zhang et al. (2025), leveraging their advancements to rigorously evaluate and propel progress in this direction. See Ap for more discussions. (iii) Multimodal CRL benchmarking: Beyond supporting multimodal causal representation learning, PersonaX has the potential to serve as dedicated benchmark for multimodal CRL, complementing existing resources such as Causal Chen al. (2025). As future work, we plan to develop standardized benchmark suite built upon PersonaX, featuring unified evaluation metries for identifiability, cross-modal invariance, and causal graph recovery, therehy enabling fair comparison and reproducible evaluation of multimodal CRL methods.
Conclusion. We presented PersonaX, two multimodal datasets linking LLM--infered behavioral traits with facial and biographical information. Our two-level analysis pipeline combines structured dependence tests with unstructured causal representation learning, addressing both theoretical and empirical aspects: theoretically, we propose novel identifiability theory tailored for multimodal, multi-measune meatummeasrrmentt CRL; empirically, we demonstrate population-specifi patterns and interpretable latent structures. These resources provide foundation for studying invariant causal mechanisms of human behavioral traits while promoting diversity, equality, and mutual respect for all human beings.
ACKNOWLEDGEMENTS
We would also like to acknowledge the support from NSF Award No. 2229881, 1, AI Institute for Societal Decision Making (AI-SDM), the National Institutes of Health (NIH) under Contract RO1HL159805, and grants from Quris AI, Florin Court Capital, MBZUAI-WIS Joint Program, and the Al Deira Causal Education project.
REFERENCES
Alp terms of use. http::/://w...pppp..o..oo///aaaa http:///teeemaaaannooooootottttt Accessed:
Aug 2025.
Bundesliga terms of use.
teems-of--sse-ervi..
Accessed: Aug 2025.
terms use. httpo///eeaa/./////l///////////r
Accessed:
Aug 2025.
Legaserica terms of use.
INTERNAT TONAL |ttp:///////////a/a//aaaaaaaaaaa
NNN.HNT TH.A.. ./A/AA..........S 2025.
20COND 220000TTTTTNNR0000TTTTRRRRAAEAEE
Accessed: Aug
Liguel terms of use. httpp:::/iiggaaa.ccc///////ggggg eccesss
Accessed: Aug 2025.
Mlb terms of use. htt.:t/swww...llmm...saaaaannttt
Accessed: Aug 2025.
Nba terms of use. https://www..bessccm///etm111111
Accessed: Aug 2025.
Nf terms of use. https///ww..rll.co//eaa////msmm Accessed: Aug 2025.
Nhl terms of use. httpp///s///waaaaooooooooooooooa
Accessed: Aug
2025.
Pga terms of use. Aug 2025.
http:s//wwwa.atttoorooo/ooo/osor Accessed:
Premierleague terms
use.
http::/ww.premmeeeeaaeeee. cmm.://..eeeaueee.mccmmmoo
terms- -anns-ann--coniiioon Accessed: Aug 2025.
10

---

Published as a conference paper at ICLR 2026
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08777 0877774 2023.
Kartik Ahuja, Amin Mansouri, and Yixin Wang. Multi-domain causal representation learning via 02854
weak distributional invariances, 2023.
Xavier Alameda-Pineda. Jacopo Staiano, Ramanathan Subramanian, Ligia Batrinca, Elisa Ricci, Bruno Lepri, Oswald Lanz, and Nicu Sebe. Salsa: novel dataset for multimodal group behavior analysis. IEEE transactions on pattern analysis and machine intelligence, 388))1777-1170,, 2015,
Anthropic. Introducing claude, 2023.
URL
Lntrooucing-Clauce
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: 0000000893, 2019.
Syeda Asra and D. Shubhangi. Personality trait identification using unconstrained cursive and mood invariant handwritten text. International Journal of Education and Management Engineering, 5: 20-31, 10 2015. doi: 155815/iemee22055.05.1
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang. Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 2023.
Joan-Isaac Biel and Daniel Gatica-Perez. The youtube lens: Crowdsourced personality impressions and audiovisual analysis of vlogs. IEEE Transactions on Multimedia, 15(1):41-55, 2012.
Stephen R Briggs and Jonathan M Cheek. The role of factor analysis in the development and evaluation of personality scales. Journal of personality, 54(1):106-148, 1986.
Cong Cai, Shan Liang, Xuefei Liu, Kang Zhu, Zhengqi Wen, Jianhua Tao, Heng Xie, Jizhou Cui. Yiming Ma, Zhenhua Cheng, et al. Mdpe: A multimodal deception dataset with personality and emotional characteristics. arXiv preprint arXiv:2407. 122744 2024.
Lei Cai and Xiaoqian Liu. Identifying big five personality traits based on facial behavior analysis. Frontiers in Public Health,
2022.
Justin M Carré and Cheryl M McCormick. In your face: facial metrics predict aggressive behaviour in the laboratory and in varsity and professional hockey players. Proceedings of the Royal Society 2008
B: Biological Sciences,
Raymond B. Cattell, Herbert W. Eber, and Maurice M. Tatsuoka. Personality and Mood by Question- naire. Institute for Personality and Ability Testing, 1970.
Fabio Celli, Elia Bruni, and Bruno Lepri. Automatic personality and interaction style recognition from facebook profile pictures. Proceedings of the 22nd ACM International Conference on Multimedia, MM 14, pp. 1101-1104, New York, NY, USA, 2014. Association for Computing Machinery, ISBN 978 450330633, doi: 01114//2647888265497777 URL httph::t///ooo /rg/
10. 868 254977
Guangyi Chen, Yunlong Deng, Peiyuan Zhu, Yan Li, Yifan Shen, Zijian Li, and Kun Zhang. Causal- verse: Benchmarking causal representation learning with configurable high-fidelity simulations. arXiv preprint arXiv:2510. 1XX1:25110.40499 2025.
Mark Chen, Jerry Tworek, Heewoo Jun, al. Evaluating large language models trained on code. In arXiv preprint arXiv:2107.03377 2021.
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, al. Chatbot arena: An open platform for evaluating Ilms by human preference. In Forty-first International Conference Machine Learning, 2024.
David Maxwell Chickering. Optimal structure identification with greedy search. Journal of machine learning research, 30N7v))507-554, 2002.
11

---

Published as a conference paper at ICLR 2026
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, et al. Palm: Scaling language modeling with pathways. In Proceedings of the International Conference on Machine Learning (ICML), 2022.
Seungeun Chung, Chi Yoon Jeong. Jeong Mook Lim, Jiyoun Lim, Kyoung Ju Noh, Gague Kim, and Hyuntae Jeong. Real-world multimodal lifelog dataset for human behavior study. ETRI Journal,
44(3):426-437, 2022.
Deborah Cobb-Clark and Stefanie Schurer. The stability of hig-five personality traits. Economics Letters, 115(1):11-15, 2012.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, et al. Training verifiers to solve math word problems. In arXiv preprint arX:::2110..4110 2021.
John B Conway. A course in functional analysis, volume 96. Springer Science & Business Media, 1994.
James Cussens. Bayesian network learning with cutting planes. In Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, pp. 153-160, 2011.
Imant Daunhawer, Alice Bizeul, Emanuele Palumbo, Alexander Marx, and Julia E Vogt. Identifiability results for multimodal contrastive learning. In The Eleventh International Conference on Learning Representations, 2023.
Kristina M DeNeve and Harris Cooper. The happy personality: meta-analysis of 137 personality traits and subjective well-being. Psychological bulletin, 124(2):197, 1998.
Shizhe Diao, Pengcheng Wang. Yong Lin, Rui Pan, Xiang Liu, and Tong Zhang. Active prompting with thain-of-thought for large language models. arXiv preprint rrXiv:2302.12246, 2023.
Dario Dotti, Mirela Popa, and Stylianos Asteriadis. Behavior and personality analysis in nonso- cial context dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2354 2354-2362, 2018
Xingho Du, Loka Li, Duzhen Zhang, and Song. Memr Memory retrieval via reflective reasoning for I1m agents. arXiv preprint arXiv:2512.202.37 2025.
Nelson Dunford and Jacob T. Schwartz. Linear Operators. John Wiley & Sons, New York, 1971.
Hugo Jair Escalante, Heysem Kaya, Albert Ali Salah, Sergio Escalera, Yagmur Güęlüüürk, Umut Güçlüü Xavier Baró, Isabelle Guyon, Julio CS Jacques Junior, Meysam Madadi, et al. Modeling, recognizing, and explaining apparent personality from videos. IEEE Transactions on Affective Computing, 13(2):89--911, 2020.
Morgane Evin, Antonio Hidalgo-Munot Adolphe James Béquet, Fabien Moreau, Helène Tattegrain, Catherine Berthelon, Alexandra Fort, and Christophe Jallais. Personality trait prediction by machine learning using physiological data and driving behavior. Machine Learning with Applications, 9: 100353, 2022.
Hans J. Eysenck and Sybil B. G. Eysenck. Manual of the Eysenck Personality Questionnaire. Hodder and Stoughton, 1975.
Shunxing Fan, Mingming Gong, and Kun Zhang. On the recoverability of causal relations from temporally aggregated iid data. arXiv preprint arXiv:2406.02199, 2024.
Luciano Floridi and Massimo Chiriatti. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30:681 1994, 2020.
Minghao Fu, Biwei Huang, Zijian Li, Yujia Zheng, Ignavier Ng, Guangyi Chen, Yingyao Hu, and Kun Zhang. Learning general causal structures with hidden dynamic process for climate analysis. 2025.
Nan Gao, Wei Shao, and Flora D Salim. Predicting personality traits from physical activity intensity. Computer, 52(7):47-56, 2019.
12

---

Published as a conference paper at ICLR 2026
Alan S Gerber, Gregory A Huber, David Doherty, and Conor M Dowling. The big five personality traits in the political arena. Annual Review of Political Science, 14(1):265-287,2 2011.
Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space bind them all. In Proceedings of the IEEE/CVF conference computer vision and patlern recognition, pp. 151 1800151990, 2023.
Lewis R. Goldberg. The structure of phenotypic personality traits. American Psychologist, 48(1): 26-34, 1993.
Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. Measuring statistical dependence with hilbert-schmidt norms. International conference on algorithmic learning theory, pp. 63-77. Springer, 2005.
Yagmur Güęlütürk, Umut Güçlü, Xavier Baró, Hugo Jair Escalante, Isabelle Guyon, Sergio Escalera, Marcel A.J. van Gerven, and Rob van Lier. Multimodal first impression analysis with deep residual networks. IEEE Transactions on Affective Computing, 9(3):316 329, 2018. doi: 10.1 10.1109/T 2017.2751469.
Irina Higgins, Loic Matthey, Arka Pal, Christopher P Burgess, Xavier Glorot, Matthew M Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. ICLR (Poster), 2017.
Yingyao Hu and Susanne M Schenmach. Instrumental variable treatment nonclassical measurement error models. Econometrica, 76(1):195-216) 2008.
Biwei Huang, Kun Zhang, Jiji Zhang, Joseph Ramsey, Ruben Sanchez-Romero, Clark Glymour, and Bernhard Schölkopf. Causal discovery from helerogencoosshoostatiinnyy data. Journal of Machine Learning Research, 21(89): 1-53, 2020.
Chin-War Huang, David Krueger, Alexandre Lacoste, and Aaron Courville. Neural autoregressive flows. In International conference on machine learning, pp. 2078 2787-207. PMLR, 2018
Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 859 359. PMLR, 2019.
Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.0m0 040088, 2024.
Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. Personallm: Investigating the ability of large language models to express personality traits. arXiv preprint 20050:2300.02547, 2023.
John Johnson. Units of analysis for the description and explanation of personality. In Handbook of personality psychology, pp. 73-93. Elsevier, 1997.
Alexander Kachur, Evgeny Osin, Denis Davydov, Konstantin Shutilov, and Alexey Novokshonov. Assessing the big five personality traits using real-life static facial images. Scientific Reports, 10 (1):8487, 2020.
Karras, S. Laine, and T. Aila. A style-based generator architecture for generative adversarial net- works. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 4401-4410, 2019.
Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: unifying framework. In International conference on artificial intelligence and statistics, pp. 2207-2217. PMLR, 2020.
Prannay Khosla, Piotr Teterwak, Chen Wang. Aaron Sama,, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 33:1866: 8661-1873, 2020.
13

---

Published as a conference paper at ICLR 2026
Mikko Koivisto and Kismat Sood. Exact bayesian structure discovery in bayesian networks. The Journal of Machine Learning Research, 5:549-573, 2004.
Meera Komarraju, Steven J Karau, Ronald R Schmeck, and Alen Avdic. The big five personality traits, learning styles, and academic achievement. Personality and individual differences, 51(4): 472 477-477, 2011.
M. Kosinski, D. Stillwell, and T. Graepel. Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. American Psychologist, 70(6):543-556, 2015.
Michal Kosinski, David Stillwell, and Thore Graepel. Private traits and attributes are predictable from digital records of human behavior. Proceedings of the national academy of sciences, 110(15): 5802-5805, 2013.
Robin SS Kramer and Robert Ward, Internal facial features are signals of personality and health. Quarterly Journal of Experimental Psychology, (3(11):2273-2287, 2010.
K. Kärkkäinen and J. Joo. Fairface: Face attribute dataset for balanced race, gender, and age for bias
measurement and mitigation. In Proceedings of the IEEE/CVF Winter Conference on Applications of "Computer Vision, pp. 1548-1558, 2021.
Yann LeCun. The mnist database of handwritten digits. http://yann.. lecun. com/exdb/mmist/ 1998.
Loka Li, Ibrahim Aldarmaki, Minghao Fu, Wong Yu Kang, Yunlong Deng, Qiang Huang, Jing Yang, Jin Tian, Guangyi Chen, and Kun Zhang. How effective your rebuttal? identifying causal models from the openreview system. In NeurlPS 2025 Workshop on CauScien: Uncovering Causality in Science.
Loka Li, Zhenhao Chen, Guangyi Chen, Yixuan Zhang, Yusheng Su, Eric Xing, and Kun Zhang. Confidence matters: Revisiting intrinsic self-correction capabilities of large language models. arXiv preprint arXiv:2402. /2.663,..556, 2024a.
Loka Li, Haoyue Dai, Hanin Al Ghothani, Biwei Huang. Jiji Zhang, Shahar Harel, Isaac Bentwich, Guangyi Chen, and Kun Zhang. On causal discovery in the presence of deterministic relations. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024b.
Loka Li, Ignavier Ng, Gongxu Luo, Biwei Huang, Guangyi Chen, Tongliang Liu, Bin Gu, and Kun Zhang. Federated causal discovery from heterogeneous data. arXiv preprint arXiv:2402.1322 13241, 2024c.
Longkang Li and Baoyuan Wu. Learning to accelerate approximate methods for solving integer programming via early fixing. arXiv preprint arXiv:2207.02087 2022.
Longkang Li, Xiaojin Fu, Hui-Ling Zhen, Mingxuan Yuan, Jun Wang. Jiawen Lu, Xialiang Tong, Jia Zeng, and Dirk Schnieders. Bilevel learning for large-scale flexible flow shop scheduling. Computers & Industrial Engineering, 168:108140 (4008810,200 2022.
Longkang Li, Siyuan Liang, Zihao Zhu, Chris Ding, Hongyuan Zha, and Baoyuan Wu. Learning to optimize permutation flow shop scheduling via graph-based imitation learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 20185-201923 2024d.
Zijian Li, Shunxing Fan, Yujia Zheng, Ignavier Ng, Shaoan Xie, Guangyi Chen, Xinshuai Dong, Ruichu Cai, and Kun Zhang. Synergy between sufficient changes and sparse mixing procedure for disentangled representation learning. arXiv preprint arXiv:2503.0063 2025a.
Zijian Li. Minghao Fu, Junxian Huang, Yifan Shen, Ruichu Cai, Yuewen Sun, Guangyi Chen, and Kun Zhang. Towards identifiability of hierarchical temporal causal representation learning. arXiv preprint arXiv:25/0.1888 183 2025b.
Juan Lin. Factorizing multivariate function classes. Advances in neural information processing systems, 10, 1997.
14

---

Published as a conference paper at ICLR 2026
Ziyi Lin, Shijie Geng, Renrui Zhang, Peng Gao, Gerard De Melo, Xiaogang Wang, Jifeng Dai, Yu Qiao, and Hongsheng Li. Frozen clip models are efficient video learners. In European Conference Computer Vision, pp. 388 488-404. Springer, 2022.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
Z. Liu, Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision, pp. 9730-3738, 2015.
Gongxu Luo, Haoyue Dai, Boyang Sun, Loka Li, Biwei Huang, Petar Stojanov, and Kun Zhang. Gene regulatory network inference in the presence of selection bias and latent confounders. arXiv preprint ar XX1v:2501.10124 2025a.
Gongxu Luo, Loka Li, Guangyi Chen, Haoyue Dai, and Kun Zhang. Characterization and learning of causal graphs with latent confounders and post-treatment selection from interventional data. arXiv preprint arXiv:2509.25800 2025b.
Muhammad Arslan Manzoor, Sarah Albarri, Ziting Xian, Zaiqiao Meng, Preslav Nakov, and Shang- song Liang. Multimodality representation learning: A survey on evolution, pretraining and its applications. ACM Transactions on Multimedia Computing. Communications and Applications, 20 (3):1-34, 2023.
Haiyi Mao, Hongfu Liu, Jason Xiaotian Dou, and Panayiotis V Benos. Towards modal causal structure and representation learning. In Machine Learning for Health, pp. 120-140. PMLR, 2022.
Juan Abdon Firanda-Correa, Mojtaba Khomami Abadi, Nicu Sebe, and loannis Patras. Amigos: A dataset for affect, personality and mood research on individuals and groups IEEE transactions on affective computing, 12(2):479-493, 2018.
Gelareh Mohammadi, Alessandro Vinciarelli, and Marcello Mortillaro. The voice of personality: Mapping nonverbal vocal behavior into trait attributions. Proceedings of the 2nd international workshop Social signal processing. ACM, 10 2010. doi: 10. 10.1145/18781 16. 1878123.
Isabel Briggs Myers, Mary H. McCaulley, Naomi L. Quenk, and Allen L. Hammer. MBTI Manual: A guide to the development and use of the Myers-Brigg Type Indicator. Consulting Psychologists Press, Palo Alto, CA, 3rd edition, 1998.
Laura P Naumann, Simine Vazire, Peter Rentfrow, and Samuel D Gosling. Personality judgments based on physical appearance. Personality and social psychology bulletin, 35(12):1661-1671, 2009.
Ignavier Ng, AmirEmad Ghassami, and Kun Zhang. On the role of sparsity and dag constraints for learning linear dags. Advances in Neural Information Processing Systems, 33:17943 7943-17954, 2020.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint arXiv: 0074487777.78 2018
OpenAl. Gpt-4 technical report, 2023. URL
Atsushi Oshio, Kanako Taku, Mari Hirano, and Gul Saeed. Resilience and big five personality traits: A meta-analysis. Personality and individual differences, 127:54-60, 2018.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):11644 2021.
Ashwin Paranjape et al. Art: Automatic reasoning and tool-use for large language models. In International Conference Learning Representations (ICLR), 2023.
G. Park. H.A. Schwartz, J.C. Eichstaedt, M.L. Kern, M. Kosinski, D.J. Stillwell, L.H. Ungar, and M.E. Seligman. Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108(6):334955.. 2015.
15

---

Published as a conference paper at ICLR 2026
Xiaokang Peng, Yake Wei, Andong Deng, Dong Wang. and Di Hu. Balanced multimodal learning via on-the-fly gradient modulation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2022.
James Pennebaker, Martha Francis, and Roger Booth. Linguistic inquiry and word count (liwc). 01 1999.
Heinrich Peters, Moran Cerf, and Sandra C. Matz. Large language models can infer personality from free-form user interactions. arXiv preprint arXiv:2405. 13052, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 1748-8763. PMLR, 2021.
Haocong Rao, Cyril Leung, and Chumyan Miao. Can chatgpt assess human personalities? a general evaluation framework, 2023.
Eric Rawls, Erich Kummerfeld, and Anna Zilverstand. An integrated multimodal model of alcohol use disorder generated by data-driven causal discovery analysis. Communications biology, 4(1): 435, 2021.
Sonia Roccas, Lilach Sagiv, Shalom H Schwartz, and Ariel Knafo. The big five personality factors and personal values. Personality and social psychology bulletin, 28(6):789-801, 2002.
J Peter Rothe. The scientific analysis of personality. Routledge, 2017.
Elena Ryumina, Dmitry Ryumin, Maxim Markitantov, Heysem Kaya, Alexey Karpov, et al. Multi- modal personality traits assessment (mupta) corpus: The impact of spontaneous and read speech. In Proceedings of ISCA International Conference INTERSPEECH, pp. 4049 4053, 2023.
Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Kc, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, (265))662--344 2021.
Gregory Srrapio-Garcia, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Marwa Abdul- hai, Aleksandra Faust, and Maja Matarić. Personality traits in large language models. 2023.
Peter Spirtes and Clark Glymour. An algorithm for fast recovery of sparse causal graphs. Social science computer review, 91):622-2, 1991.
Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, prediction, and search, MIT press, 2001.
Peter L Spirtes, Christopher Meek, and Thomas S Richardson. Causal inference in the presence of latent variables and selection bias. Conference on Uncertainty in Artificial Intelligence, 1995.
Aarohi Srivastava et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. In arXiv preprint brXiv:2206.046133 2022.
Eric V Strobl, Kun Zhang, and Shyam Visweswaran. Approximate kernel-based conditional indepen- dence tests for fast non-parametric causal discovery. Journal of Causal Inference, 7(1):20180017, 2019.
Nils Sturma, Chandler Squires, Mathias Drton, and Caroline Uhler. Unpaired multi-domain causal representation learning. Advances in Neural Information Processing Systems, 36, 2023.
Yuewen Sun, Lingjing Kong, Guangyi Chen, Loka Li, Gongxu Luo, Zijian Li, Yixuan Zhang, Yujia Zheng, Mengyue Yang, Petar Stojanov, et al. Causal representation learning from multimodal biomedical observations. The Thirteenth International Conference on Learning Representations, 2025.
Zeyi Sun, Ye Fang, Tong Wu, Pan Zhang, Yuhang Zang, Shu Kong, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang. Alpha-clip: A clip model focusing on wherever you want. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 13019-13029, 2024.
16

---

Published as a conference paper at ICLR 2026
Ronald Tallarida, Rodney B Murray, Ronald J Tallarida, and Rodney B Murray. Chi Churequrre test. Manual of pharmacologic calculations: with computer programs, pp. 140-142, 1987.
Zeyu Tang, Zhenhao Chen, Loka Li, Xiangchen Song, Yunlong Deng, Yifan Shen, Guangyi Chen, Peter Spirtes, and Kun Zhang. Reflection-window decoding: Text generation with selective refinement. arXiv preprini urXiv:2502.03678, 2025.
Gemini Team, Rohan Anil, Sebastian Borgeaud, lean-Baptises Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, al. Gemini: a family of highly capable multimodal models. arXiv preprint S.t:::2312.11805, 2023.
Marina Tiuleneva, Vadim A. Porvatov, and Carlo Strapparava. Big-five backstage: A dramatic dataset for characters personality traits& & gender analysis. In Michael Zock, Emmanuele Chersoni, Yin Hsu, and Simon de Deyne (eds.), Proceedings of the Workshop on Cognitive Aspects of the Lexicon LREC-COLING 2024, pp. 114-119, Torino, Italia, May 2024, ELRA and ICCL.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.1397 2023.
loannis Tsamardinos, Laura E Brown, and Constantin Aliferis. The max-min hill-climbing bayesian network structure learning algorithm. Machine learning, 65:31-78, 2006.
Yu-Min Tseng, Yu-Chao Huang, Teng- Yun Hsiao, Wei-Lin Chen, Chao-Wee Huang, Yu Meng, and Yun-Nung Chen. Two tales of persona in Ilms: A survey of role-playing and personalization. arXiv preprint (rXii:2406.01171 2024.
Julius Von Kügelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. Advances neural information processing systems, 34:16451-16667, 2021.
Teng Wang, Wenhao Jiang, Zhichao Lu, Feng Zheng, Ran Cheng, Chengguo Yin, and Ping Luo. Vimixer: Unpaired vision-language pre-training via cross-modal cutmix. In International Confer- ence on Machine Learning, pp. 22680-22690. PMLR, 2022.
Xuezhi Wang, Jason Wei, Dale Schuurmans, et al. Self-consistency improves chain of thought reasoning in language models. In arXiv preprint arXiv:2203. 2023.
Yilei Wang, Jiabao Zhao, Deniz S. Ones, Liang He, and Xin Xu. Evaluating the ability of large language models to emulate personality. Scientific Reports, 15(1), Jan 2025. doi: 10.1038/ 1115980024-81009555
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24883 2022.
Kathryn E Wilson and Rodney K Dishman. Personality and physical activity: A systematic review and analsss. Personality and individual differences, 72:230-242. 2015.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: 2017.
Danru Xu, Dingling Yao, Sébastien Lachapelle, Perouz Taslakian, Julius von Kügelgen, Francesco Locatello, and Sara Magliacane. A sparsity principle for partially observable causal representation learning. arXiv preprint arXiv:2403.08335. 2024.
Dingling Yao, Danru Xu, Sébastien Lachapelle, Sara Magliacane, Perouz Taslakian, Georg Martius, Julius von Kügelgen, and Francesco Locatello. Multi-view causal representation learning with partial observability, arXiv preprint arXiv:2311.04056 2023.
Xubing Ye, Yukang Gan, Xiaoke Huang, Yixiao Ge, and Yansong Tang. Voco-Ilama: Towards vision compression with large language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 29836-29844 2025.
17

---

Published as a conference paper at ICLR 2026
Wu Youyou, Michal Kosinski, and David Stillwell. Computer-basee personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences, 112(4)) 1036-1040, 2015. doi: 10. 10.1073/pnas. 1418680112
Yue Yu, Jie Chen, Tian Gao, and Mo Yu. Dag-gnn: Dag structure learning with graph neural networks. In International Conference on Machine Learning, pp. 7154-7163. PMLR, 2019.
Yue Yu, Tian Gao, Naiyu Yin, and Qiang Ji. Dags with curl: An efficient dag structure learning approach. In International Conference on Machine Learning, pp. 12156-12166. PMLR, 2021.
Changhe Yuan, Brandon Malone, and Xiaojian Wu. Learning optimal bayesian networks using a* search. In /wenty--scond international joint conference on artificial intelligence, 2011.
Leslie Zebrowitz. Reading faces: Window to the soul? Routledge, 2018.
Chao Zhang. Zichao Yang. Xiaodong He, and Li Deng. Multimodal intelligence: Representa- tion learning. information fusion, and applications. IEEE Journal of Selected Topics in Signal Processing, 14(3):478-453 493, 2020.
Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, and Xiaojie Jin. Flash-vas vstream: Efficient real-time understanding for long video streams. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 21059-21066 2025.
Kun Zhang, Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Kernel-based conditional independence test and application in causal discovery. arXiv preprint 1rXiv:1202.3775. 2012.
Kun Zhang, Shaoan Xie, Ignavier Ng, and Yujia Zheng. Causal representation learning from multiple distributions: A general setting. arXiv preprint arXiv:2402.05505, 2024.
Lecheng Zheng, Zhengzhang Chen, Jingrui He, and Haifeng Chen. Multi-mode causal structure learning and root cause analysis. arXiv preprint arXiv:2402.02357, 2024.
Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric Xing. Dags with no tears: Continuous optimization for structure learning. Advances in Neural Information Processing Systems, 31, 2018.
Yujia Zheng, Ignavier Ng, and Kun Zhang. On the identifiability of nonlinear ica: Sparsity and beyond. Advances in neural information processing systems, 35: 35:16411-16422 2022.
Klea Ziu, Slavomír Hanzely, Loka Li, Kun Zhang, Martin Takáč, and Dmitry Kamzolov. v'dgg: Projected stochastic approximation iteration for dag structure learning. arXiv preprint arXiv:2410.23862 2024.
Huiqi Zou, Pengda Wang, Zihan Yan, Tianjun Sun, and Ziang Xiao. Can Ilm" self-report"?: Evaluat- ing the validity of self-report scales in measuring personality design in Ilmebased chatbots. arXiv preprint arXiv:2412.0020) 2024.
18

---

Published as a conference paper at ICLR 2026
Appendix for
"PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits" Table of Contents:
A1 Ethics Statements and Broader Impacts
20
Details about Related Work
20
A2.1 Human Behavior Trait Analysis
20
A2.2 Causal Discovery and Causal Representation Learning
................................
A2.3 Multimodality and Representation Learning
22
A2.4 LLM Reasoning and Inference
22
A3 Details about AthlePerson and CelebPersona Datasets
24
A3.1 Full Feature Lists
24
A3.2 Distribution Plots
27
A3.3 Missing Values
28
A3,4 Details about Consent: Terms-ol of of Use Compliance
29
A4 Details about LLM Selection and Prompt Design
29
A4.1 Details about How Select LLMs (regarding Table
................................
29
A4.2 Details about the Impact of Scoring Scale (regarding Figure 2
31
A5 Details about Independent Test (IT) Results
35
A5.1 Details on Voting and Aggregation
35
A5.2 Details about IT Results of AthlePersona
36
A5.3 Details about IT Results of CelebPersona
38
A6 Theorems and Proofs
39
A6. Proof of Theorem
39
A6.2 Proof of Theorem
42
A6.3 Proof of Theorem 3
43
A7 Details about Network Training for Causal Representation Learning
46
Details about Synthetic Experiments on Variant MNIST
47
A8.1 Details about Experimental Setup
47
A8.2 Details about Results and Analysis
47
A9 Details about Real-world Behavior Trait Analysis PersonaX
48
A9.1 Details about Experimental Setup
48
A9.2 Details about Results and Analysis
48
19

---

Published as a conference paper at ICLR 2026
A9.3 Results and Analysis of Baseline MMCRL
49
A1 ETHICS STATEMENTS AND BROADER IMPACTS
Understanding human behavioral traits has broad implications for psychology, human-computer interaction, and AI personalization. By releasing two multimodal, publicly accessible datasets (CelebPersee and AthlePers
together with a two-level causal analysis framework, this work provides the community with a resource to systematically investigate behavioral traits in relation to facial and biographical features, and advance methodological research multimodal CRI..
The ethical considerations and limitations must be acknowledged. We recognize that inferring behavioral traits from public data carries risks of reinforeing stereotypes or enabling misuse in sensitive domains (e.g.. hiring, lending, surveillance). To mitigate these risks, (i) all data is sourced from consent- -baeed legally accessible platforms; (ii) no raw images or texts are released- only transformed embeddings with additional obfuscation; and (iii) enforce strict comerccllllaal usage restrictions, accompanied by a detailed guideline file (USAGE GUIDELINES md). These safeguards are intended to reduce the likelihood of misuse while maintaining research utility.
The current release is demographically limited: AthlePersona covers only male athletes, while Ce SelebPersons. focuses high-visibility celebrities. These choices were intentional: male professional leagues attract broader and more consistent public attention with accessible records, and the celebrity dataset builds upon the established CelebA benchmark. In both cases, deliberately selected cohorts with sufficient public visibility to ensure the availability of high-quality data for reliable LLM inference for behavior traits. As such, results should not be interpreted as universally representative, but rather as population-specite entertainment). We explicitly view broader demographic coverage (e.g., female athletes, less visible public figures, longitudinal data) crucial direction for subsequent dataset extensions.
analyses across complementary domains (sports and
Our overarching goal is to foster understanding of population- patterns, not deterministic inference about individuals. We actively discourage applications in high-stakes decision-making, and instead encourage the community to use PersonaX for methodological development (e.g.. causal discovery under selection bias, multimodal integration) and for examining fairness and robustness across social contexts. We also plan to update the dataset iteratively in response to community feedback, with inclusivity and transparency guiding principles.
Overall, this work aims to advance the scientific study of behavioral traits while foregrounding faireess, privacy, and ethical responsibility. By articulating clear limitations, safeguards, and future commitments, we hope to enable constructive research and minimize risks of harmful deployment.
A2 DETAILS ABOUT RELATED WORK
A2.1 HUMAN BEHAVIOR TRAIT ANALYSIS
Human behavior traits have long been central understanding how individuals reflect on their strengths, limitations, and interpersonal tendencies (Rothe 2017 Johnson 1997; DeNeve & Cooper 1998. Briggs& & Cheek 1986). In contrast to psychological personality, which emphasizes internal dispositions typically assessed through self-reports or expert evaluation, behavior traits focus on outwardly observable patterns inferred from language, facial appearance, physiological states, and digital traces. Traditional self-report instruments, such Cattell's 16PF (Cattell et al.) 1970), the Eysenck Personality Questionnaire (Eysenck & Hysenck] 1975), and the Myers-Briggs Type Indicator (Myers al. 1998), provided carly frameworks for personality assessment. The Big Five (Goldberg) 1993) has since emerged as the prevailing paradigm, supported by strong empirical evidence and predictive power (Cobb-Clark & Schurer, 2012 Oshio et al.) 2018 Komarraju et al. 2011 Roccas et al. 2002 Gerber et al. (011)) However, these methods remain vulnerable to self-report biases and limited scalability.
Computational approaches have sought to infer traits from observable signals rather than introspection. Examples include linguistic cues (Pennebaker al. 1999), handwriting (Asra & Shubhangi, 2015).
20

---

Published as a conference paper at ICLR 2026
Modalitios
Foces
Penomaity (Koinski
2015
Persomality Prodiction
Park
Psychometric Asalyai
MμPTA
Kyonig
Vidao, Audio, Tabular
Persomiliny Prodiction
MDPE
Inner
X
Video, Audio, Tahular
Deception Detection
Amugos
2012
lnntr
X
Vidco, Audio, Senso, Tabular
Affeci, Personality and Mood Prediction
SALSA
/Lumer Tunds 1: 2013
laner
Video, Audio, Sensee, Tabular
Group Behavioe and Personalay
BPAC
[ntt ara 2011
liner
Videa, Tabrltr
Belavioe Undkrstanding and Passoalitt Recogoition
Driving
[ivineral 2055
lnner
Senser. Tabular
Personality Prediction from Driving
Liczlog
Ciing 11 2022
Ouser
Sensoc, Tabular
Keal world Hehavior Study
YouTuh Yirr Dite Chica-Pore 2012
Outer
Video, Audio, Tabular
Persenulity Prodiction
V2
lisallnne
Video, Audio, Text
Fins Impression Recognition
Celebit
[Curers::
Facial Altribute Analysi
Personax (Ours)
Osser
Image. Text. Tabular
Behavior Trait Interpretation Causal Analysis
Table A1: Comparison of multimodal datasets for personality or behavior-trait research. Focus distinguishes between inner personality (psychological, self-reported) and outer behavior traits (observable signals). Score indicates whether personality or trait scores are included. Text Desc. shows whether textual trait descriptions are available. Modalities lists the input data types. Task / Focus describes the primary research application. Unlike prior datasets, PersonaX uniquely combines multimodal signals with both scores and textual descriptions, supporting systematic cross- modal and causal analyses.
speech (Mohammadi et al., (010), facial expressions (Güçlütürk et al. 2018)), and online profiles (Youyou et al. 2015| Celli et al. 2014). A landmark study by (Kosinski et al. 2013) showed that Facebook Likes could predict a wide range of sensitive traits, including personality and demographics, with accuracies comparable to psychometric tests. Beyond digital traces, physiological and behavioral signals have been linked to traits in contexts such as driving (Evin et al. 2022). smart-home daily
activities (Dotti et al., 2018). and long-term lifelogging (Chung et al. [022)) These works demonstrate
the potential of behavioral data for trait inference, often in settings where traditional questionnaires impractical.
Several multimodal datasets have been developed to study traits in richer contexts. SALSA (Alameda Pineda al., (2015) captures group behavior at social events through multimodal recordings with personality annotations. YouTube-Vloo Vlogs (Biel & Gatica-Perez 2012). FI-V2 (Escalante et al. 2020). MuPTA (Ryumina al., 2023). MDPE (Cai et al. 2024), and Amigos Sairanda-Corres
al.)
2018) integrate video, audio, and physiological data for tasks such as impression analysis or affect recognition. While valuable, these datasets are generally small-scal scale and lack explicit textual trait descriptions or unified frameworks for cross-modal analysis. Other resources like CelebA (Liu et al., 2015). FFHQ (Karras al.) 2019), and FairFace (Kärkkäinen & Joo 202T) enable large-scale analysis of facial attributes but do not provide trait personality annotations.
Complementing dataset development, empirical studies have examined how outward features in one modality can signal traits in another. The "kernel of truth" hypothesis (Zebrowitz) (018) suggests that physical cues may reflect behavioral tendencies, supported by findings linking facial morphology to health (Kramer& & Ward [2010), aggression (Carré & McCormick] 2008), and personality judgments from body images (Naumann et al. 2009). More recently, machine learning methods have shown that Big Five traits can be predicted from facial behavior (Cai & Liu 2022 Youyou et al.) 2015 Kachur ct al. 2020). Beyond vision, correlations have also been observed with activity levels (Wilson & Dishman 2015). sensor data (Dotti al. 2018), and physiological signals (Gao et al.) 2019).
Despite these advances, existing resources remain fragmented: many rely on self-reports, others are constrained to controlled laboratory settings, and most lack integration of textual, visual, and biographical information in a unified framework. Our contribution addresses this gap by introducing PersonaX, which provides two multimodal datasets, CelebP ersona and AthlePe oonaa linking facial, physical, and occupational features with LLM--ifferred Big Five behavior traits. This enables systematic exploration of cross-modal relationships and supports both predictive and causal analyses at scale.
A2.2 CAUSAL DISCOVERY AND CAUSAL REPRESENTATION LEARNING
Causal discovery (Spirtes et al. 2001) from observational data has attracted considerable attention in recent decades. Constraint based and score- based methods are two primary categories in causal discovery. Constraint-based methods, such PC (Spirtes & Glymour 1991) and FCI (Spirtes et al.
21

---

Published as a conference paper at ICLR 2026
1995). leverage conditional independence tests (CIT: (Zhang al. 2012 Strobl et al. 2019 Gretton et al. 2005 Tallarida et al. 1987; Tsamardinos et al. 2006)) to estimate the graph skeleton and then determine the orientation. For score-based methods, the approach can vary based on the search strategy, which may involve greedy search, exact search, or continuous optimization. One typical score-based method with greedy search is Greedy Equivalent Search (GES) (Chickering) 2002). The
exact score-based methods are often time-consuming, such as dynamic programming (DP) (Koivisto)
& Sood, 2004), A# (Yuan et al. 2011), and integer programming (Cussens) 2011; Li & Wu 2022).
NOTEARS (Zheng et al. 2018) is the first work to cast the Bayesian network structure learning task into a continuous constramed optimization problem (Li et al. 2022 2024d) with the least squares objective. Subsequent work GOLEM (Ng et al. (020) adopts continuous unconstrained optimization formulation with a likelihood-based basod objective. A line of works have extended NOTEARS to handle nonlinear cases via deep neural networks, such as DAG-GNN (Yu et al. 2019) and DAG-NoCurl (Yu ct al. 2021). Some methods are developed to improve the computational efficiency, e.g. DAG (Ziu et al. 2024). In recently years, there activa rescarches on causal discovery from various data constraints, including distributed data (Li et al.] 2024c), heterogeneous data (Huang et al. 2020). deterministic relations (Li et al. 2024b). latent confounder and selection bias (Luo et al. 2025a
and etc.
Causal representation learning (CRL) aims recover high-level causal variables from low-level observations, bridging machine learning and causal inference (Schölkopf et al. 2021 Li et al. 22025a] Fan et al.) 2024). It generalizes classical causal discovery (Spirtes et al. (200)) by learning structured representations that respect causal semantics. CRL methods with identifiability guarantees typically rely on additional assumptions: (1) Functional constraints on the data- aata-generating process (Xu et al.)
2024; Zheng et al., 2022; (2) Interventional or multi- cnnlliienviromment data that introduce distributional
shifts to expose latent structure (Hyvarinen et al., [201 Khemakhem et al.) (020); (3) Multimodal or multiview settings, where aligned observations across modalities help identify shared causal factors
through sample-level invariance (Yao et al.] 2023 Daunhawer et al.) 2023 Sturma et al., (023).
Recent studies unify these approaches under general invariance principles, showing that many be
seen as special cases of a broader framework (Ahuja et al. (023).
Parallel to these theoretical advances, some works focus on practical CRL applications without strict identifiability. Examples include variational methods for biomedical data (Mao et al. 2022). contrastive learning for multimodal causal analysis (Zheng al. 2024), learning causal insights from OpenReview systems (Li et al.). and causal discovery on neuroimaging datasets (Rawls et al., (021). In contrast, our work aims combine both theory and application: we derive formal identifiability conditions under multi-modality multi-measurement settings and integrate these insights into a practical estimation framework for human trait analysis.
A2.3 MUL.TIMODALITY AND REPRESENTATION LEARNING
In the context of broader machine learning, multimodal representation learning focuses on integrating information from multiple modalities, such as text, images, and audio, to learn unified representations for downstream tasks (Manzoor et al.) 2023 Zhang et al. 2020). Among these methods, contrastive
learning has emerged as a powerful approach, particularly y for weakly supervised settings, due to its
scalability and effectiveness (Daunhawer et al. 2023] Wang et al.] [0022] Peng et al., [202]] Radford
et al. 2021 Khosla al., [020 Oord al. (018). prominent example is CLIP model (Radford
ct al. 2021). ) which aligns text and image embeddings through contrastive objectives (Sun et al. 2024
Lin et al. [022]] Girdhar et al. 2023).
Unlike these methods that primarily aim for discriminative or generative performance, our work is centered on uncovering the underlying causal structure shared across modalities, with the specific goal of generating insights into personalities through principled causal representations.
A2.4 LLM REASONING AND INFERENCE
Large Language Models (LLMs) such GPT-4 OpenAl 2023). PaLM (Chowdhery al. 2022).
Gemini (Team et al. 2023). DeepSeek (Liu et al. 2024), Qwen (Bai et al. 2023), and Claude (An
thropic 2023) have demonstrated remarkable reasoning and inference capabilities across wide range of tasks, including arithmetic (Cobbe et al. 202I), commonsense reasoning (Srivastava et al. 2022).
text editing and generation (Tang al. 2025). code generation (Chen al. 2021). and scientific
22

---

Published as a conference paper at ICLR 2026
Prompt 1. (Complete Prompt for Inferring Behavior Traits in AthlePersona and CelebPersona)
Tark Description Analyze the Big Five behavior traits of the individual described below. Base the onalysis on publicly ovailable information, such as direct quotes from interviews, observed public behavior, documented career patterns, biographical details, Avoid speculation information from unreliable gossip sources. The analysis should reflect the public persona, definitive psychological diagnosis clinical evaluation.
Individual Tnformation
{name)
Gender: {gender)
Description:
{league} player,yrom {country') (AthlePersona) {occupation), h. from {country) (CelebPersona)
Instructions
of five Big Five behavior (Openners, Conscientiouness, Extraversion. Agreeablenesz, Neuroticism):
Analysis: Provide concise (1-2 sentences) analyris. If there sufficient public information, identify specific examples of behaviors, statements, patterns and explain how they relate the defunition of the mais.. there is insufficiet information, stre that clearly.
Score: Assign a score from 0 based on the soale below.
Justification: Provide bl sentence) justification for the directly referencing the evidence mentioned the analysis or the lack thereef.
2. Summary: After analyzing all five tnitts, provide a summary string containing the five scores separated by hyphens.
Anonymity:
explicitly mention the name of the individual the output, prmmouu (Hoffii ShelHer) instend.
Distinguishing Scores:
anatyzing each trait, carefully consider whether information insufficient form meoningful analysis, assign Score If there is sufficient information it leads an indecisive conclusion, dasign Score 2.
U) if
present but indecisive (Score 2). the available information too sparse
5. Striet Formatiin:: Adhere EXACTLY to the "Expected Output Formas" tomplate below, including line breaks, Do not odd any introductory concluding remarks entside this structure.
Scoring Scale
Insufficient information Not enough reliable public information assess the trait. The trait's presenze absence is
unknown unclear due to of data.
Disagree- Clear evidences contradict the trait
2 Neutral Evidence mixed or the trait is not prominent. There is enough information, but it does not strongly support
3= Agree Clear evidences support the trait
Expected Output
Openness:
Anobysis: [Analysis]
Score: [0-3]
Iustification: [Justification]
Conscientionsnes::
Amalysis: [Analysis]
Score: [0-3]
Iustification: [Justification]
Extraversion:
Analysis: [Amalysis]
Score: [0-3]
Instification: [Justification]
A Agreeableness:
Analysis: (Analysis]
Score: [0-3]
Iustification: [Jastification]
Neuroticism:
Analysis: [Analysis]
Score: [0-3]
Iustification: [Justification]
Summary: (Score ScoreC. Scovef. ScoreA ScoreN]
QA (Wang et al. 2023). These models perform zero-shot or few-shot reasoning using techniques
such thhougght prompting (Wei et al. 2022). self-consistency (Wang al. 2023), active
prompting (Diao et al. 2023),, confidence-
(f-or-Else prompting (Li et al. | (022a)). reflective
reasoning (Du et al. 2025) and tool-augmented reasoning (Paranjape al. 2023). Despite their black-box nature, LLMs have shown emergent abilities to perform structured reasoning without explicit supervision, making them powerful general-purposs inference engines.
23

---

Published as a conference paper at ICLR 2026
Table A2: Full Table of Features and Descriptions for AthlePersona.
AthlePersona Dataset
Feature
Type
Description
Missing
Rate (%)
Id
string
Unique identifier for each athlete
0
Height
float32
Height in centimeters
0
Weight
float32
Weight in kilograms
Birthyear
int32
Year of birth
0
Birthmonth
int32
Month of birth
0
Birthday
int32
Day of birth
League
string
Name of the athlete's 's league
0
Latitude
float32
Latitude of country's i central location, trans-
formed from the nationality
Longitude
float32
Longitude of country's central location
0
Chatgpt butput
string
Full trait analysis by ChatGPT encoded in embeddings
Gemini output
string
Full trait analysis by Gemini encoded in embeddings
Lltput
string
Full trait analysis by LLaMA encoded in embeddings
Chatgpt_o Chatgpt_n
int32
Big Five scores (OCEAN) by ChatGPT 0
Gemini o to Gemini
int32
Big Five scores (OCEAN) by Gemini
0
Llama o to Llama n
int32
Big Five scores (OCEAN) by LLaMA
Final_o to Final.n
int32
Final aggregate scores for Big Five traits
0
Image_1
image
First facial image embeddings of the athlete 0
Recent research has explored the extent to which Large Language Models (LLMs) can infer, simulate, and even express human personality traits. For example, (Peters et al. 2024) demonstrate that models such as GPT-4 can estimate Big Five personality dimensions from user- generaedd text with moderate accuracy, even in zero-shot settings. Similarly, (Serapio-Garce et al., 2023) show that LLMs can produce consistent personality profiles when prompted, often aligning with outputs from standardized psychometric assessments. Beyond inference, other studies examine how LLMs naturally exhibit personality-like traits in their responses. (Jiang et al.] (023) introduce methods to control and elicit desired personality traits in language model outputs, while Waan et al. 2025) analyze the emergent ability of LLMs emulate distinct personality patterns during generation. Furthermore, recent works such as (Tiuleneva et al. 2024) Rao et al. 2023) utilize L.MSs to annotate or assess personality traits from textual data, illustrating their growing role in computational personality research.
Our work addresses these limitations by providing multimodal datasets that unite visual, physical, demographic, and personality dimensions, with multiple mener-generated assessments that enable systematic evaluation.
A3
DETAILS ABOUT ATHLEPERSONE AND CELEBPERSONE DATASETS
A.3.1
FULL FEATURE LISTS
The full feature tables of AthlePersona and CelebPersona are displayed Table A2 and A3 The complete final prompt for generating personality is shown in Prompt where the blue text is the highlighted information for each individual. Summary of terms of use compliance for all different sports leagues are in Table A4
The CelebPersona dataset contains structured information about public figures, combining demo- graphic attributes, facial characteristics, and personality assessments. Key features include basic physical attributes (height, weight), birth details (day, month, year), and location information (latitude and longitude, transformed from their nationality). In addition, each celebrity is assigned a categori- cal occupation and gender label. Rich personality data is captured in the form of full-text analyses generated by ChatGPT (ChatGPT-44-latest (2025-03-26).. Gemini (Gemini-2.5. 1omminn--.5-Pro--- -1---5
24

---

Published as a conference paper at ICLR 2026
Table A3: Full Table of Features and Descriptions for CelebPersona.
CelcbPersona Dataset
Feature
Type
Description
Missing
Rate (%
Id
String
Unique identifier for each celebrity
0
Height
float32
Height centimeters
71.5
Weight
float32
Weight in kilograms
87.0
Birthday
int32
Day of birth
2.0
Birthmonth
int32
Month birth
2.0
Birthyear
int32
Year of birth
0.6
Latitude
float32
Latitude of country' central location
0.2
Longitude
float32
Longitude country's central location
0.2
Occupation category:
0 Entertainment & Performing Arts
Music
Sports
3 Media & Film Production
4 Business & Finance
Occupation Num
int32
5 Academia & Science
Hcalthcare
Legal Government
8 Arts & Culture
9 Religion & Service
10 Aviation & Space
11 Other
Gender:
Gender_Num
int32
= Male
0.2
2 Female
Chatgpt output
string
Full trait write-up by ChatGPT encoded em- beddings
Gemini_output
string
Full trait write-up by Gemini encoded in embed- dings
0
Lltput
string
Full trait write-up by LLaMA encoded in em- beddings
Big Five scores (OCEAN) by CharGPT:
Unknown
Chatgpt to Chatgpt n
int32
Disagree
0
2 Neutral
3 Agree
Gemini Gemini_n
int32
Big Five scores (OCEAN) by Gemini
Llama o to Llama_n
int32
Big Five scores (OCEAN) by LLaMA
0
Final_o to Final_n
int32
Final aggregated scores for Big Five traits
0
Binary facial feature:
Absent
Arched_Eycbrood
int32
0 Unknown
Present
Big Nose
int32
Binary facial feature
Pointy Nose
Jointy_Nose
int32
Binary facial feature
0
Bushy Eyebrows
int32
Binary facial feature
0
Big Lips
int32
Binary facial feature
Oval
int32
Binary facial feature
0
Double.Che
int32
Binary facial feature
0
Recoding Hairline
int32
Binary facial feature
Narrow Eyes
int32
Binary facial feature
High Cheekbones
int32
Binary facial feature
Image_ to Image_35
image
Up to 35 facial images embeddings per identity
and LLaMA
(0apericcc0--22EExxxrmeeeeaa) along with their respective Big Five scores (Openness, Conscientiousness, Extraversion, Agrecableness, Neuroticism, OCEAN). We choose these three models because they outperform the other models in various dimensions as shown in our preliminary experiments in Section 2.1 In order represent each celebrity with of OCEAN
25

---

Published as a conference paper at ICLR 2026
Table A4: Summary of terms-of-use compliance for different sports leagues. The column Sports League lists the league under consideration, and Official Website Reference points to the section of the terms-of-use policy from the league's 's official website. Original Statement excerpts the relevant clause directly from the website. Requires Consent? indicates whether explicit written consent is required even for non-commercial academic research use in our case.
Sports Leagues Terms of Use Compliance Original Statement
Sports Leagne
Official Website Refer-
Requires Conseut?
ence
NBA
Terms of Use + 9. NBA
"By using such NBA Statistics, you agree that: (i) any use, display, or
No
STATISTICS (nba)
publication of the NBA Statistics shall include a prominent attribution
to NBm.com in connection with such use, display, publication:
(i) the NBA Statisties may only be used, displayed, or published for
legitimate news reporting or private, nou-commercial purposes.... "You may use the Services solely for your own individual non- No
NFL
Terms and Conditions
+ I. INTRODUCTION;
commercial and informational parposes only, Any other use, including for any commercial purposes, is striedy probibited without express prior wrillen consent."
GENERAL: OWNER-
SIIP; PROHIBITIOSS
MLB
Terms of Use Agreement
you must not reproduce, prepare derivative works based upon. distribuse, perform or display the MLB Digital Propertics without first obtaining the wrillen permission MLB otherwise expressly xet forth in the terms and conditions of the MLB Digital Properties. The MIB Digital Properties must he used any unauthorized manner."
Yes
mib
NHL
Terms of Service + Z. In-
"You may siocess, use, and display the Services, but only for nou- No
tellectual Property (nbi)
commercial, informational, personal use, without modiffcatiion alter-
ation in any way, and only so long as you comply with these Terms."
Premier League
Terms of Use + Intellec-
"You may download and print material from the Website or App as
tual Property Rights pre
is reasonable for your own private and personal use. You may also
forward such material from the Wehsite App to other people for
their private and personal provided you credit its add the Website address."
and
La Liga
Legal Notice and Condi-
The User undertakes to refrain
(a) using Contents
tions of Use + 3. Use of the Website
manner... (h) reproduce or copy, distribute, allow public access through
any form of public communication, adapt, transform or modify the
Contents, unless authorised by the owner the corresponding rights or it is legally permitted..."
Serie General Terms and Con- ditions of the License Agreement + 2. Right limitations 2.2 (ii) Or-
"'Eccett in the case a sepurate writlen agreemeett between Leya Serie A and the Licensee estahlishing otherwise, the Licensee may only exploit the data related the Competitions, the Matches, the Clubs and the players the context...
ficial Dalafleg)
Bundesliga
Terms Use Services
audiovisual content available within Products made avail- able to the User for personal and non-commercial purposes only. The User is authorized to use this audiovisual content only for the purposes of information and entertainment in the privase sphere for themselves and persons personally associated with them (e.g. family members, friends and acqunintances). I Limited to these purposes, the DFI. grants the User non-exclusive, non-transfcrable, access and view audiovisual content within the Products. With the exception of the aforementioned limited right of use, the User
8. Audiovisual Content
bun
con-sub-liccnsable right
is not granted any rights to the audiovisual content."
Ligue
Terms and Conditions of
Any total or partial reproduction of the Site elements without prior written authorization from the publisher (LFP) may lead to logal
Yes
Use + 6. Intellectual
Property (ig
proceedings against infringers."
PGA Tour
Terms of 7. Con-
"You may real time scoring, statisties and other data (whether
No
duck(D) pgn
rent or archival) collected from PGATOUR COM solely for legitimate
news reporting and for personal, non-commercial purposes. You shall not use real time scoring. statistics or other data (whether current or archival) collected from PGATOUR COM for sale, license or other commencial purposes (including, without limitation, commencial gam- bling purposes), unless expressly licensed by the PGA TOUR Partics."
ATP Tour
Terms Coaditions PROHIBITED USES
"AIP owns has right use all of the data, information, text. images, streaming media, video, sounds, icons, scores, rankings, statis- tics, other content contained this Wehsite (the "Content") "), the copyrights and other intellectual property rights therein, unless oth-
A. Ownership arp
crwise noced. You may print one copy of the Content of this Website for your personal, non-commercial use."
personality scores, we aggregate all those three sets of 5-dimensional personality scores generated by 3 LLMS via voting, and we label those aggregated features as "Final". Regarding the facial attributes, manually selected 10 attributes (e.g.. Big Nose, High Cheekbones) from the original 40 attributes in CelebA dataset, those selected features are most likely to present one's inherent property in appearance, and less likely to change over short time than the others (e.g., Heavy Makeup, Wearing
26

---

Published as a conference paper at ICLR 2026
(a) Birth Year
(b) BirthMonth
52
NHL
(c) Birth Day
(d) League
0540
100
77
100
DB mn
10p 109
31--19 11
2e-13 16-154
so the
so
(c) Height
(f) Weight
South America
North America
Oceania
Europe
Atrica
Other
Asia
Nationality
Figure A1: The distributions of the features in AthlePersona Dataset.
Har). These binary facial attributes provide interpretable visual markers. Note that each image has corresponding attribute value and there are multiple images per celebrity, we therefore aggregate all these attributes from different images by majority voting, to obtain the aggregated facial attributes. For each celebrity sample, there are at least two facial images taken from different angles, and up to 35 facial images per sample, referenced via relative file paths.
The AthlePersona dataset focuses on high-profile athletes and contains similar structure to CelebPer- sona, with emphasis athletic context. It captures personal traits such as birth year, month, and day, physical measurements like height and weight, and the name of the athlete's league. Personality descriptions and Big Five scores again generated by ChatGPT, Gemini, and LLaMA, with final aggregated trait scores summarizing the predictions. Unlike CelebPersona, this dataset includes only single facial image per athlete but maintains key demographic and geographic metadata. It omits facial feature annotations and categorical occupation labels, instead reflecting the athletic domain through the league information.
A3.2 DISTRIBUTION PLOTS
The Figure A1 shows the distribution of AthlePersona. The AthlePersona dataset is dominated by athletes born between 1985 and 2005, with a uniform spread across birth months and days. Most individuals are associated with NFL, NHL and NBA. showing strong skew toward U.S. sports. Heights cluster around 180-199 cm, and weights around 90-109 kg, which aligns with physical norms for elite athletes in contact sports. Nationalities overwhelmingly North American, with minimal representation from other continents, highlighting a clear Western and U.S. -entric dataset bias.
The Figure A2 shows the distribution of CelebPersona. The CelebPersona dataset predominantly features younger individuals, with birth years peaking between 1990-1999 and shows balanced
27

---

Published as a conference paper at ICLR 2026
fete 1o
uas non
190
and
Mar reef
(a) Birth Year
(b) BirthMonth
curope
(c) Birth Day
(d) Nationality
Female
Aoseena Conncc
Male
Aittee Spane
CURAI
Walhon
League
Gender
170 T7--10
H4E 140
100
160
1ect
300
214
330
03
144
100-15
190
000
(g) Height
(h) Weight
Festure Vilue
Absort
Uninown
Present (1)
Pointy
Bushy
Eyebrows
Lipa
Ovel
Chubay
Double
Namow Pyax
High
Cheethonnes
(i) Face Attributes
Figure A2: The distributions of the features in CelebPersona Dataset.
distribution across birth months and days. Most individuals are from North America and Europe, with underrepresentation from other continents. There is notable occupational bias toward Entertainment, Music, and Sports, while fields like Healthcare and Academia are sparsely represented. Females slightly outnumber males. Height and weight distributions center around typical adult ranges, though outliers exist. The weight distribution peaks between 60-69 kg and 50-59 kg. with sharp drop after 90 kg. The range 135-139 kg and higher has minimal count. Regarding facial attributes, the majority of features are marked as absent, with smaller subset present, particularly for traits like Oval Face and High Cheekbones. The unknown values are minimal.
A3.3 MISSING VALUES
As shown in the dataset features table (Table A3). the Missing Rate column indicates the pro- portion unavailable or incomplete values for each feature. Despite efforts to retrieve missing information--aatiiullaee
from publicly accessible sources like Wikipedia-certain attributes remain incomplete, especially those considered more private or less frequently disclosed. the CelebPersona dataset, there are total of 9444 data. Height and Weight have the highest number of missing entries, with 71.5% and 87% missing records respectively. Birthday and Birthmonth are missing in 2% entries each, while Birthyear missing in 0.6% cases. Geographic coordinates (Latitude and Longitude)
28

---

Published as a conference paper at ICLR 2026
Table AS: AI Model Arena Scores and API Pricing recorded on April 10 2025.
Model Name
Company
Arena Score
API Price (1/O)
Used by Us
Gemini-2.5-Pro--pp-0-222
Google
1439
$1 $1.25/$ 10.00
yes
Lamaa-4-aaveriikk-0-2226 Experimental
Meta
1417
$5.00/$15.00
yes
ChatGPT-4o-latest (2025-03-26)
OpenAI
1410
$2.50/$ 10.00
yes
Grok-3-Preview-02-24
XAI
1403
$3.00/$
yes
GPT-4.5-Preeiew
OpenAI
1398
$75.00/$ 150.00
no
Google
1380
S0. $0.10/$0.44 40
yes
Geminn-2.00PPro-EXp-222205
Google
1380
$0.10/$0.40
DeepStek-V3-00:24
DeepSeek
1369
$0.07/$1.10
yes
DeepSeek-R1
DeepSeek
1358
SO. 14/12./2.1
yes
Gemmin-2200-Flahh-000
Google
1354
$0. 10/50.40
yes
Qwen2.5-Max
Alibaba
1340
$1.60/$6.40
yes
QwQ-32B
Alibaba
1315
$0.29/$0.39
yes
absent in 0.2% instances, and categorical attributes such as Occupation 0.05% and 0.2% missing values, respectively.
Num Num and Gender Num have
In contrast, the AthlePersona dataset (TabldA2) has been fully cleaned by removing all rows that contain any missing values. Prior to finalization, any entry with incomplete demographic, geographic, or profile information was excluded ensure consistency. As result, AthlePersona contains no missing values, making it readily usable for downstream analysis without requiring additional preprocessing or imputation.
A3.4 DETAILS ABOUT CONSENT: TERMS-OF-US COMPLIANCE
summary of terms-of-use compliance for the different sports leagues is provided in Tab. A4 In addition, we include below the core consent statements for the CelebA dataset and for WikiData.
CelebA: (i) The CelebA dataset is available for non-commercial research purposes only. (ii) You agree not reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purposes, any portion of the images and any portion of derived data. (iii) You agree not to further copy, publish distribute any portion of the CelebA dataset. Except, for internal use at a single site within the same organization it is allowed to make copies of the dataset. (iv) The face identities released upon request for research purposes only. Please contact us for details.
Wikidata: Wikidata offers a wide range general data about everything under the sun, that data is licensed CCO, "No rights reserved", for the public domain.
A4 DETAILS ABOUT LLM SELECTION AND PROMPT DESIGN
Throughout this paper, rely large language models (LLMM)) generate human personality. Fortunately, Benefiting from the recent explosion in the size and availability of LLMs (Achiam et al.
2023 Floridi Chiriatti 2020: Jiang et al. 2024 Liu al. 2024 Bai et al. 2023). some research has shown that personality measurements in the outputs of some LLMs under specific prompting configurations are valid and reliable (Serapio- Gerapio-Gareia Gareia et al. 2023 Jiang et al. 2023 Tseng al..
2024). We built our datasets from number of professional athletes and celebrities, based on the facts that they are famous and likely to have sufficient information about them online.
we present some experiments to show how to select L.MMs for personality generation, and also how to design the prompts. In the following, we will demonstrate more details.
In Section 2.1
A4.1 DETAILS ABOUT HOw TO SELECT LLMS (REGARDING TABLE
Model Choices in Table 1 We present the comparative evaluations on 10 of the sate-of-the-arr LLMs on our two persona datasets. The 10 LLMs include: 2.5-Pro-EPPPPPaae- Llama-4- Maverick-00-22--xxpeeeie ChatGPT-4o-latest (2025-03-26),
Gemini-2.0-
29

---

Published as a conference paper at ICLR 2026
Thinking-Exxpi
DeepSeek- V3-0324, DeepSeek-RI, Gemini-2..0-Flasse
Qwen2.5-
Max. and QwQ-32B. Initially, we choose the Top 10 models based on the Arena leaderboard (Chiang
et al. 2024). To enhance diversity, also included Qwen2.5-Max and QwQ-32B (Bai et al. 2023
from Alibaba, both noted for their strong reasoning capabilities. We list all those 12 LLMSs and
summarize them in Table A5 including the model name, the company name, the arena score, the API input and output price per million tokens, and whether it is used by us for further analysis in Table Specifically, GPT-4.5-Preview (Achiam et al. (023) was excluded due to prohibitively high API costs where the input and output API prices were S75 and $150 per million tokens, respectively. Gemini-2.0-Pro--E Pro--Exp-22-0- (Team et al., (023) was omitted due to inaccessibility, it was merged to the latest Gemini-2.5-Pan Pro--rrr- model. Therefore, in the end, we only considered 10 LLMs, as shown in Table
Evaluation Metries in Table 1. We list evaluation metrics in the experimental results. For each dataset, we randomly sampled T00 individuals, conducted 100 LLM queries in total, and reported the average results. The query prompt is almost the same as the Prompt except that here we considered 5-level (i.e., strong disagree, disagree, neutral, agree, strongly agree) for scoring scale instead of 3-level. As for each evaluation metric, here are detailed explanations:
Generation Time (GT) measures the computational efficiency of each large language model by recording the average inference time required to produce responses. This metric is quantified in seconds and provides insight into the practical usability of different models, with lower values indicating faster processing speeds.
Missing Rate (MR) quantifies the frequency at which language models fail to provide the requested scoring output due to limitations in their knowledge base. This metric is calculated as the percentage of instances where the model cannot generate a proper response (with output score 0- Unknown), highlighting gaps in the model's capability to handle certain types of queries or domains.
Indecisive Rate (IR) captures the proportion of responses where models express uncertainty or provide neutral answers rather than definitive judgments (with output score 2 - Neutral). This metric reflects the model's confidence level and willingness to make clear assessments, with higher rates indicating more cautious or uncertain behavior.
Privacy Preservation (PP) evaluates the model's ability to protect individual identities by effectively anonymizing personal information in its responses. This metric assesses how well the model handles sensitive data and maintains privacy standards while still providing meaningful analysis. For each response, if there contains any individual name information, return 0, otherwise return 1.
Output Formatting (OF) measures adherence to specified response structure and format requirements. This metric evaluates whether the model consistently follows given instructions regarding how responses should be organized and presented, ensuring usability and consistency. For each response, if it absolutely follows the given instructions and the output template format, return otherwise return O.
Context Consistency (CC) assesses the internal coherence between different components of the model's response, specifically examining aligmment between the analysis, assigned score, and provided justification.
Factual Accuracy (FA) measures the absence of factual errors in the model's output, evaluated through cross-validation using mutual critique between different language models. This metric is crucial for determining the reliability and trustworthiness of the generated content.
Note that both CC and FA metrics were evaluated through a generator-evaluato manner by 4 different evaluator LLMs --emini-2.5-Prr-Expp----- 40- Hatest(2025-055 20225-03-26), and Srrok33-Preview-02-222 to ensure logical consistency within responses. Basically, we collect the generated trait analysis output by 10 generator LLMss and feed into other 4 evaluator LLMs. The evaluator LLMs will return (indicating No) or I (indicating Yes). The evaluation prompt is presented in Prompt 2 There are mainly two reasons why we do not use human evaluators but instead choosing LLM evaluators: (1) First, human evaluator is expensive and costly; (2) Second, except loyal fans, most people may not have in in-depth understanding about celebrity or athlete. To that end, LLMS probably have seen more information about certain celebrity or athlete than normal human in general. Therefore, for factual accuracy evaluation, it is reasonable
2-xxerrmiai ChatGPT-
30

---

Published as a conference paper at ICLR 2026
Prompt 2.
(Evaluation Prompt for Context Consistency and Factual Accuracy)
Evaluation Task
You are an expert evaluator for behavior trait analysis results generated by LLMs. You will analyze the following output and evaluate it two specific criteria.
Input Information
Name: {name}
Gender: {gender}
Description: {league) player, from {country) (AthlePersona) {occupation). from {country) (CelehPersona)
Model Output: (behavior trait analysis generated by LLMs}
Evaluation Criteria
1. Context Consistency [0/1]
Check if each of the Big Five trait analyses consistent with the assigned score (0-5)
Check if the justification for each score aligns with the analysis
Score I if all analyses are internally consistent with their scores and justifications
Score 0 if any inconsistencies exist (e.g,, describing high extraversion traits but giving a score 2)
2. Factual Accuracy Assessment [0/1]
Check if the analysis have clear factual errors or highly speculative claims presented as facts
Check if the claims about the celebrity's behaviors, caveer patterns, or public statements appear generally accurate based on common knowledge
Score 1 if the claims appear generally accurate or the model does not make any claims due to insufficient information
Score 0 if there are clear factual errors or highly speculative claims presented as facts
Required Output Format
Context consistency: [0/1] [Justifffcic)
Factual accuracy: [0/1] [Justification]
Summary: [Scorel-Score2]
to use LLM evaluators. Note that in this way, we aim to point out any statement which absolutely violates the factuality or commonsense. As for evaluating context consistency, it turns out be text interpretation task, it is also reasonable to apply LLMs.
Overall Score (OS) provides a comprehensive performance measure by calculating the average of all evaluation metrics except Generation Time. The score calculation is:
OS 1 x + MR) + -R)).
6
offers a holistic view of each model's capabilities across the various assessment dimensions.
presents comparative evaluation of 10 LLMs. ChatGPT-40-Latest (Achiam et al. 2023) and Gemini-2.5-Pro (Team et al. 2023) achieved the highest overall scores. Performance is consistently stronger CelebPers ona than on AthlePersons indicating that assessing athlete personalities is more challenging. This is particularly reflected in the higher MR on AthlePers eereeeernaa.. which is possibly due to the limited public information available for younger less prominent athletes. While GT varies substantially across models, both PP and OF are consistently strong. IR differs notably, e.g., 0.46 for Qwen-Plus (Bai et al. 2023) while for DeepSeek-R1 (Liu al. 2024). suggesting significant variation in models' confidence calibration.
Analysis. Table T
A4.2 DETAILS ABOUT THE IMPACT OF SCORING SCALE (REGARDING FIGURE
Prior research has demonstrated the importance prompt engineering strategies in enhancing LLM performance various tasks (Serapio-Ciarcía al. 2023 Jiang et al. 2023 Tseng al. 2024
31

---

Published as a conference paper at ICLR 2026
Prompt 3. (Comparison on Scoring Scale for Different Prompts)
NNumber-L3-Inge
0 Insufficient information - Not enough reliable public information to axsess the trait. The trait '
presence or absence is unknown or unclear due to lack of data.
1 Disagree Clear evidences contradict the trait
2 = Neutral - Evidence is mixed or the trait is not prominent. There is enough information, but it
does not strongly support or contradict the trait.
3 Agree Clear evidences support the trait
"Number-133-Dee
Insufficient information Not enough reliable public information assess the trait. The trait's s
presence or absence is unknown or unclear due to lack of data.
Disagree Clear evidences contradict the trait
2 Neutral - Evidence is mixed or the trait is not prominent. There is enough information, but it
does strongly support or contradict the trait.
1 Agree Clear evidences support the trait
[Text-
Insufficient information Not enough reliable public information to assess the trait. The trait's 's
presence or absence is unknown or unclear due to lack of data.
Disagree Clear evidences contradict the trait
Neutral Evidence is mixed or the trait is not prominent. There is enough information, it does
not strongly support or contradict the truit.
Agree - Clear evidences support the trait
NNmmber-L5-in
Insufficient information - Not enough reliable public information assess the trait. The trait's 's
presence or absence is unknown unclear due lack of data.
Strongly Disagree Clear evidences contradict the trait
2 Disagree Some evidences contradict the trait
3 = Neutral- - Evidence is mixed or the trait is not prominent. There is enough information, but it
does not strongly support or contradict the trait.
4= Agree Some evidences support the trait
5 Strongly Agree Clear. consistent evidences support the trait
[Number-L5-Dee)
Insufficient information Not enough reliable public information assess the trait. The trait's s
presence or absence is unknown unclear due lack of data.
Strongly Disagree- Clear evidences contradict the trait
4 Disagree Some evidences contradict the trait
3 Neutral - Evidence is mixed or the trait is not prominent. There is enough information, but it
does strongly support contradict the trait.
2 Agree Some evidences support the trait
= Strongly Agree Clear, consistent evidences support the trait
[Text-1.5-Ine]]
Insufficient information Not enough reliable public information the trait. The trait 's
presence or absence is unknown or unclear due lack of data.
Strongly Disagree Clear evidences contradict the trait
Disagree Some evidences contradict the trait
Neutral Evidence is mixed or the trait is prominent. There is enough information, but it does
strongly support or contradict the trait.
Agree Some evidences support the truil
Strongly Agree Clear, consistent evidences support the taitt
32

---

Published as a conference paper at ICLR 2026
Wang al. 2023 Li al. 2024a Tang al. 2025). These foundational studies have established that prompt structure and presentation significantly influence model outputs, particularly in psychological assessment applications. Building on this foundation, we systematically investigate how variations in scoring scale format affect the consistency of L.MM-generaed trait assessments across the Big Five traits, with implications for reliable automated psychological evaluation.
How to Design Prompts? As shown in Fig. 2 the radar and box plots in the top and middle illustrate the extent of intra-prompt variability across the Big Five traits, while the bottom panel reports Manhattan distances between prompts to capture inter-prompt differences. Across both
CelebPersona and AthlePersoni datasets, Llama-4-Maverick (Touvron et al. 2023) stands out for its highly stable outputs, followed by Gemini-2.. 1emiimi-2.5-PP- P.o- (Team et al. (023). In contrast, Qwen2.5-Max (Bai et al. 2023) tends to produce the most variable results. Among these prompts, the "Number-L.3-Ind format consistently yields the lowest variance, suggesting that coarse, numerically formatted 3-point scales help LLMS produce more deterministic responses. Conversely, more complex prompts, especially those using Level-5 textual scales, lead to noticeably higher variability. Taken together, these findings suggest that prompt design, particularly scale granularity and formatting, plays a critical role in shaping the reliability of LLM-based trait assessment.
Experimental Design and Methodology in Figure 2). We evaluated five top- pe-pefforming LLMs using structured prompt format (Number/ Text { [L3/L5] [Inc/Dec], where elements specify response type (numerical vs. textual), scale granularity (3-1evel vs. 5-level), and ordering (increasing vs. decreasing). We list all different scoring scale in different prompts in Prompt
This systematic approach enables comprehensive analysis of how different formatting choices interact to influence model behavior. Each model was tested across 100 trials per prompt format on both CelebPersona and AthlePersona datasets, with temperature set to 0 to reduce stochastic variability (even though temperuture 0 will still have output variation) and isolate prompt-related effects. Con- sistency was quantified using standard deviation (std) of trait scores across repeated runs, providing direct measures of output stability.
Comprehensive Analysis Framework. Our analysis encompasses three complementary perspectives as shown in Figure 2 (1) Top: trait-specific variability patterns through radar plots, (2) Middle: aggregate consistency measures via box plot distributions, and (3) Bottom: inter-prompt relationship quantification using Manhattan distance matrices. This multi-faceted approach provides both granular insights into individual trait reliability and broader patterns in prompt format effectiveness.
Model Performance Hierarchy and Stability Patterns. The analysis reveals clear performance hierarchy among evaluated models. Llave---Maverick demonstrates exceptional consistency with standard deviations consistently below 0.2 across all prompt formats and behavior traits, forming tight, regular polygons in radar plots that indicate robust internal mechanisms for maintaining consistent assessments. The model's 's box plots show minimal variability between prompt formats with few outliers, suggesting sophisticated handling of diverse input structures.
Gemini-2.5-Pro occupies an intermediate position with generally low variability but occasional sensitivity to specific prompt formats, evidenced by longer box plot whiskers and more distributed quartiles. The model shows particular stability with numerical formats while demonstrating increased variance with textual scales, indicating format-dependent reliability patterns. ChaiGPT-4o-Latest exhibits moderate consistency overall but with notable prompt-dependent variations, particularly visible through outliers in box plot distributions. While generally reliable, certain prompt-model- combinations produce unexpectedly high variability. suggesting sensitivity to specific formatting choices. Grok-3- shows concerning instability, particularly in AthlePersona where some prompt formats yield standard deviations exceeding 0.8. Wide interquartile ranges indicate dramatic con- sistency variations depending on prompt format, with pronounced radar plot irregularities revealing trait-specific vulnerabilities. Qwen2.5-MB. consistently ranks as the least reliable model, exhibit- ing high median standard deviations and extensive outliers reaching above 1.0. The model's 's radar plots often show expanded, irregular shapes indicating inconsistent performance across traits, with Manhattan distances exceeding 2.0 for complex formats.
Trait-Specific Consistency Patterns. The radar plot analysis reveals compelling trait-specific reliability patterns. Openness emerges as the most stable trait across nearly all models and prompt formats, consistently showing standard deviations below 0.3. This stability suggests that LLMs
33

---

Published as a conference paper at ICLR 2026
demonstrate inherent consistency when evaluating creative and intellectual characteristics, possibly due to clearer linguistic markers for openness- related traits in training data.
Neuroticism presents notable dataset dependency, showing moderate stability in CelebPersona but considerably higher variability in AthlePersona, particularly for less stable models where standard de- viations exceed 1.0. This context-dependent pattern indicates that evaluation domain significantly influences how models interpret emotional stability markers. Extraversion and Agreeableness exhibit intermediate variability levels with distinct model-specific patterns. The geometric shapes formed by different prompt formats in radar plots reveal systematic differences: simpler formats tend to create smaller. more regular polygons, while complex textual formats often produce irregular, expanded shapes indicating inconsistent cross-trait performance.
Format Optimization and Complexity Trade-offs. The Number-L3-Ine format consistently yields the lowest variance across models and datasets, demonstrating that simple numerical 3-1evel scales en- hance deterministic responses. Box plot analyses show this format produces the tightest distributions with minimal outliers across all models. Manhattan distance matrices reveal that Number-L3- Inc and Number-L3-Dee formats show consistently low inter-prompt distances (often below 1.0), indicating that scale direction has minimal impact when using simple numerical formats.
Conversely, textual 5-level formats (Text-L.--nn/DDee)) produce significantly higher variability, with standard deviations often exceeding 0.5 and Manhattan distances reaching above 2.0 between prompt pairs. This indicates that textual formats not only increase intra-prompt variability but fundamen- tally alter response distributions compared to numerical approaches. The increased granularity of 5-level scales appears to introduce additional decision boundaries that models interpret inconsistently. Number-L5 formats show intermediate complexity, exhibiting distances that fall between L3 numeri- cal formats and textual formats. This suggests that 5-level scales represent a transitional complexity level-more more challenging than 3-level scales but not as fundamentally different as textual ones.
Cross- Dosss-Dtasst Insights and Domain Effects. Systematic comparison between CelebPersona and AthlePersona reveals important domain-dependent patterns. AthlePersona generally produces higher standard deviations and inter-prompt distances across most models, suggesting that athlete trait assessment presents inherent challenges for LLMs. This pattern may reflect training data biases, where celebrity personalities are more extensively documented in text corpora compared to athlete psychological profiles, leading to less robust assessment capabilities in athletic contexts.
Implications and Final Model Selection. These findings challenge conventional assumptions about measurement precision in automated assessment contexts. Counter-intuitively, reducing scale granularity and employing numerical rather than textual formats substantially improves reliability, suggesting that cognitive complexity reduction outweighs precision benefits of more detailed scales. Based our comprehensive analysis across multiple evaluation dimensions, we made the following strategic selections for our trait generation framework: After careful consideration of the consistency patterns, trait-specific reliability, and cross-dataset performance, chose the Number-L3-Ine format as our standardized prompt structure. This format demonstrated the lowest variance across all models and datasets, with standard deviations consistently below 0.3 and minimal inter-prompt distances, ensuring maximum reliability in automated trait assessment.
For model selection, we adopted a multi-model approach incorporating Llavere- ChatGPT-44-LLtest, and Semini-2.5-PPr.. Llama-4-Maverick serves as our primary model due to its exceptional consistency (std) ; 0.2) across all traits and formats. ChatGPT-4o-Latee provides complementary reliability with moderate consistency and broad accessibility, while Gemini-2.5-PPr offers additional validation particularly for numerical format processing. This ensemble approach leverages the strengths of multiple models while mitigating individual model limitations observed in our analysis. Notably, we excluded Grok-3-Bo 3-tta and Qwen2.5-Max from our final selection due to their concerning instability patterns, with standard deviations frequently exceeding 0.8 and inconsistent cross-trait performance that could compromise assessment reliability.
The observed trait-specific and dataset-dependent variations underscore the critical importance of careful prompt design in LLM-bbsse psychological evaluation systems. The convergent evidence across radar plots, box plot distributions, and distance matrices demonstrates that prompt engineering represents a fundamental factor in determining assessment reliability, with implications extending beyond trait evaluation to broader automated psychological assessment applications.
34

---

Published as a conference paper at ICLR 2026
Table A6: Descriptions and suitability of different independence test methods used in the paper.
Test
Full Name
Description
Variable Type
CSQ
Chi-Square Test
A classical test that evaluates whether two categorical variables statisti- cally independent.
Categorical
GSQ
G-Square Test
A likelihood-ratio version of the Chi- Square test, more robust in some small sample cases.
Categorical
RCTT
Randomized Conditional dence Test
Indepen-
A non-parametris method using ran- Continuous/Mixed domized Fourier features to approxi-
mate kernel-based CI testing.
HSIC
Hilbert-Schmidt Independence Crite- rion
A kernel-based method for measur- Continuous/Mixed
ing dependence in high-dimensional data using reproducing kemne Hilbert spaces.
KCI
Kerne--based Conditional Indepen- dence Test
A kerne-bbased extension of HSIC for testing conditional independence, suit- able for complex data.
Continuous/Mixed
A5 DETAILS ABOUT INDEPENDENT TEST (IT) RESULTS
To evaluate independence relationships across different variable types in analysis, we employ five statistical testing methods. For discrete variables, we utilize two classical approaches: the Chi-square test (Tallarida al. 1987). which evaluates statistical independence between categorical variables, and the G-square test ('samardinos et al.) 2006). a likelihood-ratio variant that demonstrates improved robustness in small sample scenarios. For continuous and mixed variable types, we implement three kernel-based methods: the Hilbert-Schmidt Independence Criterion (HSIC) (Gretton et al. 2005), which measures dependence in nigh-diimesiooal data using reproducing kernel Hilbert spaces; the Randomized Conditional Independence Test (RCIT) (Strobl et al.) 2019), a non non-parametric approach that employs randomized Fourier features to approximate kernel conditional independence testing; and the Kernel-based Conditional Independence Test (KCI) (Zhang et al. (012), which extends HSIC methodology for testing conditional independence complex data structures. This comprehensive suite of methods enables robust independence testing across diverse data types encountered in our experimental framework. A dependency deemed significant if p < 0.05, and each cell in Fig. 3 (b)(()) shows the number of methods that detect such significance, and summarize these methods in Table A3 and Table[A]
A5.11 DETAILS ON VOTING AND AGGREGATION
As described in the main paper, trait scores for each individual are obtained from three LLMs, which generate text descriptions that are mapped into Big Five trait scores. These outputs are then combined into a single score per trait using a two-step aggregation procedure. First, we discard any score of "0" (denoting Insufficient Information) to retain only confident assessments. Second, among the remaining values, we take the median, rounding up when necessary. This median- based rule is more robust to outliers than a simple mean.
Example. Suppose three LLMs output scores [2, 3, 0) for Extraversion. After discarding the '0', the remaining scores are [2, 3]. The median is 2.5, which we round up to 3 as the final aggregated trait.
For CelebPersona, each individual is associated with multiple images annotated with binary facial attributes (e.g., Big Nose, High Cheekbones). Since different images may yield different attribute values, we aggregate them by majority voting across all available images. If the votes are unequal, the majority determines the attribute value: -1 for "Absent" and +1 for "Present." In the case of an exact tie (equal votes), assign the value 0, denoting an indeterminate outcome. This process ensures that each celebrity has a consistent, person-level attribute vector, while explicitly flagging ambiguous cases.
35

---

Published as a conference paper at ICLR 2026
Open.
Cons,
Extra.
Agree.
Neuro.
Open.
Cons.
Extra.
Agree.
Birth_Year- Year- 0.000
0.011
0.000
0.000
0.000
0.8
Birth Year- 0.000
Birth_Year-
0.015
0.000
0.000
0.000
96
Birth_ Month 0.300
0.468
0.038
0.001
0.231
Birth Month 0.243
0.540
0.018
0.001
0.250
Birth_ Day 0.547
0.905
0.080
0.846
0.564
Birth_Dar Day 0.376
0.811
0.052
0.667
0:556
-02
League- 0.000
0.000
0.000
0.000
0.000
League
0.000
0.000
0.000
0.000
0.000
-00
-00
(a) CSQ
(b) GSQ
Open.
Cons.
Extra.
Agres.
Neuro.
Open.
Cons.
Extra.
Agree.
Neuro.
Birth, Year 0.000
0.000
0.043
0.000
0.000
Birth, Year 0,000
0.000
0.000
0.000
0.000
Birth_Month- 0.100
0.180
0.191
Birth_Month. 0.067
0.600
0.154
0.002
0.341
Birth 0.475
0.690
0.364
63
Birth 0.634
0.362
0:954
64
League-
League-
Latitude 0.000
0.000
0.000
0.076
Latibude 0.000
0.000
0.000
0.000
61
Longitude- 0.000
0.000
0.000
0.000
0.000
Longitude - 0.000
0.000
0.000
0.000
0.000
Height 0.001
0.059
0.367
0.000
0.079
Height 0.004
0.003
0.042
0.002
0.063
Weight 0.000
0.000
0.000
0.000
0.000
Weight 0.000 0.000
0. 0.000
0.000
0.000
(c) RCIT
(d) HSIC
Open.
Cons.
Extra.
Neuro.
Birth, Year 0.000
0.000
0.000
0.000
0.000
Birth_ Month- 0.063
0.358
0.136
0.006
Birth_ Day 0.703
0.152
Opon.
Cons.
Extra.
Agrez.
Naur.
10
Birch Year-
5/5
League
0.000
0.000
Birth Monch
6/5
0/5
4/5
0/5
Latitude 0.000
0.000
0.000
0.000
Day
6.5
0/5
0/5
1/5
Loogue-
5/5
Longitude- 0.000
0.000
0.000
0.000
0.000
Latiudd
3/3
3/3
2/3
3/3
Height- 0.001
0.141
0.447
0.000
0.031
Longitude
3/3
3/3
3/3
3/3
3/3
Haight-
3/3
Weight 0.000
0.000
0.000
0.000
0.000
Weight-
3/3
3/3
3/3
3/3
(e) KCI
(I) Aggregated IT
Figure A3: AthlePersona: Heatmap of P-value obtained from different independence test.
Overall, this aggregation strategy increases robustness by filtering uncertain outputs, reducing sensitivity outliers, and providing interpretable features at the individual level.
A5.2 DETAILS ABOUT IT RESULTS OF ATHLEPERSONA
Figure A3 presents heatmaps of p-vauues from different statistical independence. The Chi- Chi-Square Test (CSQ) and G-Square Test (GSQ) show remarkably similar patterns, which expected given their shared theoretical foundation for categorical variables. Overall, the independence test analysis reveals limited but significant demographic-trree trait dependencies the AthlePersona dataset. Most relationships show p-values well above the 0.05 significance threshold, indicating statistical independence between demographic features and behavior traits. However, notable exceptions include birth year and league's strong dependence with all big five traits, birth month associations with Agreeableness in the CSQ test (p = 0.001), ) which represents the strongest dependency detected. Birth day shows somewhat clear independence with trait in most methods. The -aaed methods (RCTT, HSIC, KCT) generally
36

---

Published as a conference paper at ICLR 2026
Open. Cons. Extra, Agree. Neuro.
Open. Cons. Extra. Agree, Neuro.
Birth, Year- 0.000 0.162 0.000 0.000 0.000
Birth, Year 0.000 0.966 0.021 0,000
0.000
Dirth_Month 0.2221 0.329 n. 455 0.251
0.546
Birth_Month 0.260 0.259 0.368
0,267 0.542
Birth Day- 0.799 0.963 0.843 0.316 0.707
Birth Birth_Day 0.789 0.921
0.825
0.270 0.679
12
Gender- 0.001 0.000 0.000
0.000
Gender- 0.001
0.000
0.000
Occupation 0.000 0.000 0,000 0.000 0.000
Occupation 0.000 0.000 0,000 0,000
0.000
Big Le__Nose-OO0 0.001 0,030 0,000 0.000 0.000
Big Hig_Nose-DDook 0,014 0.000 0,000 0.000
Big Lips 0.000 0.000 0,000 0.000 0.000
Big Lips 0.001 0.000 0.000 0,000 0.000
Oval_Face Face 0:953 0.002 0,012 0,000 0.002
Oval_Face 0.951 0.001 0.012 0,000 0.001
Double thin-0.005 0.059 0.045 0.002 0.000
Double Chin 0.071 0.040 0.071 0,005 0.000
Pointy_Nose- 0.000 0.006 0.000 0.000 0.000
Pointy_Nose 0.000 0.005 0.000 0,000 0.000
Narrow Eyes 0.173 0:296 0.104 0.146 0.001
Narrow Eyes 0.224 0.408 0.091 0.213 0.004
High Cheekbones- 0.105 0,000 0,000 0.000 0.000
High_ Cheekbones - 0,099 0,000 0,000 0,000 0.000
High
arched_Eyebrowe 0.000 0.604 0,000 0.000 0.000
arched_Eyebrowe 0.000 0.667 0,000
0.000
0.000
Bushy_ Eyebrows 0.870 0.916 0.628
0.003 0.425
Eyyebrow 0.881 0.900 0.794
0,003 0.378
BecedinngHHiiiinne 0.003 0.001 0.001
0,000
HacedinngHairiineee 0.059 0,000 0.002
0.000
(a) CSQ
(h) GSQ
Open, Cons, Extra, Agree, Neuro.
Open, Cons, Extra. Agree, Neuro.
Birth, Birth_ Yoar- 0.000
0.164 0,000 0,000 0,000
Birth, Birth_YYeer- Year- -0.00000000000
0.281 0,000 0.000
0,000
Birth Mirth_ Month-001336
782 0 502 0.584 0.029
dirth_Month -0.248
0.838 0.426 0.459
0.101
Dirtn_Day Day
Day
Gender- 0.000 0.007 0,000 0,000 0.000
Gender- 0.000 0.005
0.000
0,000
0.000
Occupation- 0.000 0,000 0,000 0,000 0.000
Occupation- 0.000 0,099 0,000
0,000
0.000
Latitude
Latitude
Longitude: 0.000 0.000 0,000 0.005 0.103
Longitude- 0.000 0,000 0,000
0,000
0.000
Height- 0.000 0.102 0,000 0.017 0.021 Weight.
Height- 0.000 0.355 0,000
0.015
0.015
0.001 0,005 0.032
Weight
0.077
Big. 0.133 1.000 0,000 0,001 0.000
Big joo_Noxe..o 00044 0.456 0.000
Noo0e.0.044
0.004 0.000
Big Lpp 0.001 0.000 0,000 0.228 0.001
Big_Ups
Bip_Lips Big Lips 0.002 0,000 0,000 0,045 0.000
Oval_Face- 1.000 0.000
0,004 0,000 0.071
Oval_Face 0.911 0.000
0.022 0.000 0.036
Double_ Chin.
Double_Cni Chin
Pointy_ Nose: 0.000 0,000 0. 0.000 0.005 0.067
Pointy_ Nose 0,000 6 0.000 0 0.000 0.015 0. 0.000
Harro__yye 0.675
0.373
0.069 0.146 0.201
Norrow_Eyes 0.247 0.396
0.074 0.256 0.152
High_
55
High
Arched_ Eyebrows. 0.000 1,000
0,000
0.002 0.000
Arched_ Eyebbrowswwws... 0.000 6 741 0.000 0,000 0.000
Receding 583 0.327 0.383
Buchy_Eyebreon Eyebrowe
0.002 0.478
Dushy_Eyebroow Eyebroow 0.551
0.298 0.377 0.018 0.389
0.000
ding Hairline
0.000
(c) RCIT
(d) HSIC
Open, Cons. Extra, Agree Neuro
Open
Cone.
Extra
Arrea
Neum.
Birth, Dirth_Year-- mmar-0.000 0.190 0,000 0,000 0,000
Sirth Year.
5/5
0.5
5/5
5/5
5/5
-1
Birth, Morth-0.171
0.770 0.408 0.439 0.025
Brth, Month
0.5
0.5
25
6/5
2/5
Birth_Day-
0.070 0.177
Bitt.Day
0.5
955
85
0/5
US
Gender_0.00m 0.010 0,000 0,000 0.000
Gender.
5/5
5/5
5/5
5/5
Occupation 0.000 0,000 0,000 0.000 0.000
Occupation.
5/5
5/5
5/5
5/5
5/5
Latitude- 0.000 0.000 0,000 0.067 0.000
Latitude
3/3
3/3
3/
2/5
3/3
Lengitude-
Longltude-
Height. 0.000 0.126 0.000
0.112 0.000
Heigh
3/9
029
313
2/3
3/3
Weight 0.000 0,000 0,000 0.099 0.019
ghg
3/3
3/3
3/3
2/3
Lig_ips-or 0.004 0.000
0.000 0.060 0.000
5/5
5/5
5/5
315
5/5
Oval_Face 0.914 0.000 0,005 0.001 0.056
Ouh Fare
0.5
5.55
5/5
5/5
1/5
Chin-
Pointy Nose- 0.000 0.000 0.000 0.038 0.000
Pose.
5/5
3/5
Narro_Eyes,
www._yyyys--111 0oles-0.112 0, 352 0.065 0.339 0.069
Narrow Eyes:
0.5
0.5
055
0/5
High_
Cheelzoones
Arched Byabrows 0.000 0.702 0.000 0.001 0.000
Archod Eystrowa
5/5
9/5
3/5
3/5
Bushy_Eyee _Eeebrowsswws-. 0.532 0.301 0.364
0.049 0.420
8 Bustyy_yeebrre
0.5
0.5
3/5
615
Receding Baaddinn_Hiiiiiee... 0.096 0.000 0.002
0.027 0.000
Recading,
15
5/5
315
5/5
(e) KCI
(I) Aggregated IT
Figure A4: CelebPersona: Heatmap of P-value obtained from different independence test.
produce lower p-llus,s, indicating stronger evidence for dependencies. Most relationships show p-values between 0-0.01, suggesting statistical dependence among variables such birth year, league,
37

---

Published as a conference paper at ICLR 2026
latitude, longitude, weight, and the Big Five behavior traits. Interestingly, weight is more dependent on openness, agrecableness, and neuroticism, while being more independent of conscientiousness and extraversion.
Trait-specific analysis reveals that most Big Five dimensions operate independently of the measured demographic factors in athletic populations. Agreeableness shows the most consistent evidence of demographic sensitivity, particularly with birth timing variables, though significant relationships (p < 0.05) remain infrequent across methods. Openness, Conscientiousness, Extraversion, and Neuroticism demonstrate predominantly dependent relationships with demographic features, with p- p-values typically samller than 0.05 across most variable- method combinations. Particularly, league affiliation, geographic coordinates (latitude, longitude). and birth year show consistent results, with most methods yielding very low p-values (near 0.000) suggesting dependence, while birth month and birth day produce high p-values indicating independence.
The malti-method validation approach reveals important methodological insights about dependency detection reliability. Classical categorical tests (CSQ, GSQ) occasionally detect marginal associations that kernel-based methods (RCIT, HSIC, KCT) fail to identify, suggesting method-specific sensitivities rather than robust dependencies. The independence test heatmap shows mixed results: some variables like birth month. birth day. and height demonstrate low consensus scores (0-2 out of 5 methods achieving p i 0.05), indicating weak inconsistent dependencies. However, several variable- trait combinations achieve moderate to high consensus scores, primarily involving league, latitude, longitude, and weight. This pattern suggests a nuanced relationship where certain demographic factors (geographic and league-related variables) show more consistent associations with behavior traits in athletic populations than temporal or physical characteristics.
The dependencies between Big Five behavior traits and league, latitude, longitude, and weight in athletic populations likely reflect complex interplay of self-selection, envirommental influences, and sport-specific demands. League affiliations may attract distinct behavior trait profiles- team sports favoring extraversion and agrecableness for collaboration, while individual sports might select for conscientiousness and controlled neuroticism. Geographic variables (latitude/longitude) capture regional cultural differences in values like individualism versus collectivism, as well as environmental factors such as climate that research has linked to behavior trait development. Weight dependencies may emerge through multiple pathways: conscientiousness influencing self-regulation of diet and exercise, neuroticism affecting stress-related eating behaviors, openness driving willingness to experi- ment with training regimens, and sport-speciff body type requirements that indirectly link physical characteristics to the behavior traits favored in those sports. These relationships represent genuine demographic-traii associations rather than statistical noise because they align with theoretically plau- sible mechanisms involving cultural adaptation, environmental pressures, and the mutual influence between behavior traits and lifestyle choices in clite athletic contexts.
A5.33 DETAILS ABOUT IT RESULTS OF CELEBPERSONA
Figure shows heatmaps of p-values from different statistical independence tests evaluating the relationship between facial/demographic features and Big Five personality traits in the CelebPersona dataset. Features like birth year, gender, occupation, latitude, longitude, pointy nose and big lips frequently show strong associations with Big Five traits. In contrast, attributes like birth day, narrow eyes and bushy eyebrows generally appear independent of traits.
The CelebPersona dataset reveals several robust dependency patterns with p-values consistently below 0.05 across multiple methods. Birth timing variables demonstrate the strongest dependencies: birth day shows significant associations with openness, conscientiousness, and extraversion across kernel- based methods, suggesting developmental timing effects on trait formation. Birth month exhibits dependencies with conscientiousness and moderate associations across other traits. Among facial features, big nose demonstrates consistent dependencies with conscientiousness across kernel meth- ods, while bushy eyebrows shows significant associations with openness and extraversion. Weight exhibits notable dependencies with agreeableness and neuroticism, indicating body composition linkages. Narrow eyes shows dependencies with conscientiousness and agreeableness, while oval face demonstrates associations with neuroticism and other traits.
The aggregated IT results confirm these dependencies with higher consensus scores for birth day (3-4/5 methods), bushy eyebrows (3-4/5 methods), and weight (2-3/5 methods), indicating genuine
38

---

Published as a conference paper at ICLR 2026
associations rather than statistical noise. Classical methods (CSQ, GSQ) detect fewer significant relationships, suggesting that non-linear dependency structures dominate celebrity morppoloyyg associations. These findings support evolutionary psychology theories linking facial morphology to behavior traits, particularly the relationship between eyebrow prominence and openness/extraversion and nose characteristics with conscientiousness. The effects of the timing of the birth may reflect sea- sonal developmental influences or cohort effects specific to the career trajectories of the entertainment industry, where certain combinations of behavior trait and timing of the birth provide advantages in celebrity achievement.
The dependency patterns in celebrity populations reveal intriguing Somain-specific insights that distinguish them from general populations. The pronounced birth timing effects, particularly the strong associations between birth day and multiple behavior traits, suggest that developmental timing may interact with entertainment industry selection pressures in unique ways. Celebrities born on certain days may possess behavior configurations that enhance their ability to navigate public scrutiny, media attention, and performance demands. The facial feature dependencies present complex picture of appearance-behavinn relationships: the consistent association between bushy eyebrows and openness/exttaaveriion aligns with research facial masculinity and dominance signaling, while the nose masse-coscientiuuseess relationship may reflect underlying genetic correlations between facial development and self-regulatory capacity. Weight dependencies with agreeableness and neuroticism indicate that body image management, a critical aspect of celebrity careers, may both influence and be influenced by behavior traits related to social harmony and emotional stability. The higher dependency rates detected by kernel methods compared to classical approaches suggest that celebrity behavior- morphology relationships involve complex, non-linear interactions that traditional statistical methods fail to capture, possibly reflecting the multifaceted nature of public persona where appearance, behavior trait, and career success form intricate feedback loops.
A6 THEOREMS AND PROOFS
In this section, we will present more details about the theorems and their proofs. In Theorem 1, We begin by showing how the modality-specific latent subspaces [2m, s], where Zm is modality-specifie latent variables and s is modality-shared latent variables, can be recovered in nonparametric manner using multiple measurements. Building on this result, then in Theorem 2, we demonstrate the identifiability of the shared latent variable s by leveraging the information across multiple modalities. Finally in Theorem 3., conditioned on the recovered s, we establish the identifiability of each modality-specifii latent variable Z, up to minor indeterminacies, i.e., component wise identifiability with an inner-modality permutation. The logical dependencies among the theorems are summarized in the flowchart as shown in Figure A5
A6.1 PROOF OF THEOREM 1
Theorem 1. (Identifiability of Subspace) Under the causal model described above, if the estimated observations matches the true joint distribution of any (Xm, A, Xm,cI (they are exchangable) which are three measurements draw from one modality, and:
(Well-Posed Probability): The joint, marginal, and conditional distributions of (xm.B, Zm) are all bounded and continuous.
t (Modality Variability): The operators Lxm. clam and Lxm. alxm are injective.
ii (Measurement Changes): For any 2 2i (2) E Z, where zi 4 2i we have p(xm. 1 2. X (
p(xm,nale (2)
iv (Differentiability): There exists a functional M such that M [pxm. axm.a Zm, s)] for all Zmm. Zm. and S, where is differentiable.
h h( S)
Then we have [2m, S] h(Zm, 5), where h is an invertible and differentiable function.
Discussion on Insufficient Measurements. Importantly. Theorem I is not limited to the use of multiple measurements within single modality for recovering latent variables. It also reveals that,
39

---

Published as a conference paper at ICLR 2026
Theorem
Theoram
Subspace 4m, +] Identifiaility
Sharad Latent Hdenifabilit
Theorem
Shared
Latent
?
Subepace Identifabilty
Componert-wiie Idenéfabiiity
End
Zm
Figure A5: The high-level flowchart of the our theorems.
when the number of measurements in one modality is insufficient (i.e., fewer than 3), additional modalities can provide complementary information, provided that the required assumptions met.
We first introduce an additional operator to represent pointwise distributional transformations, a concept widely used in the nonparametric identification literature (Hu & Schennach 2008 Fu et al.) 2025 et al.] 2025b). To preserve generality, we denote any two variables by a and b, with A and B, respectively.
corresponding support
Definition 1. (Linear Operator) (Dunford & Schwartz 1971) Consider two random variables a and b with support A and B, the linear operator L-bja is defined as mapping from probubility function Pa in some function space ((A) onto the probability function Pa = Loja o Pa in some function space F(B),
F(A) + F(B): Po Lojo pa= Po
(5) Definition (Diagonal Operator) Consider two randotn vuriable a and b, density functions po and po are defined on some support A and B, respectively. The diagonal operator Doja maps the density function Pa to another density function Db o Pa defined by the pointwise multiplication of the function pbja fixed point b:
[b
Pp = Dsja Pa: where Doja = (bja 1).
(6)
For notational convenience, we define wm = [mm s). with support set Wm-
Proof. The vectors Xm, Xm,B, B, and X conditionally independent given Wms which implies the following two equations:
p(xm,A X, Wm) p(xm,A Wm). p(Xm.C Xm, B: X wm) p(xm,c wm).
(7)
We can directly obtain p(Xm,C Xm.B Xm.A) from the observed quantities p(Xm.A) and p(x P( B, Xm,A). The corresponding density transformation is then given by
p(Xm,O, Xm.B Xm.A)
J p(Xm, X, Wm X Am.
(8)
intogration over W..
Jmm p(Xm,. Xm,B, W., Xm, A) Wm Xmm,))dW..
(9)
Enctorization of joint conditional probubility
p(Xm,. Wm)pp
Xm,
(10)
Jm
by p(xm, of Xm, Xm,, )=pppmpmmmmmmmm (wm)
Jym p(Xm,.
Wm Wm))(XX,,p Win )p(W)) (wm Xm, JddW
(11)
by pixmmalkee www.) (m)
We begin by marginalizing the variable Xm.A using the transformation structure defined in Equation
p(Xm,C, Xm.B
Xm. (on.A)p(xm.AA AdxxAm
Xma.
Jyn / p(xm, wm))pxxm.n Wm)p(wm. Xxm.a)((Xm,. Ad dd
Xmia Wm
(12)
40

---

Published as a conference paper at ICLR 2026
This joint density can be equivalently expressed in terms of the linear operators defined in Definition and Definition
|
[m, _/xm, x p| (xm,c) [I clww Dxm, (w Iow... [x,, A.l (Xm,c).
(13)
Thus, the composed operators satisfy the following identity:
T'xm, Bi Ki..
(Xm,A
Tixm.c W
Wm D Dxm. Wm Tow,, IXm.
(14)
Then, we integrate both sides over the B E Am,B:
Jenn Jem.EE Lxm, Bi Xm,C A,
B EX
(15)
Lxm,C Wm Dxm. Wm Tow m. &mm.A
dxm,...
Xm, nmXm,
Since integrating out B corresponds to marginalizing the joint representation, obtain
Lxx,ellm clm,, Lxxm, o Wm L Xxm,A'
(16)
Assuming that Lxm,cl c w... is injective (see Assumption ii we may invert this operator to obtain
Ixxm, A Lo Amm.""
(17)
Xm, c WW
Substituting Equation (17 into the operator composition Equation
14 yields
Ix B Xmm c Xmm.
Lxxmmc/ w/ Dxm, W/Wm L _
x Wm Ix Xmm
(18)
Multiplying both sides of Equation 18) by Xm, Xmm. gives
Lxx, B (. . Xm,,A Xm, Olx.A Xxm.A
Lix., Ww Dxm, Ww Xx,,ch (wn.
(19)
The right-hand side of Equation 19 takes canonical conjugation form. Under Assumption and by the uniqueness spectral decomposition (see (Conway 1994), Chapter VII, and (Dunford & Schwartz 1971). Theorem XV.4.5), we have
Ix c. (. Dxm, . (Wm. Xm,.G Wwn. (Clxxm clww. P) D Xm. Ww P)(P-1 L-1 Xm.. Ww C²1),
I-1
(20)
where C' is nonzero scalar and P is an invertible operator representing a permutation of the eigenbasis.
This yields identification up to permutation and scaling:
Lixm clw.. CC
Dxm, /w... P- Dx... m/w...
(21)
Equation Equation 21 provides unique spectral decomposition up to permutation and scaling indeterminacies. We next show how these indeterminacies can be resolved, and when they cannot, what informative conclusions may still be drawn.
First, the normalization condition
Jaan
Xmm.c Pxm.c/Wm. dxm,C
(22)
must hold for every wm. Hence, the only solution C' =
Next, consider
Dx Xm, Bi W.. PP1 DD_1D Ww
For fixed X..BB the operator D Dxm. w.,. corresponds to the collection (Px.. a w... (xm.B over all wm. Since P only permutes entries, this collection admits unique solution:
Wm)}
(Dxm. (w.. (xm,1 Wm)) {Pxm. a/w. (Xm,11 Wm)), V Wwm, Wm:
(23)
41

---

Published as a conference paper at ICLR 2026
Because these sets are unordered, consistent matching requires reindexing of the conditioning variables. Specifically,
(Pxm. D Wm (xm.B ( )...)... {Pxm. W (xm,B (i-(()) )
(24)
[Pxm, wm (Xm,1 B w (r((1)), )...] [Psm it /Wom (Xm, B
w (xx(1)) )
),...
(25)
where π denotes a permutation of indices.
Let denote the corresponding relabeling map. Then
Pxm. (m (Xm,B B (wwm)) Pxm. WW (Xm.B Wm),
V Wm Wm-
(26)
By Assumption jii distinct Wm correspond to distinct conditional densities, implying that h one-to-one and invertible. Moreover, Assumption iii ensures that each conditional density uniquely determines its conditioning variable, so that
Pxm, Wa. (Xm.B h ())m) Pxm, Win (Xm,B
Wm)
W
h (w.m
(27)
Finally, Assumption iy implies that h must be differentiable. Since the VAE architecture is differen- tiable, it can learn such a function h. Writing Wm. h(Wm), we have
M [Pxm.e)vi (- wm)] = M [Px... a wm [((()))] = (w.m)
(28)
which equals wm precisely when h is differentiable.
A6.2 PROOF OF THEOREM 2
Theorem D establishes that the modality-specifi latent variables Wm are block-wise identifiable. Given multiple instances of block-wise identifiability for [2m,8) s] across different modalities m, the shared component is expected be identifiable as well. To support this insight, we first present a related lemma from multi-view causal representation learning.
Lemma (Identifiability from Set of Views (Yao et al. (023)). Consider a set of modality obser- vations Xm. that satisfy Assumption 2.I in (Yao et al.) 2023). Suppose there exists a of modality- specific encoders, each mapping to common latent space. Let gxi denote a family of encoders aimed at recovering the shured latent variables by minimizing the total entropy: Eketan H (gxl (xx)). Then, under the stated assumptions, the shared latent variables are block- bonnk-idetii
Theorem 2. (Identifiability of Shared Subspace) Suppose assumptions are hold true for all the modality and the whole latent space, and we further assume
i (Entropy Regularization): 9=.!, represent of shared latent variable encoders that minimizes
Exe(m]
(gxe (xx)).
Then we have the i h, (s), where he is an invertible function.
Proof. We now relate our results Lemma In Yao al. 2023). identifiability is established under the assumption that multiple measurement views are available for shared latent space, and that each measurement process invertible. This setting guarantees block identifiability of the latent space by aligning the outputs of modality-speciff encoders. Specifically, for each modality we have:
2 s) h(zm, s),
(29)
where the key insight is that & corresponds to the shared component across all modality-specific representations Wm. extracted via their respective encoders.
Furthermore, Lemma establishes that any set of encoders minimizing the total entropy
Σ H (ي=g)) (x+))
(30)
KE[M]
recover the ground-trui -ouut shared latent variables from each modality Xm. Xm,, up bijective transformation h,:
$ (s).
(31)
42

---

Published as a conference paper at ICLR 2026
That is, the shared latent content is block-identified from the multi-view observations (xm Ie
Finally, since each podality-speccifee latent variable Zn is causally influenced by the shared component may apply the identifiability conditions in (Von Kügelgen et al., (2221) base case. This allows us to further identify Zm up to a modality-speciife bijection hg:
2m 12 ).
(32)
Hence, both the shared latent component and the modality components 2m block- identifiable.
Discussion. the final step of our proof, we build on the identifiability result from Yao al. 2023), which assumes that multiple invertible measurement processes are available to recover the shared latent variables. contrast, our framework relaxes this assumption by not requiring each measurement process to be invertible. Instead, Theorem T ensures block identifiability of each mopelity-specific latent variable Wy by exploiting the information-sharing structure inherent in multi-modal and multi mulli-mmasurementt settings.
We further leverage structural prior where the shared component s is a common cause of the -peeiit variables, rather than effect. This causal asymmetry eliminates the need for stronger conditions such as global optimization or invariance constraints. Consequently, the conditions in (Von Kügelgen al. 2021) apply, providing identifiability guarantees for the modality-specific latent variables Zm
A6.3 PROOF OF THEOREM 3
We begin by presenting a useful lemma from (Zhang al. 2024 which connects group-wise transformations to component-wise transformations in a Markov network. This lemma is instrumental for the subsequent proof, particular, it enables us to first recover the latent variables within groups of adjacent nodes in the Markov network.
Lemma (Identifiability of Hidden Causal Variables), Ifz, a function of most one of 2g and a and given that and z, are adjacent in Markoy network My. at most one of them is a function of 2 or 2t. Then, there exists permutation of the estimated hidden variables, denoted 2 such that each (4) is function of (a subset of) the variables in {zi) Vz:-
Theorem 3. (Component-wise Identifiability) Suppose the assumptions (a lot abuse) in Theorem 1 Theorem D satisfied, suppose we have
i (Sufficient Variability): Denote MM..
as the number of edges in Markov network My.. Let
((m) =( -(2²
8d log p(zm/s) 02?,, Usa,
Dz Osa, (o
log p(zmlls)
(33)
8 log p(zmlss)
8 log plzm (s)
8 logg(zmlss)
d2m,10sd,
02m.dm O8d.
dee, Oce, (iJ)EE(Mem)
where ④ denotes concatenation operation and (i, E E(Ma..) denotes all pairwise indice such that Zm.i: Zm. are adjacent Mz.. For m E [I.... ,n), there exist 4n [Ma.. different values of Sa,, such that the An + [Mz...l values of vector vector functions w(m) are linearly independent.
(Sparsity Regularization); Let G E 10,1)dxxd. denote the true adjacency matrix of the latent causal graph, and G E {0, 1]d. xd. be the estimated adjacency matrix. assume that the estimated graph is at most as dense as the true graph:
|| < ||G||a;
where || !lo denotes the element-wise Eo norm, i.e., the number of nonzero entries. Then we have 2m,s has (z)z),a(j)) where h2 is an invertible and differentiable function.
Proof. By Theorem
we have
h(2)
Ph(2) Pr:
43

---

Published as a conference paper at ICLR 2026
Let Jn, be the Jacobian matrix of h. The sfauge-of-vvrrable formula implies
p(2[s)] det Jn p(z|s) log p(2]8) = log p(2s) + log | det Jnl
(34)
Suppose 2n and 2 are conditionally independent given 2 ())EE i.e., they are not adjacent in the Markov network over 2. For each by (Lin 1997), have
0 log ((2
0.
(35)
02
To see what implies, we find the first-order derivative of Eq. equation 34
i Olg
12
i log p(z)s) dz i log det J.l
02k
Oz.
02:
02k
Let
r/(s)) p(zls)
n(s)) = logp(z|s),
Oz.
T (s) = 8 log p(z)s)
14,1 = dz
Ozz Oz Oē:
WM
O2,02,
We then derive the second-order derivative w.r.t. 2n and 21 and apply Eq. equation 35
0 p(z(s) dz, oz: Olog(íz(s) Oz.
O2 + 0² log det
Oz.
Oz,dn,
Oh, Oik
oz, Oz, 8 log (2|s))
0 (ogg(z)s)) Ozg Oz.
Σ
E
Ož d2k
O2,0z, Oū, Oz:
i:( (EE((M))
d oogg(z|s)
8z.
8a log det Jol
(36)
òz
02
O2,
82 log det Jol
(s) M
Σ
Σ
(s)
Σ {)(s)) +
O2
j=1
((
(37)
Recall that E(M2) denotes the set of edges in the Markov network over In the equation above, we made of the fact that if z, and zj are not adjacent in the Markov network, then 8 log pimls) by (Lin 1997).
By Assumption consider the 2d, + [M&ll + values of s, i.e., s(") with u 0. 2d, + M22 such that Eq. hold. Then, we have 2d, + [Mal + 1 such equations. Subtracting each equation corresponding to s("), u- I,..., 2d, + M2l with the equation corresponding to s()) results in 2d + M2 equations:
n
n
Σ (nl(s M)
Σ
Σ
(1)((((1)) - (1)
is (x (E)ja.)EE(MM
(1l(s(uu) 1/((s (0)
where u 1,...,2d1 2d + [Mal. Since Ps twice continuously differentiable, we have
(s())) (s (s(w)) n/f1 (s))),
and therefore Eq. equation 38 can be written as
44

---

Published as a conference paper at ICLR 2026
0 Σ -
-
Σ
(ni (s M.
(E(2))
+ Σ (n)(s(()) ())
Consider the vectors formed by collecting the corresponding coefficients in the equation above where 1..... .d, + [M2l.. By Assumption A2, these 2d + [M2 vectors are linearly independent. Thus, for any i and such that {z,, 21} } E E(M2). we have the following equations:
M4, 0,
(38)
K + h 0,
(39)
MM O.
It remains to show h h 0. Suppose by contradiction that
M b/, # O,
(40)
which implies h1,x * 0. By Eq. equation 38 we have h1 O, which, by plugging into Eq. equa- tion 39 indicates h R/11 This is contradiction with Eq. equation 40 Thus, must have h " j. O, which indicates that z, is function of most one of 22 and 2r, and given that z; and 2j adjacent in Markov network Mz, most of them function of êg or 2,
Then, using Lemma 2 we can obtain that there exists a permutation æ of the estimated hidden variables, denoted Zx, such that each (x(s) is function of (a subset of) the variables in {z,} Vz,. is worth noting that in many cases, the above result already enables us to recover some of the hidden variables up to a component-win -ise transformation, that is, ž., (z where h2 is an invertible function.
We next present a proposition that shows how an arbitrary permutation over all components can be resolved into permutation within each modality block.
Proposition 1. (Resolving Block- Wise Permutation) i hlz and 2m Imm (Zm) for any m E [M]. we have 2m,i = h hz(2m,=(5). where h; an invertible function.
Proof. Since the global mapping is given by å = h1(z), where h = [hh1, h2,.... harl acts block-wise on each modality Zm. the Jacobian Jn (z)
02
is block-diagonal:
8z
Jn, (z))
0
(n)
0
Jusa (zns)
This implies that each 2m depends only on Zm.-
Given the global identifiability condition 2.,; = 14(2,, and the fact that both 2.,; and 2, must lie in the same modality due the block dlock-diagonal structure, we conclude:
2m.i = h2 (Zm,7)))
Discussion. We demonstrate that multi-monn molliitmodality information enables the use of the shared con- founder continuous conditional prior over the modality-specifie latent variables Zm This represents the key distinction from conventional multi-modality or multi-view frameworks (Sun et al. 2025 Yao al. 2023 Von Kügelgen et al. 1021). By conditioning on s-for example, gene-level representation we can achieve component- wise identifiability of latent variables and recover their causal graph under milder assumptions. Furthermore, Proposition [ shows that the modality sppciffcc latent structure z obtained via Theorem 2 facilitates the resolution of permutation indeterminacies across the latent spaces associated with different modalities.
45

---

Published as a conference paper at ICLR 2026
A7 DETAILS ABOUT NETWORK TRAINING FOR CAUSAL REPRESENTATION LEARNING
In this section, inspired by identifiability results shown in the Theorems, will introduce estimation framework which enforces the proposed assumptions as constraints to identify the latent variables in each modality, in total we several loss functions as constraints. The details given as follows.
Network Architecture. For the high-ding dighesiieenii data, we large foundation model to extract high-dimensional feature first, and then use the 3-layer multi- perception (MLP) for the encoders and decoders. Specifically, for image data, we utilize ImageBind (Girdhar al. 2023) extract 1024-dimensional embedding vectors, as this model excels at multi-modal embedding extraction. For text descriptions, we employ the gte- Dewen2-7B-insiie is specifically designed for long-sentence embedding tasks and demonstrates superior performance in capturing semantic representations from extended textual content. After this gte model, we will get 5584-dimensions embedding vector for each input text description.
model from Alibaba (Bai et al.] 2023), which
Encoder and decoder. Each modality Xm is given as an input to the corresponding encoder and outputs the estimated modality-specifff latent 2,,. exogenous variables fm and shared latent variables s across different modalities. In one modality, to ensure the conditional independence among different Xm.k given 2m. X are passed to their corresponding k-th decoders, respectively, to reconstruct the observations &m.k in each measurement. The reconstruction loss calculated using the mean squared error (MSE)
M
die
C Recon
||m,kk - 8m
Conditional independence constraints. We enforce the conditional independence condition Xm.j 1 Xm,k Zm (where Xm.,j and Xm,k refer to the j-th and k-th measurements in m-th modality) and the independence condition on 17 _. Zm by enforcing the independence among components in y = [[2m IM 1 ) [= To implement it, we assume that y follows an independent prior distribution p(7), such standard isotropic Gaussian, and enforee the independence by matching the distribution of f to the prior distribution. Specifically, we minimize the KL divergence between the posterior and Gaussian prior distribution as follows:
Lind = KL(((K I)).
Proposition (Conditional Independence Condition). Denote Xm.. and Xm,k are two different measurements in one modality for the m-th modality with modality-ppecffc latent variable 2.n- Zm C. Z is the set of block-identified latent variables, and Tm C. 1) are exogenous variables in modality m. We have Xm.j Lmm,k . Zm Em.j 1 Em,k.
Proposition (Independent Noise Condition). Denote z and n as the srook-ideetifieed latent variables and exogenous variables across all modalities. e's are the nuusallyrelateer noise terms. We have 77 1 - 1j 1
Sparsity regularization. We use normalization flow (Huang et al. 2018) to estimate the exogenous variables and implement the causal relations through learnable adjacency matrix A. The binary values in À represent the causal generation process between latent variables, e.g. A 1 indicates 2j is the parent of é,, while A = 0 means ž, dose not contribute to the generation of á,. For each component â, select its parents Pa(2,) based on the estimated causal adjacency matrix, and apply the flow transformation from Pa(2,) to éG.
To encourage sparsity among the latent variables é, we introduce a regularization term on the learned adjacency matrix. The sparsity assumption indicates that the optimal causal graph should be the minimal one which still allows the model to successfully match the ground truth observational distribution. In particular, we reduce the dependencies between different components of 2 by adding Li penalty on the adjacency matrix, s.t.,
46

---

Published as a conference paper at ICLR 2026
Table A7: Key hyperparameters used in experiments.
Hyperparameter
MNIST
PersonaX
Learning Rate
2e-6
3e-4
Training Epochs
3000
3000
Reconstruction Loss Coefficient
2
Conditional Independence Loss Coefficient
le-2
le-2
Sparsity Loss Coefficient
le-3
1e-3
Spp
||
Network Training. objective:
In summary, the model parameters are optimized using the combination
L ORRecnn CRecon OInd + CsppSsp
(41)
A8 DETAILS ABOUT SYNTHETIC EXPERIMENTS ON VARIANT MNIST
In this section, we will introduce the synthetic experiments designed validate our proposed causal representation learning framework. We conduct comprehensive evaluations using carefully constructed datasets with known causal relationships, allowing us systematically assess the performance of our method against established baselines.
A8.1 DETAILS ABOUT EXPERIMENTAL SETUP
To systematically evaluate our proposed causal representation learning framework, we construct synthetic dataset with known ground-truth causal relationships using variants of the MNIST dataset. Our synthetic dataset consists two modalities: colored MNIST and fashion MNIST, each containing causally related latent variables. For colored MNIST, define horizontal position latent cause that influences image transparency, where digits are positioned at different horizontal locations and their transparency varies accordingly. For fashion MNIST, we establish vertical position as a latent cause that affects grayscale intensity of the clothing items. The causal structure connects these modalities through a cross-modas relationship: the horizontal position in colored MNIST serves as a causal factor for the vertical position in fashion MNIST, creating meaningful inter- modal dependency. Notably, our dataset design reflects different measurement characteristics across modalities: for fashion MNIST, each sample contains single image representing one measurement, while for colored MNIST, we generate three images with different background colors (red, green, blue) for each sample, providing three distinct measurements that capture different aspects of the same underlying latent variables. The generated image examples are shown in Figure (). The key hyper-parameters are listed in Table A7
Ground Truth Causal Graph and Training Configuration. The underlying causal relationships in synthetic dataset are illustrated in Fig.re(S]b). The causal graph demonstrates how latent variables within and across modalities interact: horizontal position in colored MNIST causally influences both the image transparency within the same modality and the vertical position in fashion MNIST across modalities. Subsequently, the vertical position in fashion MNIST determines the grayscale intensity of the fashion items. This carefully designed causal structure enables us evaluate whether our method can correctly identify and disentangle these known causal relationships from the observed multi-modal data.
A8.2 DETAILS ABOUT RESULTS AND ANALYSIS
We compare our approach against several baseline methods including MCL, Beta and MMCRL using two key metrics: R² (coefficient of determination) and MCC (Matthews Correlation Coefficient). As shown in Figure Figure3]c). method consistently outperforms all baseline approaches across both evaluation metrics. Specifically, our approach achieves R² scores of 0.96 and MCC scores of 0.92, demonstrating superior performance both regression and classification tasks for causal
47

---

Published as a conference paper at ICLR 2026
[S
e.g. cultural background
[S,] e.g. education
[S
g growing environment
12 e.g. clothing
[22,
1 e.g. positive language
[Z 2 e.g. social context
[ 1,2] e-g. fitness
[ egg. friendliness
[2 e.g. lighting
12,4 e.g. hair style
[Z e.g. language
1 e.g. event context
[Z,,5] e.g. expresiveness
[ .l e.g. overall style
1,100l e.g. approachability
[ 1 good looking
1 e.g. posture
[2, e.g. visibility
Figure A6; The causal graph with latent variables learned from CelebPersona dataset. Red, blue, and green nodes correspond shared latents, facial image latents, and trait text latents.
variable identification. The substantial improvement over strong baselines like MMCRL (R² 0.90, MCC 0.85) validates the effectiveness of proposed framework in learning causally meaningful representations from multi-modal observations. These results confirm that our method successfully captures the underlying causal structure while maintaining high fidelity in representation learning, even when dealing with asymmetric measurement structures across different modalities.
A9 DETAILS ABOUT REAL-WORLDD BEHAVIOR TRAIT ANALYSIS ON PERSONAX
A9.1 DETAILS ABOUT EXPERIMENTAL SETUP
We conduct real-world behavior trait analysis by training our network to extract latent representations from both the image and text modalities of the Celebberson dataset, followed by the application of causal discovery to reveal underlying structures. The key hyper-parameters are listed in Table A7 The resulting causal graph for AthlePersona is at Fig. 6 For CelebPersona the causal graph is shown in Fig. A6 identify three shared latent variables (S1. S2, Sz), ten latent variables derived from facial images (Z1,1 to Z1.10). and five latent variables extracted from behavior trait descriptions (Z2,1 52 Each variable is grounded in real- wol-w interpretable features, enabling meaningful analysis of the causal pathways.
A9.2 DETAILS ABOUT RESULTS AND ANALYSIS
We interpret the shared latent variables S1, Sz. and S3 as representing education, cultural background. and growing environment, respectively. Notably, S, influences Z2.4, which we interpret as cultural background shaping one's language use, while S3 influences Z2,1, suggesting that the growing environment affects the use of positive language. Furthermore, expressiveness (Z2.6) is found to causally influence approachability 1Z1, reinforcing the idea that one's ability to convey emotions plays key role in how approachable they appear. On the visual side, observe that variations in event context (Z1.s) and lighting conditions (Z1.)) lead to changes in hairstyle (Z,,4), which in turn influence face visibility (Z,,)). overall style (Z),)), and how good-looking (Z,,7) the person appears.
To validate our example, we conducted an RCIT test between the Big Five traits (Final to Final N) and two sets of latent variables: five derived from trait descriptions across both datasets. We also
48

---

Published as a conference paper at ICLR 2026
Final_o 0.024
0.019
0.000
0.354
000
Final_o
0.002
0.004
0.061
0.004
0.417
Finalc
0.252
0.139
0.002
0.176
014
Rinal_
0.037
0.337
0.232
0.007
0.253
Finale
0.000
0.030
0.178
0.000
058
Final e
0.281
0.265
0.174
-0.8
Finala
0.032
0.200
0.018
0.158
000
Final_a
0.000
0.004
0.534
0.113
0.520
Pinaln
0.314
0.419
0.382
0.484
302
0.490
0.034
0.089
0.443
0.336
Z2_1
22_2
Z2_3
z2_4
22.5
Z2
22.3
Z2
(a) AthlePersona Personality
(b) CelebPersona Personality
Arched_Eycbrrws
0.001
0.001
0.202
0 000
0.000
0.000
0.000
0.000
0.002
0.005
Povalue
Nose
0.000
0.000
000
0.000
0.000
0.000
Pointy_Nose
0. 0.000
0.000
0.000
0 000
0.000
0.000
0.000
0.000
0.000
0.000
-0.4
Eyebrows
0.336
Big Big_Lips
1:000
0.613
0.986
0 552
0.604
0.540
0.685
0.760
0.451
0.232
Ova_Face
0. 0.000
0.000
0.000
000
0.000
0.000
0.000
0.000
0.000
0.000
Double_Chin
0.001
0.002
0.001
001
0.001
0.000
0,000
0.001
0.003
0.015
Receding Hairtine
0.000
0.000
0.011
0 000
0.000
0.000
0.000
0.001
0.000
0.000
0.318
0.388
0.252
Nigh_Chheebones
0.000
0.000
0.000
0 000
0.000
0.000
0.000
0.000
0.000
0.000
zi'
z1'2
zi.3
z1.4
zi.5
21.6
21 7
z1.
21.9
21
-00
(c) CelebPersona Facial
Figure A7: The RCIT test between the Big Five traits (Final_O to Final.N) and two sets of latent variables: five derived from behavior trait descriptions across both datasets, (a) AthlePersona and (b) CelebPersona. (c) refer to the same test on ten facial attributes from CelebPersona and ten latent variables derived from facial images.
carry out the same tests ten facial attributes from CelebPersona and ten latent variables derived facial images. shown A7 (a), confidence (Z2,1) exhibits strong statistical dependence with Openness, Extraversion, and Agreeableness. In contrast, Self-awareness (Z2,4) is significantly associated only with Extraversion, suggesting that more extraverted individuals tend to be more self-aware, likely due to their expressiveness, social engagement, and sensitivity.
For the test result of CelebPersona in Figure A7 (b), positive language use (Z2.1) has significant dependence with Agrecableness indicates that more agrecable individuals are likely to warmer and more positive language, aligning with their prosocial and empathetic tendencies. On the other hand, the high p-values across all Big Five traits suggest that expressiveness (Z2,s) operates independently of stable behavior trait dimensions in this dataset, possibly reflecting more situational or behavior factors captured by self-reported traits. In Figure A7 (c). the p-value heatmap confirms that many facial attributes are significantly influenced by latent appearance factors like clothing style (Z,,1,, lighting (Z1,)). and event context (Z1,5), as shown in the causal graph. Traits like Nose. Pointy Nose, and Oval Face are tightly linked to hairstyle and good looking (Z1,7).
A9.3 RESULTS AND ANALYSIS OF BASELINE MMCRL
Similarly above, We conduct real-world behavior trait analysis by the baseline method MMCRL (Sun et al. 2025) to extract latent representations from both the image and text modalities of the Ce GelebPere dataset, followed by the application of causal discovery reveal underlying struc-
49

---

Published as a conference paper at ICLR 2026
[S
e.g. cultural background
[s,] e.g. education
[S
e. growing environment
[22, 1l e.g. positive language use
12 e.g. clothing
[ 2, e.g. social context
[ 1,2] e-g. posture
[Z e.g. friendliness
[2 e.g. lighting
12,4 hair style
|Z e.g. language
1 e.g. event context
[ e.g. expresiveness
[ e.g. overall style
1 e.g. approachability
[ 1 good looking
1 e.g. posture
1,8 e.g. visibility
Figure A8: The causal graph by baseline method MMCRL learned from CelebPersona dataset.
Red, blue, and green nodes correspond shared latents, facial image latents, and trait text latents.
[S,] e.g. mindset
[S e.g. culture
[ e.g. Complexion tone
[Z 1 confidence
[Z, 1 e.g. presentation style
[Z, e.g. achievement
[Z, [2, e.g. facial harmony
[Z 2,33 e.g. regulaatio
[Z, [Z,,4] e.g. expressivity
[Z2,4] e.g. social adaptability
[2] e.g. visual charisma
[2 e.g. positivity
Figure A9; The causal graph by baseline method MMCRL learned from AthlePersona dataset. Red, blue, and green nodes correspond to shared latents, facial image latents, and trait text latents.
tures. The key hyper-parameters are listed in Table A7 The resulting causal graph for AthlePersona Fig. A9 For CelebPersona the causal graph is shown in Fig. A8 identify three shared latent
variables (Si. S2, Sz), ten latent variables derived from facial images (Z1,1 to Z1,10). and five latent variables extracted from behavior trait deseriptions (Z2,1 52 Each variable is grounded in real-world interpretable features, enabling meaningful analysis of the causal pathways.
We similar way analyze each latent variables. The results show that MMCRL, which assumes mingle-measurement data, produces denser and less interpretable causal graphs. In contrast, our multi- measurement CRL yields sparser, Quantitatively, our method achieves higher causal alignment metrics (for example, MCC and R² in synthetic results as shown Section 4.4) and here qualitatively shows clearer separation between visual and behavioral latent factors (in this real-world experiments). These findings confirm the advantages of our design for real-world multimodal causal analysis.
stable, and semantically coherent cross-modal structures.
50


Reviews:
[{"review_id": "d5RmETRtJ6", "reviewer": "Reviewer_vCuK", "summary": "This paper introduces PersonaX, a set of curated multimodal datasets—CelebPersona and AthlePersona—incorporating LLM-inferred behavioral trait assessments (Big Five), facial embeddings, and structured biographical metadata for public figures and professional athletes. The authors present a two-level analytical framework: (1) statistical independence tests exploring modality-specific dependencies between behavioral traits and structured features, and (2) a novel causal representation learning (CRL) model with theoretical identifiability guarantees for multimodal, multi-measurement data. Empirical validations are performed using both synthetic and real-world datasets, demonstrating the proposed approach’s effectiveness in cross-modal trait analysis and causal discovery.", "questions": "What is the impact of prompt selection and LLM choice on downstream causal graph structure and variable interpretability? Are the reported findings robust to these design choices?\nHow would the proposed method scale to or generalize with with populations beyond public celebrities and male athletes, and what steps are planned to mitigate cohort-specific biases?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "PersonaX provides a valuable resource that bridges LLM-inferred behavioral traits with biographical and visual data across thousands of public personas, addressing a clear gap in multimodal behavioral trait datasets. \nMethodological advances in causal representation learning: The paper proposes an end-to-end causal modeling framework tailored to multimodal, multi-measurement data, supported by reasonably rigorous identifiability theory.\nExtensive experiments are conducted both on synthetic settings and real-world data, with multiple baselines including BetaVAE, MCL, and MMCRL.\nThe data pipeline and experimental protocols are well-illustrated, with detailed breakdowns in supplementary materials.", "weaknesses": "Limited diversity and representativeness in dataset sampling. CelebPersona and AthlePersona focus on public figures and male athlete, which are not demographically representative of broader populations. This constrains the generalizability of reported findings and may bias downstream analyses, as acknowledged but not addressed in either analysis or ablation.\nExperiments lack real-world baseline comparisons for CRL. The effectiveness of the CRL component is shown mostly on synthetic MNIST and real-world data in isolation; it would be more persuasive to demonstrate improvements over state-of-the-art real-world baselines or alternative causal discovery methods.\nCausal graph interpretability and stability concerns. The mapping between learned latents and original human-interpretable traits is indirect.\nPresentation and clarity issues in tables/figures. Table 1 is visually dense; important metrics such as overall score should be more clearly emphasized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YmcIctjixc", "reviewer": "Reviewer_e8Dt", "summary": "This paper proposes PersonaX, a collection of multimodal datasets including LLM-inferred behavior trait assessments with facial imagery and structured biographical features, and the paper introduces a two-level analysis pipeline: (1) structured independence testing to discover trait-attribute dependencies (2) a multimodal, multi-measurement causal representation learning (CRL) framework with identifiability guarantees that learns shared and modality-specific latent factors. The authors provide dataset construction details, systematic LLM selection and prompt design, synthetic benchmarks, and real-world analyses on PersonaX.", "questions": "1.Strengthen label validity. Incorporate human-annotated subsets or downstream behavioral prediction tasks to quantitatively verify the reliability and utility of the inferred traits.\n2.Address cohort bias. Broaden demographic coverage or conduct sensitivity and subgroup analyses to isolate dataset-specific dependencies and clarify the scope of generalization.\n3.Clarify theoretical grounding. Provide ablations and diagnostics verifying identifiability-related assumptions and assess the impact of the embedding obfuscation transformation on model performance.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.Novel multimodal benchmark. PersonaX fills a clear gap by uniting LLM-derived personality assessments with facial and biographical modalities, enabling large-scale, cross-modal trait analysis not available in existing resources.\n2.Rigorous LLM measurement pipeline. The authors comprehensively benchmark multiple LLMs, prompt templates, and scoring schemes, reporting consistency and missing-rate analyses that enhance transparency and reproducibility.\n3.Integration of theory and practice. The proposed CRL framework offers identifiability guarantees and is validated through both synthetic benchmarks and real-world data, effectively bridging theoretical development and empirical evaluation.", "weaknesses": "1.Insufficient external validation of LLM-inferred traits. Despite internal consistency checks, the lack of human-annotated validation or behavioral benchmarks undermines confidence in the real-world accuracy of the inferred personality traits.\n2.Limited demographic diversity and potential bias. The datasets primarily include male athletes and high-profile celebrities, introducing demographic and socioeconomic biases that constrain generalization and may lead to spurious correlations.\n3.Under-specified theoretical assumptions in practice. The identifiability results rely on strong technical conditions whose practical validity for noisy, LLM-based embeddings is not empirically examined, limiting interpretability of real-world findings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p24qY8AsAU", "reviewer": "Reviewer_aPN1", "summary": "The paper presents PersonaX, a multimodal dataset framework designed to explore behavioral trait inference through large language models (LLMs). It introduces two datasets: CelebPersona, comprising 9,444 public figures, and AthlePersona, including 4,181 professional athletes from seven major leagues. Each record combines LLM-inferred Big Five personality traits, facial embeddings, and biographical metadata. The authors conduct a two-level analysis: structured statistical independence testing and a causal representation learning (CRL) framework with identifiability guarantees, evaluated on both synthetic and real data. The results reveal interpretable cohort-specific dependencies, and the dataset is released under privacy-preserving, non-commercial terms.", "questions": "1. Have you performed any human validation or external benchmarking to assess the reliability of the LLM-inferred Big Five traits, even on a small subset of samples?\n2. The causal representation learning (CRL) framework depends on assumptions such as measurement noise independence and modality injectivity. What empirical checks or sensitivity analyses did you perform to verify that these assumptions approximately hold for PersonaX?\n3. How stable are the learned causal graphs across random seeds or when a modality (e.g., images or text) is removed? Do key dependency patterns persist?\n4. Have you compared the CRL embeddings to any non-causal multimodal baselines (e.g., CLIP-style or multimodal autoencoders) to verify that the causal formulation provides additional structural or representational advantages?\n5. Could you elaborate on the privacy-preserving mechanism applied to the released facial embeddings, specifically, whether the transformation is invertible or resistant to re-identification attacks?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Comprehensive dataset linking visual and behavioral attributes through LLM inference.\n2. Multi-level analysis combining structured statistical tests and causal representation learning.\n3. Privacy-aware release strategy (embeddings only, non-commercial use).\n4. Systematic evaluation of LLMs and prompts for reliability.\n5. Interpretability and cross-domain insights (e.g., appearance importance for celebrities vs. organizational traits for athletes).\n6. Well-documented, reproducible methodology.", "weaknesses": "1. No human-ground-truth validation for LLM-inferred Big Five scores. The inferred personality traits are never compared against human annotations or standardized psychometric assessments, making it difficult to evaluate their validity. A small-scale validation study would substantially improve confidence in the dataset’s behavioral realism.\n2. Gender and domain bias (athletes are all male, celebrities are mostly Western). These sampling biases limit generalizability and may influence the discovered dependencies, as LLMs and facial embeddings could internalize social and cultural stereotypes from biased data sources.\n3. Causal interpretations rely on untestable assumptions. The proposed causal representation learning (CRL) framework depends on theoretical assumptions such as independent measurement noise, injectivity of modality-specific measurement functions, and sufficient diversity across modalities to ensure identifiability. These assumptions are difficult to verify empirically in real-world multimodal datasets like PersonaX, where both LLM-inferred traits and facial embeddings are high-dimensional and potentially correlated. Without diagnostic checks or robustness analyses, the causal graphs may reflect associations rather than true underlying causal structure, making interpretability plausible but not necessarily causal.\n4. Lack of downstream benchmark tasks to demonstrate the utility of learned latents.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DsQTG2jY4c", "reviewer": "Reviewer_GsZk", "summary": "This paper introduces PersonaX, a pair of multimodal datasets for studying LLM-inferred behavioral traits: CelebPersona and AthlePersona. Each example includes (i) LLM-generated Big Five trait descriptions/scores, (ii) facial information, and (iii) biographical fields. The authors analyze PersonaX at two levels: (1) applying five statistical independence tests between inferred traits and structured/visual attributes; and (2) proposing a Causal Representation Learning (CRL) framework for multi-modality & multi-measurement data with identifiability guarantees, validated on synthetic data, and then used it to learn latent representations and causal graphs on PersonaX.", "questions": "1. Has any effort been made to assess the consistency between LLM-inferred and human-assessed Big Five personality traits?\n2. Could the inclusion of league information in the prompt introduce or reinforce stereotypes?\n3. How can the current analysis of athletes and public figures offer for generalizing to broader populations?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tA New Multimodal Dataset.\nThe paper introduces PersonaX, a multimodal dataset that aligns two corpora and pairs LLM-inferred Big Five personality traits with facial imagery/embeddings and biographical attributes, enabling large-scale cross-modal analysis.\n2.\tModel and Prompt Selection Strategy\nThe authors conduct a detailed evaluation (Figure 2) of various models and prompt templates to select the most reliable combination for dataset construction. \n3.\tTwo-Level Analysis Pipeline.\nThe proposed analysis framework consists of two levels:\n\t* Level I applies five statistical independence tests to systematically characterize dependencies between Big Five scores and structured/visual features.\n\t* Level II introduces a causal representation learning framework with identifiability guarantees. This framework is first validated on synthetic data and then applied to the real multimodal embeddings, uncovering interpretable cross-modal structures.", "weaknesses": "1.\tLack of Human Validation for LLM-Inferred Personality Traits.\nThe Big Five personality traits inferred by LLMs have not undergone human validation. While prior work has shown that LLMs can somehow perform personality analysis tasks, the release of a dataset built on such inferences should be accompanied by human verification to ensure quality. Although the authors carefully selected the model and prompt to reduce variance and improve consistency, this does not guarantee alignment with human judgments.\n2.\tLimited Generalizability.\nThe AthlePersona subset is restricted to male athletes, and the CelebPersona subset primarily consists of public figures. As such, the conclusions drawn in this paper may not generalize well to broader or more diverse populations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)