TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation

## Abstract

Abstract Large Language Models (LLMs) are exhibiting emergent human-like abilities and are increasingly envisioned as the foundation for simulating an individual’s communication style, behavioral tendencies, and personality traits.
However, current evaluations of LLM-based persona simulation remain limited: most rely on synthetic dialogues, lack systematic frameworks, and lack analysis of the capability requirement.
To address these limitations, we introduce TwinVoice, a comprehensive benchmark for assessing persona simulation across diverse real-world contexts.
TwinVoice encompasses three dimensions: Social Persona (public social interactions), Interpersonal Persona (private dialogues), and Narrative Persona (role-based expression).
It further decomposes the evaluation of LLM performance into six fundamental capabilities, including opinion consistency, memory recall, logical reasoning, lexical fidelity, persona tone, and syntactic style.
Experimental results reveal that while advanced models achieve moderate accuracy in persona simulation, they still fall short of capabilities such as syntactic style and memory recall.
Consequently, the average performance achieved by LLMs remains considerably below the human baseline. \metadata [ Project Page] https://twinvoice.github.io \metadata [ Code] https://github.com/TwinVoice/TwinBench \metadata [Ξ Leaderboard] https://huggingface.co/spaces/bangdedadi/TwinVoice-Leaderboard \metadata [ Dataset] https://huggingface.co/datasets/bangdedadi/TwinVoice

## 1 Introduction

Large Language Models (LLMs) are rapidly evolving from basic text generators into human-like agents [8, 47, 9].
Existing studies have shown that the most advanced LLMs are capable of producing text indistinguishable from human writing [19, 20, 18].
Consequently, the research focus is shifting toward a highly specific challenge: Can we construct “digital twins” of specific individuals that are indistinguishable from themselves?
To address this challenge, existing studies have relied on LLM-based persona simulation, which replicates a person’s unique style of talking, behavior, and personality [39, 33] based on their data.
LLM-based persona simulation is supposed to unlock a series of applications, including highly personalized assistants [29, 25], social simulations [24, 37, 38], healthcare [3], and marketing [16].
Despite growing interest in creating digital twins with LLM-based persona simulation, its current ability remains unexplored due to the lack of systematic evaluation [45, 53].

Figure: Figure 1: The conceptual framework of TwinVoice: (Left) The evaluation is structured across three core dimensions that represent distinct aspects of identity expression: Social Persona (public facing), Interpersonal Persona (private interaction), and Narrative Persona (fictional scenarios). The LLMs are prompted with a person’s historical context to simulate their behavior. The LLM’s ability for persona simulation is categorized into six fundamental capabilities. (Right) Experimental results averaged over three dimensions are presented.
Refer to caption: x1.png

To address this issue, current evaluations have attempted to test LLM’s ability in imitating and predicting human behaviors.
For example, BehaviorChain [27] evaluates continuous persona-based behavior by requiring models to iteratively predict the next action given persona profile and history, with performance degrading as chains lengthen.
Human Simulacra [48] and PersoBench [1] assess human-likeness and personalized response quality, while other studies probe persona-driven decision making, counterfactual adherence, and large-scale dynamic profiling [49, 23, 17].
However, those evaluation benchmarks face limitations in both their scope and granularity.
On the one hand, the predominant reliance on synthetic dialogues [41, 46] prevents benchmarks from capturing the rich expression of human identity across diverse real-world contexts (Scope Limitation).
On the other hand, current benchmarks are often evaluated based on an LLM’s accuracy in predicting human behavior, leaving a critical gap in understanding the fundamental capabilities—such as memory, reasoning, and lexical fidelity—that a model must possess for authentic simulation (Granularity Limitation).

To bridge the gap between the vision of digital twins and the current capabilities of persona simulation, we introduce TwinVoice, a comprehensive benchmark designed for realistic and fine-grained persona evaluation (see Table [1](https://arxiv.org/html/2510.25536v2#S1.T1) for a comparison with prior persona‑simulation benchmarks; “persona size” denotes the number of distinct, independent personas per benchmark).
To address the scope limitation, TwinVoice is grounded in both real-world and synthetic data across three complementary dimensions in persona simulation (see Figure [1](https://arxiv.org/html/2510.25536v2#S1.F1)): Social Persona, Interpersonal Persona, Narrative Persona.
The Social Persona dimension leverages real-world social media data to evaluate a public-facing identity, while the Interpersonal Persona dimension utilizes multi-session dialogue data to assess a more private, relational self.
While these two dimensions are grounded in authentic digital footprints, the Narrative Persona is designed to complement such data with fictional scenarios to test behaviors and narrative consistency in more diverse contexts.
Addressing the granularity limitations of holistic accuracy evaluations, we shift from end-to-end scoring to capability-level assessment.
Building on psycholinguistic evidence that language conveys both what people say and how they say it [34], we group persona fidelity into Mindset Coherence and Linguistic Expression.
Mindset Coherence assesses the logical and factual consistency of the content, including Opinion Consistency [51], Memory Recall [11], and Logical Reasoning [21].
Linguistic Expression evaluates the language’s stylistic form, encompassing Lexical Fidelity [30, 22], Persona Tone [7], and Syntactic Style [6].
Digital twins face two practical interaction modes: constrained selection among plausible replies and open-ended generation under realistic stimuli.
Accordingly, TwinVoice instantiates both modes in its tasks to balance objectivity and ecological validity. We later evaluate the tasks with a multiple-choice protocol and a judge-based protocol, and include a human consistency check (see Sections 5.3 and 5.4).

**Table 1: A comparison of TwinVoice with Prior LLM Persona‑Simulation Benchmarks.**
| Benchmark | Persona<br>Size | Real-World<br>Sourcing | Multiple<br>Dimensions | Multi‑Paradigm<br>Evaluation | Human<br>Baseline | Fine-Grained<br>Capabilities | Multilingual<br>Coverage |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Human Simulacra [48] | 11 | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ |
| BehaviorChain [27] | 1,001 | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ |
| PersonaEval [52] | 130 | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ |
| PERSONAMEM [17] | 20 | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ |
| \rowcolorOurRow<br>TwinVoice OURS | 4,553 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |

We evaluate a series of state-of-the-art LLMs on TwinVoice under both paradigms and reveal several key insights into current capabilities and limitations in persona simulation.
In the discriminative task, GPT-3.5-Turbo achieves an accuracy of 47.5%, while more advanced models show considerable gains, reaching 71.2% and 76.2% for GPT-5 and Claude-Sonnet-4, respectively [2].
In the generative tasks, GPT-5 [32] leads with 48.5% accuracy in generative-ranking tasks and a score of 2.13 in the generative-scoring tasks.
The average performance across different models demonstrates that the dataset can significantly differentiate the advancement of LLMs.
However, the persona simulation performance achieved by LLMs still falls short of human baselines.
On the subset under the discriminative task, the human majority vote accuracy is 66.0%, while GPT-5 reaches 60.0%, indicating clear headroom.
The performance also varies across different dimensions and capabilities.
For dimensions, we observe that the average performance is highest under the Narrative persona, while Social and Interpersonal lag.
This finding suggests that simulating human behavior within Social and Interpersonal Personas presents a greater challenge, likely due to the demands of dynamic interaction and long-term consistency inherent to these dimensions.
Across capabilities, models perform best on Lexical Fidelity and Opinion Consistency and worst on Persona Tone and Memory Recall, indicating an imbalance in their persona simulation abilities.
These patterns will guide subsequent research and upgrades to LLM persona simulation.

Contributions of this work are threefold: (1) We introduce TwinVoice, a comprehensive benchmark for evaluating LLM-based persona simulation across multiple real-world scenarios with systematic competency decomposition; (2) We develop novel evaluation methodologies combining discriminative assessment with LLM-as-a-Judge for generative tasks; and
(3) We provide extensive empirical analysis showing the limitations of the most advanced LLMs in person simulation and offer crucial insights for advancing personalized AI systems.

## 2 Related work

### 2.1 Personalized Agents and Digital Twins

The construction of digital twins, virtual replicas of specific individuals, is an emerging challenge in AI [39, 33]. Originating in engineering as counterparts to physical systems [14], the concept now extends to AI agents that capture a person’s communication style, preferences, and personality. Recent efforts have operationalized this vision across diverse domains. Examples include reviving anime characters [24], simulating agent societies from novels [37, 28], and evaluating impersonation of writing styles and memories [42]. Applications have been explored in healthcare [3], marketing [16, 43], and through industry systems like SecondMe [40] for lifelong personal modeling.
While these human-centered digital twins promise highly personalized chatbots [29, 25] and ubiquitous computing applications [13], prior research has often focused narrowly on style imitation, overlooking the broader competencies required for authentic persona simulation.

### 2.2 Datasets, Benchmarks, and Evaluation for Persona Simulation

Progress in this area depends on high-quality datasets and benchmarks. Recent resources have begun to fill this gap, offering diverse evaluation protocols. Benchmarks have been developed from large-scale surveys of human traits [45, 10], persona-based behavior chains [27], psychology-guided agent evaluations [48], persona-driven decision-making tasks [1, 49], and multi-party dialogue role identification [53]. More recent work explores challenging settings like counterfactual simulation [23] and dynamic user profiling [17].

Despite this growing landscape, evaluations remain fragmented and often rely on synthetic data, limiting their ecological validity. This highlights the need for a unified framework to advance digital twin research rigorously. Our TwinVoice benchmark addresses these limitations by leveraging real-world social media, conversational, and fictional data to provide authentic and systematic evaluation across multiple persona dimensions.

## 3 Task Formulation

Figure: Figure 2: TwinVoice experiment evaluation overview: Top: The LLMs are prompted with a specific persona’s history and tasked with a stimulus. Bottom: Three evaluation protocols: Discriminative: the model chooses among A–D, one of which is the ground truth persona behavior. Generative-Ranking: the model writes and an LLM‑as‑a-Judge selects the best candidate, yielding Acc.(Gen). Generative–Scoring: the model writes and the Judge rates similarity on opinion, logic, and style, yielding Score(Gen).
Refer to caption: x2.png

### 3.1 Problem Definition

TwinVoice evaluates LLMs’ ability to simulate human personas through a unified task paradigm that captures the essence of digital twin functionality. Formally, we define the persona simulation task as follows:

Given a persona’s historical data $\mathcal{H}=\{h_{1},h_{2},\ldots,h_{n}\}$ and a current stimulus $s$, the history is instantiated per dimension (Social, Interpersonal, or Narrative) as social posts, multi-session conversations, or narrative materials, respectively.
The objective is to generate a response $r$ that maximally approximates the ground truth response $r^{*}$ that the original persona would produce in stimulus $s$, which can be formulated as an optimization problem:

$$ $r^{*}=\arg\max_{r}P(r|\mathcal{H},s,\theta_{\text{persona}}),$ (1) $$

where $\theta_{\text{persona}}$ represents the latent persona characteristics learned from historical data $\mathcal{H}$. The evaluation objective is to assess how well an LLM $M$ can approximate this optimal response:

$$ $\text{Score}=f_{\text{sim}}(M(\mathcal{H},s),r^{*}),$ (2) $$

where $f_{\text{sim}}$ denotes a similarity function that measures persona consistency across multiple dimensions.

TwinVoice instantiates this general framework across three dimensions, each defined by its history source and interaction stimulus:

Across all three settings, we adopt a capability-centric evaluation rather than a single holistic score. The decomposition and scoring criteria are detailed in Section [4.2](https://arxiv.org/html/2510.25536v2#S4.SS2).

### 3.2 Evaluation Methodologies

To balance objectivity and ecological validity, we pair a discriminative multiple-choice evaluation (objective, low-variance accuracy under controlled distractors) with a generative evaluation (open-ended persona fidelity via LLM-as-a-Judge in ranking and scoring).

#### 3.2.1 Discriminative Evaluation

The discriminative evaluation transforms the generation task into a multiple-choice selection problem. For each test instance $(s,r^{*})$, we construct a candidate set $\mathcal{C}=\{r^{*},r_{1},r_{2},r_{3}\}$ where $r^{*}$ is the ground truth response and $\{r_{1},r_{2},r_{3}\}$ are distractors.
The evaluated LLM must select the most persona-consistent response from the shuffled candidate set.

The construction of distractors varies across dimensions to ensure realistic evaluation scenarios:

Discriminative evaluation provides direct accuracy measurements:

$$ $\text{Accuracy}=\frac{1}{N}\sum_{i=1}^{N}\mathbf{1}[M(\mathcal{H}_{i},s_{i})=r^{*}_{i}],$ (3) $$

where $N$ is the total number of test instances and $\mathbf{1}[\cdot]$ is the indicator function.

#### 3.2.2 Generative Evaluation

While discriminative evaluation offers clear interpretability, real-world digital twin applications require open-ended generation capabilities. Our generative evaluation employs LLM-as-a-Judge [15, 50] protocols to assess response quality along multiple dimensions.

We implement two distinct judging approaches:

Scoring-based Evaluation. The judge model rates generated responses against ground truth using structured evaluation criteria. Given a stimulus $s$, generated response $r_{\text{gen}}$, and ground truth $r^{*}$, the judge assigns a score on a 1–5 scale based on three key dimensions: opinion consistency, logical coherence, and stylistic fidelity. The scoring rubric emphasizes faithful persona replication, with higher scores awarded to responses that demonstrate comprehensive alignment across all dimensions.

Ranking-based Evaluation. The judge identifies the most persona-consistent response from a candidate set containing the generated response and the same distractors used in discriminative evaluation. This approach mirrors discriminative evaluation while leveraging the judge’s nuanced understanding of persona consistency.

The generative evaluation score is computed as:

$$ $\text{Score}_{\text{gen}}=\frac{1}{N}\sum_{i=1}^{N}\text{Judge}(r_{\text{gen},i},r^{*}_{i},s_{i}),$ (4) $$

where $\text{Judge}(\cdot)$ represents either the scoring or ranking function implemented by GPT-5.

## 4 Benchmark Construction

### 4.1 Data Pre-processing

##### Social Persona.

We constructed this dataset from the PChatbot Chinese microblog corpus [36]. To mitigate noise and ensure each evaluation instance is meaningful, we started with 8,045 samples and applied our PCCD (Persona-Clarity and Choice-Distinctiveness) framework. We filtered for users with rich histories (average reply length of more than 10 characters; Type-Token Ratio not in the bottom 20th percentile) and for tasks with unambiguous choices (response option cosine similarity less than 0.95). We then ranked the remaining samples by a persona-choice alignment score, calculated as the similarity to the true response minus the similarity to the most similar distractor, to select the final 2,000 high-quality instances.

##### Interpersonal Persona.

We used the Pushshift Telegram corpus [4] to evaluate memory-grounded consistency. Our curation process followed a multi-stage filtering funnel to distill a high-quality message set from 438,975 raw messages. We first selected high-activity users (active in three or more channels with 500 or more total messages and 100 or more per channel). We then processed their messages by removing short utterances of fewer than 5 tokens, retaining only the top 10% most informative instances by TF-IDF, and applying semantic deduplication (similarity threshold of 0.90), resulting in 6,150 messages. From these, we extracted 2,500 multilingual tasks (including several languages like EN, RU, ES, PT), using GPT-5 to generate challenging distractors to ensure the task tests deep persona understanding rather than superficial cue matching. We also incorporated users’ cross-channel chat history as memory to test for consistency across different social contexts.

##### Narrative Persona.

We selected eight novels from the Project Gutenberg corpus [35] to test the model’s ability to mimic the speaking styles of the given characters. From these novels, we extracted 1,187 speech segments covering more than 50 characters. To obtain these data, we first segmented each novel into short, indexed chunks, and from each chunk we extracted at most one utterance together with its context. We then matched the speakers to the list of main characters, whose profiles contained their personality traits, goals, motivations, and utterance histories. Once we finished collecting these speeches, each accompanied by the relevant profile and context, we constructed our test dataset, which included both multiple-choice questions and open-ended generative tasks. For the former, we paired each extracted utterance with three distractor options created based on the personalities of the other main characters. For the latter, we provided the context to the model and let it generate the most plausible utterance under the given circumstances.

### 4.2 Capability Decomposition

Guided by psycholinguistic evidence that language simultaneously conveys what people say (content) and how they say it (style) [34], we coarsely group persona fidelity into two complementary dimensions: *mindset coherence* and *linguistic expression*.
This view is consistent with stable individual differences in language documented across psychology and linguistics and their computational operationalizations [12, 5, 44, 31, 26]. We then instantiate these dimensions with six fundamental capabilities: mindset coherence comprises Opinion Consistency [51], Memory Recall [11], and Logical Reasoning [21], whereas linguistic expression comprises Lexical Fidelity [30, 22], Persona Tone [7], and Syntactic Style [6].

Annotation follows a prompt-aligned rubric: for each instance, annotators choose exactly one primary capability and independently assess all six capabilities as true or false under strict criteria.
Capabilities are non-orthogonal by design, so multiple capabilities can be true while a single primary label captures the best-fit signal.
Full instructions, criteria, and prompt excerpts appear in Appendix [B](https://arxiv.org/html/2510.25536v2#A2), with seed examples and the JSON output format for reproducibility.

## 5 Experiments

### 5.1 Experimental Setup

We evaluate digital twin fidelity across Social, Interpersonal, and Narrative personas with a discriminative multiple-choice task and a generative evaluation that consists of generative-ranking and generative-scoring.
The generative-scoring task in evaluation uses a GPT-5 as a judge to score the generated content from 1 to 5.
Dataset scale and average prompt length are summarized in Table [2](https://arxiv.org/html/2510.25536v2#S5.T2).

### 5.2 Overall Results and Key Findings

The overall results across the three dimensions and both evaluations are reported in Table [5.2](https://arxiv.org/html/2510.25536v2#S5.SS2), performance on each capability is shown in Figure [3](https://arxiv.org/html/2510.25536v2#S5.F3), and text similarity metrics are presented in Table [5.4.3](https://arxiv.org/html/2510.25536v2#S5.SS4.SSS3).
Strong models, notably GPT-5-Chat and Claude-Sonnet-4, lead across tasks.
The generative evaluation remains harder than the discriminative formulation: for the same model, Acc. (Gen) and Score (Gen) are consistently lower than discriminative accuracy.
Models are relatively stronger on Lexical Fidelity and Opinion Consistency and weaker on Persona Tone and Memory Recall.
Agreement between the judge and humans appears in Table [4](https://arxiv.org/html/2510.25536v2#S5.T4).
Given that humans are imperfect simulators in this task, we treat human accuracy as a practical reference point rather than a strict upper bound.
BLEU-1, METEOR, and BERT-Score provide complementary evidence in Table [5.4.3](https://arxiv.org/html/2510.25536v2#S5.SS4.SSS3).
Overall, the results point to remaining gaps in persona tone realization and in recalling and using persona-specific details during generation.

On average, Claude-Sonnet-4 achieves the best discriminative accuracy, while GPT-5-Chat achieves the best aggregate generative metrics.
Among open-source models, DeepSeek-V3 leads overall and attains the best Score (Gen) on the Social Persona dimension (2.77).
The Narrative Persona yields the highest scores; Social and Interpersonal are lower.
Within the Interpersonal dimension, Acc. (Gen) and Score (Gen) produce close but not identical orderings for top models, consistent with the distinction between ranking and absolute scoring protocols.

**Table 2: Dataset statistics across three dimensions. Each instance corresponds to a unique persona (#Users = #Instances). Avg = average; Gen = generative; Disc = discriminative. The instruction template is counted into Token counts.**
| Dimension | Instances | Avg history turns | Avg prompt tokens (Disc) | Avg prompt tokens (Gen) |
| --- | --- | --- | --- | --- |
| Social Persona | 2000 | 15.0 | 1371.1 | 1215.2 |
| Interpersonal Persona | 2500 | 30.0 | 1163.5 | 1139.4 |
| Narrative Persona | 1187 | 15.7 | 934.3 | 910.7 |

**Table 3: Benchmark results for Digital Twin models: We evaluate models using three distinct metrics: Acc. (%) is the accuracy on the discriminative task. Acc. (Gen) (%) is the accuracy where a generative model’s output is evaluated via multiple choice questions by a Judge. Score (Gen) is a pairwise comparison score against the ground truth for generative outputs by a Judge. Higher values indicate better performance. The best result and the second best result are in Bold and underlined, respectively.**
| \rowcolorgray!15 | Social Persona | Interpersonal Persona | Narrative Persona | Average |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| \rowcolorgray!15      Model / Tasks | Acc.<br>(%) | Acc.<br>(Gen)(%) | Score<br>(Gen) | Acc.<br>(%) | Acc.<br>(Gen)(%) | Score<br>(Gen) | Acc.<br>(%) | Acc.<br>(Gen)(%) | Score<br>(Gen) | Acc.<br>(%) | Acc.<br>(Gen)(%) | Score<br>(Gen) |  |
| LLM | GPT-3.5-Turbo | 34.9 | 26.0 | 2.57 | 41.2 | 40.1 | 1.53 | 66.3 | 46.2 | 1.98 | 47.5 | 37.4 | 2.03 |
| Qwen2.5-14B | 36.2 | 30.1 | 2.56 | 49.6 | 42.0 | 1.56 | 60.5 | 44.6 | 1.68 | 48.8 | 38.9 | 1.93 |  |
| GPT-4o-mini | 35.3 | 26.9 | 2.61 | 39.2 | 41.3 | 1.50 | 63.1 | 46.5 | 1.91 | 45.9 | 38.2 | 2.01 |  |
| GPT-OSS-20B | 39.1 | 24.1 | 2.39 | 63.3 | 46.0 | 1.47 | 43.9 | 48.0 | 1.77 | 48.8 | 39.4 | 1.88 |  |
| DeepSeek-V3 | 42.6 | 34.1 | 2.77 | 70.0 | 52.7 | 1.51 | 81.0 | 48.6 | 1.90 | 64.5 | 45.1 | 2.06 |  |
| GPT-5-Chat | 46.9 | 38.7 | 2.73 | 77.4 | 54.0 | 1.63 | 89.4 | 52.9 | 2.03 | 71.2 | 48.5 | 2.13 |  |
| Claude-Sonnet-4 | 53.9 | 37.5 | 2.67 | 84.4 | 52.9 | 1.67 | 90.2 | 53.4 | 2.02 | 76.2 | 47.9 | 2.12 |  |

### 5.3 Capability-wise Analysis

We analyze performance at the capability level within our framework and present the results in Figure [3](https://arxiv.org/html/2510.25536v2#S5.F3), aggregating discriminative accuracy with the two judge protocols (ranking and scoring).

From Figure [3](https://arxiv.org/html/2510.25536v2#S5.F3), we have the following observations.
First, the performance ranking of models across different capabilities closely matches their ranking based on average performance.
This indicates that stronger systems improve relatively uniformly rather than only in a narrow skill.
Second,
on average,
models score highest on *Lexical Fidelity* and *Opinion Consistency*, and lowest on *Persona Tone* and *Memory Recall*.
This suggests that current systems capture surface wording and stable stances better than persona-consistent tone and long-horizon recall.
Third, individual models show distinct comparative advantages; for example, DeepSeek-V3 approaches GPT-5 on *Lexical Fidelity* despite trailing on others.

Figure: Figure 3: Performance across six capabilities. Each panel shows one capability. For each model, bars give scores on the three dimensions—Social, Interpersonal, and Narrative. Purple diamonds indicate the mean across the three dimensions for that model. The y-axis is the average over the three evaluation protocols: discriminative, generative ranking, and generative scoring. The gray dashed line denotes chance level (25%).
Refer to caption: x3.png

### 5.4 Generative Evaluation

#### 5.4.1 Judge-based scoring and ranking evaluation

We assess the generative evaluation with two judge-based protocols introduced earlier, scoring (from 1 to 5) and ranking, and we aggregate their outcomes as Acc.(Gen) and Score(Gen).
We instantiate the judge with GPT-5.
Full results appear in Table [5.2](https://arxiv.org/html/2510.25536v2#S5.SS2), with prompt templates and rubrics in Appendix A.

Key results are as follows: GPT-5-Chat attains the strongest aggregate generative performance (Acc.(Gen) 48.5%, Score(Gen) 2.13), closely followed by Claude-Sonnet-4 (47.9%, 2.12).
DeepSeek-V3 is competitive and achieves the best Score(Gen) on the Social Persona dimension (2.77), despite trailing the leaders on other dimensions.
Compared with discriminative evaluation, generative performance is systematically lower across models, underscoring the added difficulty of free-form persona simulation and the substantial headroom for improvement.

#### 5.4.2 Reliability of the Judge and Human Study

We validate the LLM-as-a-Judge methodology with a human study.
Three expert annotators evaluated a stratified sample of 50 items per judging mode (ranking and scoring), following our instruction set (Appendix [E](https://arxiv.org/html/2510.25536v2#A5)).
Annotators worked independently and were blinded to each other’s labels.

Agreement between the judge and humans is reported in Table [4](https://arxiv.org/html/2510.25536v2#S5.T4) and is comparable to human inter-annotator reliability: for ranking, Cohen’s $\kappa$ is 0.646 (judge vs. human) versus 0.673 (human–human); for scoring, Spearman’s $\rho$ is 0.591 (judge vs. human) versus 0.605 (human–human). These results indicate that the judge is reliable, while the human inter-annotator agreement supports the quality and consistency of our annotation protocol.

**Table 4: Agreement of the judge against human annotations and inter-annotator reliability.**
| Task | Agreement judge vs. human | Inter-annotator reliability |
| --- | --- | --- |
| Ranking (four choice) | 0.646^κ | 0.673^κ |
| Scoring (one to five) | 0.591^ρ | 0.605^ρ |

#### 5.4.3 Text Similarity Metrics

To provide an objective reference, we also evaluate free-form generations with standard text similarity metrics: BLEU-1, METEOR, and BERT-Score. Results are presented in Table [5.4.3](https://arxiv.org/html/2510.25536v2#S5.SS4.SSS3).
Averaged over the three dimensions, Claude-Sonnet-4 attains the best BERT-Score (76.90) and METEOR (18.24), while GPT-5-Chat achieves the best BLEU-1 (19.13).
The resulting model ranking is broadly consistent with our judge-based evaluation.
These metrics serve as complementary evidence, as they emphasize lexical and local semantic overlap rather than opinion alignment, reasoning, or persona tone.

**Table 5: Text similarity metrics for Digital Twin models. We evaluate the generative outputs against the ground truth using three distinct metrics. BLEU-1 $\uparrow$ measures unigram precision. METEOR $\uparrow$ considers precision, recall, and synonymy. BERT-Score $\uparrow$ measures semantic similarity using contextual embeddings. Higher values are better for all metrics. Bold numbers denote the best result and underlined numbers denote the second best in each column.**
| \rowcolorgray!15 | Social Persona | Interpersonal Persona | Narrative Persona | Average |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| \rowcolorgray!15      Model / Tasks | BLEU-1<br>$\uparrow$ | METEOR<br>$\uparrow$ | BERT-<br>Score<br>$\uparrow$ | BLEU-1<br>$\uparrow$ | METEOR<br>$\uparrow$ | BERT-<br>Score<br>$\uparrow$ | BLEU-1<br>$\uparrow$ | METEOR<br>$\uparrow$ | BERT-<br>Score<br>$\uparrow$ | BLEU-1<br>$\uparrow$ | METEOR<br>$\uparrow$ | BERT-<br>Score<br>$\uparrow$ |  |
| LLM | GPT-3.5-Turbo | 16.03 | 15.50 | 62.96 | 24.76 | 22.52 | 81.54 | 12.06 | 12.86 | 84.10 | 17.62 | 16.96 | 76.20 |
| Qwen2.5-14B | 17.68 | 15.38 | 63.25 | 26.09 | 23.76 | 81.57 | 11.67 | 11.92 | 83.99 | 18.48 | 17.02 | 76.27 |  |
| GPT-4o-mini | 15.94 | 15.19 | 62.89 | 23.48 | 21.38 | 81.26 | 12.50 | 13.34 | 84.13 | 17.31 | 16.64 | 76.09 |  |
| GPT-OSS-20B | 14.55 | 12.87 | 61.90 | 20.67 | 19.20 | 81.17 | 10.81 | 10.59 | 84.36 | 15.34 | 14.22 | 75.81 |  |
| DeepSeek-V3 | 16.85 | 15.49 | 63.25 | 26.86 | 25.21 | 82.65 | 11.11 | 11.58 | 84.12 | 18.27 | 17.43 | 76.67 |  |
| GPT-5-Chat | 18.67 | 14.09 | 63.26 | 27.18 | 25.30 | 82.67 | 11.54 | 11.59 | 84.27 | 19.13 | 16.99 | 76.73 |  |
| Claude-Sonnet-4 | 18.68 | 18.14 | 64.19 | 25.22 | 23.45 | 82.14 | 12.38 | 13.12 | 84.37 | 18.76 | 18.24 | 76.90 |  |

### 5.5 Human vs. Model Performance

We benchmark human performance on the Social Persona discriminative task.
Three expert annotators labeled a stratified set of 50 items following our guidelines (Appendix [E](https://arxiv.org/html/2510.25536v2#A5)).
The task involves long contexts and implicit cues, which makes it challenging even for humans.
Therefore, we treat human accuracy as a reference rather than a strict upper bound.

Table [6](https://arxiv.org/html/2510.25536v2#S5.T6) compares models to human baselines.
GPT-5-Chat reaches 0.60 accuracy, below the human mean of 0.64 and the majority-vote aggregate of 0.66.
Agreement with humans is high but short of human–human reliability: Cohen’s $\kappa$ is 0.634 for model vs. human and 0.690 for inter-annotator agreement.

These results indicate that state-of-the-art models approach human reliability on this discriminative formulation yet still trail aggregated human judgments, leaving measurable headroom.

**Table 6: Discriminative evaluation against a reference standard**
| Task | Accuracy | Agreement ($\kappa$) |  |  |  |
| --- | --- | --- | --- | --- | --- |
| GPT-5 | Human mean | Human vote | Model vs human | Inter-annotator |  |
| Discriminative | 0.60 | 0.64 | 0.66 | 0.634 | 0.690 |

Summary of Findings.
Across three persona dimensions and two task formulations, strong models (GPT-5-Chat, Claude-Sonnet-4) lead consistently, yet
generative evaluation
remains notably harder than multiple-choice selection.
Capability analysis pinpoints style control and memory recall as primary bottlenecks, while lexical fidelity and opinion consistency are comparatively robust.
Dimension-wise, performance is generally higher on Narrative than on Social and Interpersonal.

Across tasks, results exhibit substantial variance between models without evident ceiling effects.
There remains clear headroom in three areas: maintaining persona coherence over extended contexts and across sessions, producing a persona-consistent tone, and recalling and using persona-specific facts during generation.

## 6 Conclusions and Discussions

This paper addressed the evaluation of LLM-based persona simulation by introducing TwinVoice.
Built on real-world and fictional data from three dimensions, TwinVoice aims at testing LLMs’ ability in persona simulation by decomposing it into six capabilities of mindset coherence and linguistic expression.
Our extensive evaluation of state-of-the-art models reveals a crucial gap: while leading models like GPT-5-Chat and Claude-Sonnet-4 show improved accuracy over their predecessors, their performance still falls significantly short of human-level capabilities.
We also find that LLMs are adept at mimicking surface-level linguistic styles, they consistently fail to maintain long-term consistency, particularly in memory recall and opinion stability.
By establishing the first fine-grained baselines in this domain, TwinVoice not only exposes the key limitations of current models but also provides a clear roadmap towards personalized AI and digital twins built with LLMs.

Rationale for Three Dimensions.
TwinVoice is constructed based on Social, Interpersonal, and Narrative personas to balance realism, coverage, and privacy.
Social and Interpersonal tracks are built on real interaction traces because evaluating digital twins requires performance in authentic public and private contexts; synthetic or model-generated corpora alone underestimate the difficulty of sustaining identity over long horizons. For Narrative persona, full real-world narrative streams are hard to obtain and raise privacy concerns; we therefore use curated fiction to probe role‑consistent expression under controlled, ethically tractable settings.

Evaluation Design.
Digital twins must go beyond constrained selection to produce persona‑consistent language under open prompts. We therefore pair a discriminative multiple‑choice protocol (with carefully constructed, topically plausible distractors) with a generative protocol that assesses free‑form responses using two LLM‑as‑a‑Judge variants (ranking and scoring) along opinion consistency, logical/factual fidelity, and stylistic similarity. Judge reliability is supported by a human study with three expert annotators: GPT‑5‑as‑a‑Judge reaches agreement close to human inter‑annotator levels (ranking $\kappa\!\approx\!0.646$ vs. $0.673$; scoring $\rho\!\approx\!0.591$ vs. $0.605$).

Usability, Reproducibility, and Robustness.
We release precise task definitions, prompts, and data paths so researchers can plug in fine‑tuning, RAG, long‑term memory, or multi‑agent controllers on the same inputs. For generation, we fix temperature=0.0 and publish decoding settings, seeds, and candidate‑construction scripts; we log model build identifiers where available and release raw outputs to mitigate closed‑API drift. Social Persona derives from PChatbot; to reduce leakage we enforce semantic distinctiveness in choice sets and apply persona–choice alignment filters, and we plan annual refreshes to retire suspect items. With parallelism set to 10, end‑to‑end evaluation per model per dimension completes within 2 hours on our setup.

Coverage and Limitations.
TwinVoice currently spans three dimensions and five languages: Social (Chinese), Interpersonal (English, Spanish, Portuguese, Russian), and Narrative (English). Despite this breadth, language balance within each dimension remains imperfect, and phenomena such as code‑switching and dialectal variation are underrepresented. Future releases will expand per‑dimension language coverage and diversify domains where consented and de‑identified data are available.

Maintenance and Outlook.
We will maintain TwinVoice with annual updates to address potential contamination, accommodate new model behaviors, and extend language and domain coverage. Planned upgrades include longer‑horizon tasks that jointly stress memory and opinion stability, adversarial tone/stance confounders for robustness, and, where ethically permissible, additional dimensions and task types. All releases will be versioned, with code and results publicly available for reproducibility.

## Ethics Statement

We follow standard ethical guidelines for dataset usage, evaluation, and model deployment. All datasets used in this paper are publicly available under their original licenses, and we removed personally identifiable information (PII) where applicable. No human subjects experiments were conducted beyond voluntary annotation; annotators (if any) received fair compensation and provided informed consent.
We prohibit the misuse of our benchmark and models for profiling or harmful decision-making about individuals.
Third-party models/APIs used in our experiments comply with their terms of service. Upon acceptance, we will release our code, prompts, and evaluation scripts with a research license and a model card detailing limitations and appropriate use.

## Reproducibility Statement

We enable independent re-implementation of our evaluation by disclosing all essential ingredients in the paper and appendices:

- •
Prompts & Protocols: Full templates for the discriminative MCQ task, generative persona imitation, and LLM-as-a-Judge (ranking and scoring), together with the 1–5 scoring rubric aligned with opinion, logic/facts, and style.
- •
Data Construction Recipes: Step-by-step textual recipes for all three dimensions, including sources and filtering thresholds (e.g., average reply length $>10$, bottom-20% TTR removal, option cosine similarity $<0.95$ for Social; token-length cleaning $<5$, TF–IDF top-10% selection, and semantic deduplication at $0.90$ for Interpersonal), and the rules used to form distractors.
- •
Dataset Statistics: Per-dimension instance counts and summary statistics as reported in the main text.
- •
Evaluation Definitions: Exact metrics and equations (e.g., Accuracy and $\mathrm{Score}_{\mathrm{gen}}$) used throughout.
- •
Model Usage: The list of model families evaluated and our access window (06/2025–09/2025). We set the decoding temperature to 0 (temperature=0); all other generation hyperparameters (e.g., top_p, max_tokens, presence/frequency penalties) used provider defaults.

All experiments are inference-only (no supervised training). With these disclosed materials, readers can re-implement the pipeline and obtain comparable results under the same inputs and judging criteria.

## References

- Afzoon et al. [2024]
Saleh Afzoon, Usman Naseem, Amin Beheshti, and Zahra Jamali.
Persobench: Benchmarking personalized response generation in large language models.
*arXiv preprint arXiv:2410.03198*, 2024.
- Anthropic [2025]
Anthropic.
Introducing claude 4.
URL: [https://www.anthropic.com/news/claude-4](https://www.anthropic.com/news/claude-4), May 2025.
Official announcement of the Claude 4 model family, including Opus 4 and Sonnet 4.
- Barricelli et al. [2020]
Barbara Rita Barricelli, Elena Casiraghi, Jessica Gliozzo, Alessandro Petrini, and Stefano Valtolina.
Human digital twin for fitness management.
*IEEE Access*, 8:26637–26664, 2020.
[10.1109/ACCESS.2020.2971576](https:/doi.org/10.1109/ACCESS.2020.2971576).
- Baumgartner et al. [2020]
Jason Baumgartner, Savvas Zannettou, Megan Squire, and Jeremy Blackburn.
The pushshift telegram dataset, 2020.
[https://arxiv.org/abs/2001.08438](https://arxiv.org/abs/2001.08438).
- Biber [1991]
Douglas Biber.
*Variation across speech and writing*.
Cambridge university press, 1991.
- Biber [1995]
Douglas Biber.
*Dimensions of register variation: A cross-linguistic comparison*.
Cambridge University Press, 1995.
- Brown [1987]
Penelope Brown.
*Politeness: Some universals in language usage*, volume 4.
Cambridge university press, 1987.
- Bubeck et al. [2023]
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al.
Sparks of artificial general intelligence: Early experiments with gpt-4.
*arXiv preprint arXiv:2303.12712*, 2023.
- Chang et al. [2024]
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al.
A survey on evaluation of large language models.
*ACM transactions on intelligent systems and technology*, 15(3):1–45, 2024.
- Chen et al. [2025]
Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey.
Persona vectors: Monitoring and controlling character traits in language models.
*arXiv preprint (Anthtropic technical report)*, 2025.
- Clark and Brennan [1991]
Herbert H Clark and Susan E Brennan.
Grounding in communication.
1991.
- Costa and McCrae [1992]
Paul T Costa and Robert R McCrae.
Normal personality assessment in clinical practice: The neo personality inventory.
*Psychological assessment*, 4(1):5, 1992.
- Fast et al. [2016]
Ethan Fast, William McGrath, Pranav Rajpurkar, and Michael S Bernstein.
Augur: Mining human behaviors from fiction to power interactive systems.
In *Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems*, pages 237–247, 2016.
- Grieves and Vickers [2017]
Michael Grieves and John Vickers.
Digital twin: Mitigating unpredictable, undesirable emergent behavior in complex systems.
2017.
- Gu et al. [2024]
Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al.
A survey on llm-as-a-judge.
*arXiv preprint arXiv:2411.15594*, 2024.
- Hornik and Rachamim [2025]
Jacob Hornik and Matti Rachamim.
Ai-enabled consumer digital twins as a platform for research aimed at enhancing customer experience.
*Management Review Quarterly*, 05 2025.
[10.1007/s11301-025-00527-3](https:/doi.org/10.1007/s11301-025-00527-3).
- Jiang et al. [2025]
Bowen Jiang, Zhuoqun Hao, Young-Min Cho, Bryan Li, Yuan Yuan, Sihao Chen, Lyle Ungar, Camillo J. Taylor, and Dan Roth.
Know me, respond to me: Benchmarking llms for dynamic user profiling and personalized responses at scale, 2025.
[https://arxiv.org/abs/2504.14225](https://arxiv.org/abs/2504.14225).
- Jones and Bergen [2024]
Cameron Jones and Ben Bergen.
Does gpt-4 pass the turing test?
In *Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)*, pages 5183–5210, 2024.
- Jones and Bergen [2025]
Cameron R Jones and Benjamin K Bergen.
Large language models pass the turing test.
*arXiv preprint arXiv:2503.23674*, 2025.
- Jones et al. [2025]
Cameron Robert Jones, Ishika Rathi, Sydney Taylor, and Benjamin K Bergen.
People cannot distinguish gpt-4 from a human in a turing test.
In *Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency*, pages 1615–1639, 2025.
- Kahneman [2011]
Daniel Kahneman.
*Thinking, fast and slow*.
macmillan, 2011.
- Koppel et al. [2009]
Moshe Koppel, Jonathan Schler, and Shlomo Argamon.
Computational methods in authorship attribution.
*Journal of the American Society for information Science and Technology*, 60(1):9–26, 2009.
- Kumar et al. [2025]
Sai Adith Senthil Kumar, Hao Yan, Saipavan Perepa, Murong Yue, and Ziyu Yao.
Can llms simulate personas with reversed performance? a benchmark for counterfactual instruction following, 2025.
[https://arxiv.org/abs/2504.06460](https://arxiv.org/abs/2504.06460).
- Li et al. [2023]
Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi Mi, Yaying Fei, Xiaoyang Feng, Song Yan, Haosheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, and Haozhen Sun.
Chatharuhi: Reviving anime character in reality via large language model, 2023.
[https://arxiv.org/abs/2308.09597](https://arxiv.org/abs/2308.09597).
- Li et al. [2025a]
Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, and Tat-Seng Chua.
Hello again! llm-powered personalized agent for long-term dialogue, 2025a.
[https://arxiv.org/abs/2406.05925](https://arxiv.org/abs/2406.05925).
- Li et al. [2016]
Jiwei Li, Michel Galley, Chris Brockett, Georgios P Spithourakis, Jianfeng Gao, and Bill Dolan.
A persona-based neural conversation model.
*arXiv preprint arXiv:1603.06155*, 2016.
- Li et al. [2025b]
Rui Li, Heming Xia, Xinfeng Yuan, Qingxiu Dong, Lei Sha, Wenjie Li, and Zhifang Sui.
How far are llms from being our digital twins? a benchmark for persona-based behavior chain simulation, 2025b.
[https://arxiv.org/abs/2502.14642](https://arxiv.org/abs/2502.14642).
- Lin et al. [2024]
Shuhang Lin, Wenyue Hua, Lingyao Li, Che-Jui Chang, Lizhou Fan, Jianchao Ji, Hang Hua, Mingyu Jin, Jiebo Luo, and Yongfeng Zhang.
Battleagent: Multi-modal dynamic emulation on historical battles to complement historical analysis.
*arXiv preprint arXiv:2404.15532*, 2024.
- Ma et al. [2023]
Xiao Ma, Swaroop Mishra, Ariel Liu, Sophie Su, Jilin Chen, Chinmay Kulkarni, Heng-Tze Cheng, Quoc Le, and Ed Chi.
Beyond chatbots: Explorellm for structured thoughts and personalized model responses, 2023.
[https://arxiv.org/abs/2312.00763](https://arxiv.org/abs/2312.00763).
- Mehl et al. [2006]
Matthias R Mehl, Samuel D Gosling, and James W Pennebaker.
Personality in its natural habitat: manifestations and implicit folk theories of personality in daily life.
*Journal of personality and social psychology*, 90(5):862, 2006.
- Neuman [2016]
Yair Neuman.
*Computational personality analysis: Introduction, practical applications and novel directions*.
Springer, 2016.
- OpenAI [2025]
OpenAI.
Introducing gpt-5.
URL: [https://openai.com/index/introducing-gpt-5/](https://openai.com/index/introducing-gpt-5/), August 2025.
Official announcement of the GPT-5 model, a unified system with built-in reasoning capabilities.
- Park et al. [2023]
Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein.
Generative agents: Interactive simulacra of human behavior.
In *Proceedings of the 36th annual acm symposium on user interface software and technology*, pages 1–22, 2023.
- Pennebaker et al. [2003]
James W Pennebaker, Matthias R Mehl, and Kate G Niederhoffer.
Psychological aspects of natural language use: Our words, our selves.
*Annual review of psychology*, 54(1):547–577, 2003.
- Project Gutenberg [1971–]
Project Gutenberg.
Project gutenberg.
[https://www.gutenberg.org](https://www.gutenberg.org), 1971–.
- Qian et al. [2021]
Hongjin Qian, Xiaohe Li, Hanxun Zhong, Yu Guo, Yueyuan Ma, Yutao Zhu, Zhanliang Liu, Zhicheng Dou, and Ji-Rong Wen.
Pchatbot: a large-scale dataset for personalized chatbot.
In *Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval*, pages 2470–2477, 2021.
- Ran et al. [2025]
Yiting Ran, Xintao Wang, Tian Qiu, Jiaqing Liang, Yanghua Xiao, and Deqing Yang.
Bookworld: From novels to interactive agent societies for creative story generation, 2025.
[https://arxiv.org/abs/2504.14538](https://arxiv.org/abs/2504.14538).
- Rossetti et al. [2024]
Giulio Rossetti, Massimo Stella, Rémy Cazabet, Katherine Abramski, Erica Cau, Salvatore Citraro, Andrea Failla, Riccardo Improta, Virginia Morini, and Valentina Pansanella.
Y social: an llm-powered social media digital twin, 2024.
[https://arxiv.org/abs/2408.00818](https://arxiv.org/abs/2408.00818).
- Shanahan et al. [2023]
Murray Shanahan, Kyle McDonell, and Laria Reynolds.
Role play with large language models.
*Nature*, 623(7987):493–498, 2023.
- Shang et al. [2024]
Jingbo Shang, Zai Zheng, Jiale Wei, Xiang Ying, Felix Tao, and Mindverse Team.
Ai-native memory: A pathway from llms towards agi, 2024.
[https://arxiv.org/abs/2406.18312](https://arxiv.org/abs/2406.18312).
- Shen et al. [2023]
Tianhao Shen, Sun Li, Quan Tu, and Deyi Xiong.
Roleeval: A bilingual role evaluation benchmark for large language models.
*arXiv preprint arXiv:2312.16132*, 2023.
- Shi et al. [2025a]
Quan Shi, Carlos E. Jimenez, Stephen Dong, Brian Seo, Caden Yao, Adam Kelch, and Karthik Narasimhan.
Impersona: Evaluating individual level lm impersonation, 2025a.
[https://arxiv.org/abs/2504.04332](https://arxiv.org/abs/2504.04332).
- Shi et al. [2025b]
Yunxiao Shi, Wujiang Xu, Zeqi Zhang, Xing Zi, Qiang Wu, and Min Xu.
Personax: A recommendation agent oriented user modeling framework for long behavior sequence.
*arXiv preprint arXiv:2503.02398*, 2025b.
- Stamatatos [2009]
Efstathios Stamatatos.
A survey of modern authorship attribution methods.
*Journal of the American Society for information Science and Technology*, 60(3):538–556, 2009.
- Toubia et al. [2025]
Olivier Toubia, George Z. Gui, Tianyi Peng, Daniel J. Merlau, Ang Li, and Haozhe Chen.
Twin-2k-500: A dataset for building digital twins of over 2,000 people based on their answers to over 500 questions, 2025.
[https://arxiv.org/abs/2505.17479](https://arxiv.org/abs/2505.17479).
- Tu et al. [2024]
Quan Tu, Shilong Fan, Zihang Tian, and Rui Yan.
Charactereval: A chinese benchmark for role-playing conversational agent evaluation.
*arXiv preprint arXiv:2401.01275*, 2024.
- Wei et al. [2022]
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al.
Emergent abilities of large language models.
*arXiv preprint arXiv:2206.07682*, 2022.
- Xie et al. [2025]
Qiujie Xie, Qiming Feng, Tianqi Zhang, Qingqiu Li, Linyi Yang, Yuejie Zhang, Rui Feng, Liang He, Shang Gao, and Yue Zhang.
Human simulacra: Benchmarking the personification of large language models.
In *The Thirteenth International Conference on Learning Representations*, 2025.
[https://openreview.net/forum?id=BCP5nAHXqs](https://openreview.net/forum?id=BCP5nAHXqs).
- Xu et al. [2024]
Rui Xu, Xintao Wang, Jiangjie Chen, Siyu Yuan, Xinfeng Yuan, Jiaqing Liang, Zulong Chen, Xiaoqing Dong, and Yanghua Xiao.
Character is destiny: Can role-playing language agents make persona-driven decisions?, 2024.
[https://arxiv.org/abs/2404.12138](https://arxiv.org/abs/2404.12138).
- Ye et al. [2025]
Ziyi Ye, Xiangsheng Li, Qiuchi Li, Qingyao Ai, Yujia Zhou, Wei Shen, Dong Yan, and Yiqun Liu.
Learning llm-as-a-judge for preference alignment.
In *The Thirteenth International Conference on Learning Representations*, 2025.
- Zaller [1992]
John Zaller.
*The nature and origins of mass opinion*.
Cambridge university press, 1992.
- [52]
Jialing Zhang, Lingfeng Zhou, Jin Gao, Mohan Jiang, and Dequan Wang.
Personaeval: Benchmarking llms on role-playing evaluation tasks.
- Zhou et al. [2025]
Lingfeng Zhou, Jialing Zhang, Jin Gao, Mohan Jiang, and Dequan Wang.
Personaeval: Are llm evaluators human enough to judge role-play?, 2025.
[https://arxiv.org/abs/2508.10014](https://arxiv.org/abs/2508.10014).

## Appendix A Evaluation Protocols and Full Prompts

This appendix details our evaluation protocols and the full instruction templates used across multiple data forms, including public social interactions, interpersonal messaging, and narrative dialogue. We adopt a unified instruction design and provide template variants for different data shapes when needed. Unless otherwise noted, the LLM-as-a-Judge component is instantiated with GPT-5.

### A.1 Scope and Alignment with Competencies

Our evaluation comprises (1) discriminative multiple-choice selection and (2) generative evaluation, including persona imitation (free-form generation) and LLM-as-a-Judge assessment via ranking and scoring. The judge scoring rubric is organized along three pillars—Opinion Consistency, Logical & Factual Fidelity, and Stylistic Similarity—which align with the six fundamental capabilities defined in the main text. We offer equivalent template variants per evaluation mode to fit different data shapes; metrics and scoring criteria remain identical across variants.

### A.2 Unifying Instructions and Placeholders

We use a single instruction family per evaluation mode. Differences are limited to how inputs are presented. We standardize placeholders as follows:

- •
{history}: persona-establishing prior content by the same user or character.
- •
{context}: the situation/post/message/scene the user or character is responding to (replacing earlier {anchor} or {anchor_post}).
- •
{ground_truth_reply} or {groundtruth_response}: the human-written reply.
- •
{lmut_reply} or {generated_content}: the model-generated reply to be evaluated.

### A.3 Discriminative Evaluation (Multiple-Choice Selection)

##### Canonical template (General)

##### Alternative template (Dimension 2: Interpersonal Messaging).

##### Distractor Generation for Discriminative Data (Dimension 3: Narrative).

##### Notes.

- •
Placeholders are standardized: {history}, {context}, and option texts {a}, {b}, {c}, {d}. In narrative data, the distractor writer prompt is used to construct options and is not itself a judging template.

### A.4 Generative Evaluation: Persona Imitation (Free-form Generation)

##### Canonical template (General, text-only output)

##### Variant (Dimension 2: Messaging, JSON output).

##### Variant (Dimension 3: Narrative, single-line JSON).

##### Notes.

- •
Use {context} as the reply trigger across all variants. The narrative variant mandates a single-line JSON output.

### A.5 LLM-as-a-Judge: Ranking-Based Evaluation

##### Canonical template (JSON + concise reasoning).

##### Letter-only MAP Prompt (Dimension 3: Narrative).

##### Notes.

- •
Ranking supports two outputs: a JSON object with brief reasoning (general) and a letter-only output (narrative variant).

### A.6 LLM-as-a-Judge: Scoring-Based Evaluation

##### Canonical template (applies as-is).

##### Notes.

- •
Inputs are standardized as {history}, {context}, {ground_truth_reply} (or {groundtruth_response}), and {lmut_reply} (or {generated_content}).

### A.7 Implementation Note: Judge Model

We instantiate the LLM-as-a-Judge with GPT-5 for both ranking- and scoring-based evaluation, unless otherwise specified. Ranking includes a letter-only variant for narrative data.

## Appendix B Capability Annotation Prompts and Labeling Protocol

We annotate each example to identify which capability a model must primarily exercise to replicate a user’s reply, while also recording the presence of all six capabilities. Each annotation unit contains three elements: {history} (persona-establishing prior content), {context} (the situation the user is replying to), and {groundtruth_response} (the user’s actual reply). An expert LLM performs the annotation to ensure consistency and structured outputs (we use GPT-5 with temperature set to 0).

##### Canonical Annotation Prompt.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

##### Alternative template (Dimension 2: Interpersonal Messaging).

##### Distractor Generation for Discriminative Data (Dimension 3: Narrative).

##### Notes.

- •
Placeholders are standardized: {history}, {context}, and option texts {a}, {b}, {c}, {d}. In narrative data, the distractor writer prompt is used to construct options and is not itself a judging template.

### A.4 Generative Evaluation: Persona Imitation (Free-form Generation)

##### Canonical template (General, text-only output)

##### Variant (Dimension 2: Messaging, JSON output).

##### Variant (Dimension 3: Narrative, single-line JSON).

##### Notes.

- •
Use {context} as the reply trigger across all variants. The narrative variant mandates a single-line JSON output.

### A.5 LLM-as-a-Judge: Ranking-Based Evaluation

##### Canonical template (JSON + concise reasoning).

##### Letter-only MAP Prompt (Dimension 3: Narrative).

##### Notes.

- •
Ranking supports two outputs: a JSON object with brief reasoning (general) and a letter-only output (narrative variant).

### A.6 LLM-as-a-Judge: Scoring-Based Evaluation

##### Canonical template (applies as-is).

##### Notes.

- •
Inputs are standardized as {history}, {context}, {ground_truth_reply} (or {groundtruth_response}), and {lmut_reply} (or {generated_content}).

### A.7 Implementation Note: Judge Model

We instantiate the LLM-as-a-Judge with GPT-5 for both ranking- and scoring-based evaluation, unless otherwise specified. Ranking includes a letter-only variant for narrative data.

## Appendix B Capability Annotation Prompts and Labeling Protocol

We annotate each example to identify which capability a model must primarily exercise to replicate a user’s reply, while also recording the presence of all six capabilities. Each annotation unit contains three elements: {history} (persona-establishing prior content), {context} (the situation the user is replying to), and {groundtruth_response} (the user’s actual reply). An expert LLM performs the annotation to ensure consistency and structured outputs (we use GPT-5 with temperature set to 0).

##### Canonical Annotation Prompt.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

##### Distractor Generation for Discriminative Data (Dimension 3: Narrative).

##### Notes.

- •
Placeholders are standardized: {history}, {context}, and option texts {a}, {b}, {c}, {d}. In narrative data, the distractor writer prompt is used to construct options and is not itself a judging template.

### A.4 Generative Evaluation: Persona Imitation (Free-form Generation)

##### Canonical template (General, text-only output)

##### Variant (Dimension 2: Messaging, JSON output).

##### Variant (Dimension 3: Narrative, single-line JSON).

##### Notes.

- •
Use {context} as the reply trigger across all variants. The narrative variant mandates a single-line JSON output.

### A.5 LLM-as-a-Judge: Ranking-Based Evaluation

##### Canonical template (JSON + concise reasoning).

##### Letter-only MAP Prompt (Dimension 3: Narrative).

##### Notes.

- •
Ranking supports two outputs: a JSON object with brief reasoning (general) and a letter-only output (narrative variant).

### A.6 LLM-as-a-Judge: Scoring-Based Evaluation

##### Canonical template (applies as-is).

##### Notes.

- •
Inputs are standardized as {history}, {context}, {ground_truth_reply} (or {groundtruth_response}), and {lmut_reply} (or {generated_content}).

### A.7 Implementation Note: Judge Model

We instantiate the LLM-as-a-Judge with GPT-5 for both ranking- and scoring-based evaluation, unless otherwise specified. Ranking includes a letter-only variant for narrative data.

## Appendix B Capability Annotation Prompts and Labeling Protocol

We annotate each example to identify which capability a model must primarily exercise to replicate a user’s reply, while also recording the presence of all six capabilities. Each annotation unit contains three elements: {history} (persona-establishing prior content), {context} (the situation the user is replying to), and {groundtruth_response} (the user’s actual reply). An expert LLM performs the annotation to ensure consistency and structured outputs (we use GPT-5 with temperature set to 0).

##### Canonical Annotation Prompt.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

##### Notes.

- •
Placeholders are standardized: {history}, {context}, and option texts {a}, {b}, {c}, {d}. In narrative data, the distractor writer prompt is used to construct options and is not itself a judging template.

### A.4 Generative Evaluation: Persona Imitation (Free-form Generation)

##### Canonical template (General, text-only output)

##### Variant (Dimension 2: Messaging, JSON output).

##### Variant (Dimension 3: Narrative, single-line JSON).

##### Notes.

- •
Use {context} as the reply trigger across all variants. The narrative variant mandates a single-line JSON output.

### A.5 LLM-as-a-Judge: Ranking-Based Evaluation

##### Canonical template (JSON + concise reasoning).

##### Letter-only MAP Prompt (Dimension 3: Narrative).

##### Notes.

- •
Ranking supports two outputs: a JSON object with brief reasoning (general) and a letter-only output (narrative variant).

### A.6 LLM-as-a-Judge: Scoring-Based Evaluation

##### Canonical template (applies as-is).

##### Notes.

- •
Inputs are standardized as {history}, {context}, {ground_truth_reply} (or {groundtruth_response}), and {lmut_reply} (or {generated_content}).

### A.7 Implementation Note: Judge Model

We instantiate the LLM-as-a-Judge with GPT-5 for both ranking- and scoring-based evaluation, unless otherwise specified. Ranking includes a letter-only variant for narrative data.

## Appendix B Capability Annotation Prompts and Labeling Protocol

We annotate each example to identify which capability a model must primarily exercise to replicate a user’s reply, while also recording the presence of all six capabilities. Each annotation unit contains three elements: {history} (persona-establishing prior content), {context} (the situation the user is replying to), and {groundtruth_response} (the user’s actual reply). An expert LLM performs the annotation to ensure consistency and structured outputs (we use GPT-5 with temperature set to 0).

##### Canonical Annotation Prompt.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

##### Variant (Dimension 2: Messaging, JSON output).

##### Variant (Dimension 3: Narrative, single-line JSON).

##### Notes.

- •
Use {context} as the reply trigger across all variants. The narrative variant mandates a single-line JSON output.

### A.5 LLM-as-a-Judge: Ranking-Based Evaluation

##### Canonical template (JSON + concise reasoning).

##### Letter-only MAP Prompt (Dimension 3: Narrative).

##### Notes.

- •
Ranking supports two outputs: a JSON object with brief reasoning (general) and a letter-only output (narrative variant).

### A.6 LLM-as-a-Judge: Scoring-Based Evaluation

##### Canonical template (applies as-is).

##### Notes.

- •
Inputs are standardized as {history}, {context}, {ground_truth_reply} (or {groundtruth_response}), and {lmut_reply} (or {generated_content}).

### A.7 Implementation Note: Judge Model

We instantiate the LLM-as-a-Judge with GPT-5 for both ranking- and scoring-based evaluation, unless otherwise specified. Ranking includes a letter-only variant for narrative data.

## Appendix B Capability Annotation Prompts and Labeling Protocol

We annotate each example to identify which capability a model must primarily exercise to replicate a user’s reply, while also recording the presence of all six capabilities. Each annotation unit contains three elements: {history} (persona-establishing prior content), {context} (the situation the user is replying to), and {groundtruth_response} (the user’s actual reply). An expert LLM performs the annotation to ensure consistency and structured outputs (we use GPT-5 with temperature set to 0).

##### Canonical Annotation Prompt.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

##### Variant (Dimension 3: Narrative, single-line JSON).

##### Notes.

- •
Use {context} as the reply trigger across all variants. The narrative variant mandates a single-line JSON output.

### A.5 LLM-as-a-Judge: Ranking-Based Evaluation

##### Canonical template (JSON + concise reasoning).

##### Letter-only MAP Prompt (Dimension 3: Narrative).

##### Notes.

- •
Ranking supports two outputs: a JSON object with brief reasoning (general) and a letter-only output (narrative variant).

### A.6 LLM-as-a-Judge: Scoring-Based Evaluation

##### Canonical template (applies as-is).

##### Notes.

- •
Inputs are standardized as {history}, {context}, {ground_truth_reply} (or {groundtruth_response}), and {lmut_reply} (or {generated_content}).

### A.7 Implementation Note: Judge Model

We instantiate the LLM-as-a-Judge with GPT-5 for both ranking- and scoring-based evaluation, unless otherwise specified. Ranking includes a letter-only variant for narrative data.

## Appendix B Capability Annotation Prompts and Labeling Protocol

We annotate each example to identify which capability a model must primarily exercise to replicate a user’s reply, while also recording the presence of all six capabilities. Each annotation unit contains three elements: {history} (persona-establishing prior content), {context} (the situation the user is replying to), and {groundtruth_response} (the user’s actual reply). An expert LLM performs the annotation to ensure consistency and structured outputs (we use GPT-5 with temperature set to 0).

##### Canonical Annotation Prompt.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

##### Notes.

- •
Use {context} as the reply trigger across all variants. The narrative variant mandates a single-line JSON output.

### A.5 LLM-as-a-Judge: Ranking-Based Evaluation

##### Canonical template (JSON + concise reasoning).

##### Letter-only MAP Prompt (Dimension 3: Narrative).

##### Notes.

- •
Ranking supports two outputs: a JSON object with brief reasoning (general) and a letter-only output (narrative variant).

### A.6 LLM-as-a-Judge: Scoring-Based Evaluation

##### Canonical template (applies as-is).

##### Notes.

- •
Inputs are standardized as {history}, {context}, {ground_truth_reply} (or {groundtruth_response}), and {lmut_reply} (or {generated_content}).

### A.7 Implementation Note: Judge Model

We instantiate the LLM-as-a-Judge with GPT-5 for both ranking- and scoring-based evaluation, unless otherwise specified. Ranking includes a letter-only variant for narrative data.

## Appendix B Capability Annotation Prompts and Labeling Protocol

We annotate each example to identify which capability a model must primarily exercise to replicate a user’s reply, while also recording the presence of all six capabilities. Each annotation unit contains three elements: {history} (persona-establishing prior content), {context} (the situation the user is replying to), and {groundtruth_response} (the user’s actual reply). An expert LLM performs the annotation to ensure consistency and structured outputs (we use GPT-5 with temperature set to 0).

##### Canonical Annotation Prompt.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

##### Letter-only MAP Prompt (Dimension 3: Narrative).

##### Notes.

- •
Ranking supports two outputs: a JSON object with brief reasoning (general) and a letter-only output (narrative variant).

### A.6 LLM-as-a-Judge: Scoring-Based Evaluation

##### Canonical template (applies as-is).

##### Notes.

- •
Inputs are standardized as {history}, {context}, {ground_truth_reply} (or {groundtruth_response}), and {lmut_reply} (or {generated_content}).

### A.7 Implementation Note: Judge Model

We instantiate the LLM-as-a-Judge with GPT-5 for both ranking- and scoring-based evaluation, unless otherwise specified. Ranking includes a letter-only variant for narrative data.

## Appendix B Capability Annotation Prompts and Labeling Protocol

We annotate each example to identify which capability a model must primarily exercise to replicate a user’s reply, while also recording the presence of all six capabilities. Each annotation unit contains three elements: {history} (persona-establishing prior content), {context} (the situation the user is replying to), and {groundtruth_response} (the user’s actual reply). An expert LLM performs the annotation to ensure consistency and structured outputs (we use GPT-5 with temperature set to 0).

##### Canonical Annotation Prompt.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

##### Notes.

- •
Ranking supports two outputs: a JSON object with brief reasoning (general) and a letter-only output (narrative variant).

### A.6 LLM-as-a-Judge: Scoring-Based Evaluation

##### Canonical template (applies as-is).

##### Notes.

- •
Inputs are standardized as {history}, {context}, {ground_truth_reply} (or {groundtruth_response}), and {lmut_reply} (or {generated_content}).

### A.7 Implementation Note: Judge Model

We instantiate the LLM-as-a-Judge with GPT-5 for both ranking- and scoring-based evaluation, unless otherwise specified. Ranking includes a letter-only variant for narrative data.

## Appendix B Capability Annotation Prompts and Labeling Protocol

We annotate each example to identify which capability a model must primarily exercise to replicate a user’s reply, while also recording the presence of all six capabilities. Each annotation unit contains three elements: {history} (persona-establishing prior content), {context} (the situation the user is replying to), and {groundtruth_response} (the user’s actual reply). An expert LLM performs the annotation to ensure consistency and structured outputs (we use GPT-5 with temperature set to 0).

##### Canonical Annotation Prompt.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

##### Notes.

- •
Inputs are standardized as {history}, {context}, {ground_truth_reply} (or {groundtruth_response}), and {lmut_reply} (or {generated_content}).

### A.7 Implementation Note: Judge Model

We instantiate the LLM-as-a-Judge with GPT-5 for both ranking- and scoring-based evaluation, unless otherwise specified. Ranking includes a letter-only variant for narrative data.

## Appendix B Capability Annotation Prompts and Labeling Protocol

We annotate each example to identify which capability a model must primarily exercise to replicate a user’s reply, while also recording the presence of all six capabilities. Each annotation unit contains three elements: {history} (persona-establishing prior content), {context} (the situation the user is replying to), and {groundtruth_response} (the user’s actual reply). An expert LLM performs the annotation to ensure consistency and structured outputs (we use GPT-5 with temperature set to 0).

##### Canonical Annotation Prompt.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

##### Inputs for the prompt.

We pass a single JSON object per example with three keys: history, context, and groundtruth_response. No length truncation is applied.

## Appendix C Capability Distinguishing Case Studies

This section presents case studies that illustrate how our six capabilities appear in practice. The examples are drawn from our public social persona corpus (dimension 1). For readability we show faithful translations and only the key slices. If any discrepancy arises, the original Chinese dataset is authoritative. Explanatory remarks appear outside the boxes. Inside each box, ······ marks omitted portions of longer cases.

### C.1 Opinion Consistency

The user maintains a specific stance across contexts, namely choosing shows based on a favorite actor and praising acting skill. The new reply preserves this granular stance rather than defaulting to generic positivity.

Why this shows Opinion Consistency: The historical pattern is watch for a specific actor and praise that acting. The ground truth reply mirrors the same stance toward another named actor, preserving topic granularity and evaluative angle.

### C.2 Memory Recall

The reply uses a nickname that is not introduced in the immediate context, presupposing shared knowledge from prior interactions. Understanding the line fully requires recalling who that nickname refers to.

Why this shows Memory Recall: The affectionate nickname Wang Sansui is not grounded in the current context and relies on earlier persona knowledge to resolve the reference.

### C.3 Logical Reasoning

The user’s pattern is Observation then Deduction. In history, a physical observation supports an inference. The reply replicates this approach by citing scene features to argue against an assumption.

Why this shows Logical Reasoning: The reply marshals concrete observations (narrow channel, road nearby) to support a causal judgment, matching the user’s habit of evidence based inference.

### C.4 Lexical Fidelity

A personal catchphrase recurs across contexts. The reply deploys the same idiosyncratic exclamation seen in history, signaling a learned lexical signature.

Why this shows Lexical Fidelity: The same colloquial exclamation equivalent to OMG appears in both history and reply, demonstrating consistent, user specific lexical choice.

### C.5 Persona Tone

The user favors playful hyperbole and adoring expressions that are nonliteral. The reply echoes that tone with a different bodily metaphor, preserving the same stylistic stance.

Why this shows Persona Tone: Both history and reply use exuberant, nonliteral bodily metaphors (ears pregnant, nosebleed) as playful, adoring exaggerations that define the user’s persona.

### C.6 Syntactic Style

Beyond words and tone, the user’s structure features stacked, breathless exclamations with intensifiers. The reply reproduces that sentence shape.

Why this shows Syntactic Style: The reply stacks short, exclamatory clauses with intensifiers and colloquial particles, recreating the user’s distinctive, breathless rhythm observed in history.

## Appendix D Radar Charts across Three Dimensions

We present capability-wise radar charts for the three persona dimensions: Dimension 1 (Social Persona), Dimension 2 (Interpersonal Persona), and Dimension 3 (Narrative Persona).
For each dimension, we report four evaluation configurations: (i) Combined Average (aggregated across protocols), (ii) Discriminative (multiple-choice selection), (iii) Generative Ranking (LLM-as-a-Judge; Acc.(Gen)), and (iv) Generative Scoring (LLM-as-a-Judge; Score(Gen), 1–5).
Each radar covers six capabilities: Opinion Consistency, Memory Recall, Logical Reasoning, Lexical Fidelity, Persona Tone, and Syntactic Style.

### D.1 Social Persona (Dimension 1)

Figure: (a) Dimension 1 (Social Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x4.png

### D.2 Interpersonal Persona (Dimension 2)

Figure: (e) Dimension 2 (Interpersonal Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x8.png

### D.3 Narrative Persona (Dimension 3)

Figure: (i) Dimension 3 (Narrative Persona): Combined Average radar over six capabilities (all labeled capabilities). Aggregates across discriminative and generative protocols; higher is better along each spoke.
Refer to caption: x12.png

## Appendix E Human Annotation Guidelines

### E.1 Task Background and Objectives

This study aims to evaluate the performance of Large Language Models (LLMs) as judges in digital twin tasks. To validate the reliability of model judgments, we need human annotators to independently annotate selected data to establish a trustworthy benchmark.

The annotation task consists of three subtasks corresponding to different evaluation modes: discriminative tasks, generative ranking tasks, and generative scoring tasks. Each annotator will annotate the same 50 data samples to ensure consistency and comparability in evaluation.

Important Note: All provided content (anchor posts, reply history, choices) is in Chinese. You should analyze and understand the content within the Chinese language context, but your reasoning and annotations should be provided in English when specified.

### E.2 Discriminative Task Annotation

#### E.2.1 Task Description

In the discriminative task, you need to act as a specific social media user, becoming their digital twin. Based on the given conversation history and anchor post, select the most appropriate reply from four candidates that best matches the user’s personal style and language habits.

#### E.2.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task. Please follow the same reasoning approach:

> Your task is to act as a specific social media user, becoming their digital twin. Note: All provided text (history, post, choices) is in Chinese. You must analyze the user’s style directly within the Chinese language context. Based on the user’s reply history, think and respond with their mindset, tone, and style. Your reply history: (Note: "AnchorPost" is another user’s post, and "UserReply" is your own reply.) Now, you see a new post: [anchor post] Below are 4 candidate replies. Which one is most likely something you would say? Please respond by explaining your choice from the user’s perspective using "I".

#### E.2.3 Evaluation Criteria

- •
Style Consistency: Does the reply maintain consistency with the user’s language style demonstrated in conversation history?
- •
Tone Matching: Does the reply’s tone (formal/informal, humorous/serious, etc.) match the user’s characteristics?
- •
Vocabulary Usage: Are the vocabulary choices and expressions consistent with the user’s habits?
- •
Logical Coherence: Is the reply content logically related to the anchor post and historical context?

#### E.2.4 Additional Human Guidance

- •
Carefully read through the entire conversation history to understand the user’s communication patterns
- •
Pay attention to recurring phrases, greeting patterns, and emotional expressions
- •
Consider the user’s typical response length and level of detail
- •
Think from the user’s perspective: "If I were this user, which response would I most likely choose?"

#### E.2.5 Annotation Method

Please fill in the option number (0, 1, 2, or 3) that you consider most appropriate in the human_choice field, corresponding to the index position in the choices array.

### E.3 Generative Ranking Task Annotation

#### E.3.1 Task Description

In the generative ranking task, you need to identify which candidate reply is most similar to a reference reply in terms of style, tone, vocabulary, sentiment, and topic.

#### E.3.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following prompt for this task:

> You are an expert evaluator of writing style. Your task is to compare several candidate replies against a known "Reference Reply" written by a specific user. Your goal is to identify which candidate is the most similar to the reference in terms of style, tone, vocabulary, sentiment, and topic. Now, determine which single candidate is the closest match to the Reference Reply. The reasoning should be concise, limited to 2-3 sentences, focusing on the stylistic similarities.

#### E.3.3 Evaluation Criteria

- •
Style Similarity: Lexical choices, sentence structure, formality level
- •
Tone Matching: Emotional tone, attitude, and mood
- •
Vocabulary Consistency: Use of similar words, phrases, or expressions
- •
Sentiment Alignment: Overall emotional orientation and sentiment
- •
Topic Relevance: Relevance and approach to the main topic

#### E.3.4 Additional Human Guidance

- •
Focus on stylistic elements rather than factual content
- •
Look for subtle language patterns and preferences
- •
Consider both what is said and how it is said
- •
Compare the "voice" and "personality" reflected in each candidate

#### E.3.5 Annotation Method

Please fill in the letter (A, B, C, or D) of the option you consider best matching in the human_choice field.

### E.4 Generative Scoring Task Annotation

#### E.4.1 Task Description

In the generative scoring task, you need to assess how well a generated reply replicates a ground truth reply, providing a score from 1-5 based on comprehensive evaluation criteria.

#### E.4.2 LLM Prompt (Use the Same Evaluation Standard)

The LLM uses the following detailed evaluation framework:

> You are a meticulous and objective evaluator for a digital twin benchmark. Your task is to assess how well a ’Generated Reply’ replicates a ’Ground Truth Reply’ for a given social media post. The evaluation rests on three key pillars: 1. Opinion Consistency: Does the Generated Reply express the exact same core opinion, stance, and sentiment as the Ground Truth? 2. Logical & Factual Fidelity: Is the Generated Reply based on the same reasoning and facts as the Ground Truth? 3. Stylistic Similarity: How closely does the Generated Reply match the Ground Truth in terms of lexical, tone, and syntactic elements?

#### E.4.3 Scoring Rubric (1-5 Scale)

- •
5 - Perfect Replication: Perfect match across all three pillars. Feels like a natural, alternative expression from the same user.
- •
4 - High Fidelity: Opinion and Logic/Factual pillars are perfectly matched. Only minor, subtle differences in Style.
- •
3 - Core Alignment, Detail Loss: Core opinion is consistent, but noticeable loss of detail in Logic or Style pillars.
- •
2 - Partial Relevance, Major Deviation: Major failure in at least one of the three pillars.
- •
1 - Irrelevant or Contradictory: Almost nothing in common with the Ground Truth or expresses contradictory opinion.

#### E.4.4 Additional Human Guidance

- •
First identify the core opinion/stance in the ground truth reply
- •
Check if the generated reply maintains the same logical flow and reasoning
- •
Evaluate stylistic elements: word choice, sentence length, formality, emotional tone
- •
Consider the reply as a whole - would it serve as an acceptable substitute?
- •
Be objective and consistent across all annotations

#### E.4.5 Annotation Method

Please fill in your score (1, 2, 3, 4, or 5) in the human_score field.

### E.5 General Guidelines and Notes

#### E.5.1 Quality Assurance

- •
Read all conversation history carefully to understand the user’s communication patterns
- •
Maintain objectivity and consistency throughout the annotation process
- •
Avoid letting personal preferences influence your judgment
- •
Each data sample should be annotated independently
- •
When facing difficult decisions, choose the relatively best option
- •
Double-check for missing annotations or format errors after completion

#### E.5.2 Language Considerations

- •
All content is in Chinese - analyze within the Chinese language context
- •
Pay attention to Chinese-specific expressions, internet slang, and cultural references
- •
Consider Chinese punctuation and writing conventions
- •
Understand the social media context and communication norms

## Appendix F Use of Large Language Models

### F.1 Scope of Use

LLMs assisted with (i) prompt drafting and refinement, (ii) minor code refactoring suggestions,
(iii) generating synthetic evaluation items (e.g., distractor options and candidate responses),
and (iv) light copy-editing of non-technical prose. LLMs did *not* originate novel claims,
conduct final analyses, or decide conclusions; all substantive results are author-verified.

### F.2 Models and Access

We used the following LLMs via API/local inference:
GPT-5-Chat (OpenAI), Claude-Sonnet-4 (Anthropic),
DeepSeek-V3 (DeepSeek), GPT-4o-mini (OpenAI),
GPT-3.5-Turbo (OpenAI), GPT-OSS-20B (OpenAI),
Qwen2.5-14B (Alibaba / Qwen Team).
Access window: 06/2025–09/2025.

### F.3 Human Oversight

All LLM outputs were screened by the authors; items entering quantitative evaluation were validated via
deterministic scripts or double review.

### F.4 Reproducibility

We include the full evaluation prompts and protocols, the 1–5 scoring rubric, the textual recipes for constructing multiple-choice questions, the data filtering thresholds per dimension, dataset sizes/statistics, and the evaluation equations and metrics. These disclosures are sufficient to re-implement our evaluation.

### F.5 Data Privacy and Safety

Only public data were processed; no PII or sensitive user data were sent to third-party services.
We complied with provider Terms of Service and applied toxicity/safety filters where applicable.

### F.6 Limitations

LLM outputs may reflect training-data biases or hallucinations. We mitigated these via rule-based validators and manual review; residual errors may remain.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 4
presentation: 2
questions: ## Question A. Experimental method

A1. Why did the authors selected those seven models? What is the main criteria of selection?
A1-1. If there exists some criterion, could the authors link the findings with those criteria? Are there any factors affected the result?

A2. The authors used GPT-5 for LLM-as-a-judge and GPT-5-Chat for answer generation. As the model stems from the same training data distribution, the judge might prefer its family which can show a similar generation behavior to them. Why did the authors selected same model for both judge and generation?
A2-1. Does this selection affect the result? How?

A3. The authors used Gutenberg project, which is a very well known corpus in the community. As the corpus was commonly used for training language models, it is highly likely that the models already learned the corpus. Doesn't this possibly affect the result for narrative aspect?
A3-1. Can we check whether the model already learned the corpus, using the methods for checking data contamination?

A4. This is a minor but could highly affect the result on human evaluation. The authors mentioned that they used three annotators. But they used Cohen's kappa, which is limited to two annotators because of its statistical assumption. In the case of three or more annotators, researchers usually adopt Fleiss' kappa, which is an extension of Cohen's kappa. So the question is that how did the authors provide the result of kappa? Were the authors averaged the result of all pairs?
A4-1. For more statistically sound result, could the authors provide Fleiss' kappa instead of Cohen's?

A5. Though the authors used temperature 0, it does not ensure deterministic behavior. Did the authors run the experiment multiple times? If so, could the authors provide the statistical errors or other statistical comparison results to support their discussion?
A5-1. It could be better provide the result of statistical tests.

A6. Lastly, how the authors provide history to generate chat in interpersonal experiment? Did the authors input all elements in the history?
A6-1. Studies in persona-based dialogue generation usually use RAG or summarization systems to compact the history. If the authors used the entire history without summarization, can the difference affect the result? How?

## Question B. Previous work

B1. This question is in the same line with Question A6. Previous methods on persona-based dialogue generation use different methods. As these methods might affect the result, the authors should shortly and clearly state the difference between their method and the recent advances. For example, please refer to [1].

[1] Saber Zerhoudi et al., PersonaRAG: Enhancing Retrieval-Augmented Generation Systems with User-Centric Agents, https://arxiv.org/abs/2407.09394

B2. Previously, researchers have discussed specific capabilities. What is the difference between this work and theirs, except for building a summative framework? For example, please refer to [1], [2], [3], and [4]. Note that these are few of related studies; there are lot of relevant studies.
B2-1. Could the findings be connected with these studies?

[2] Junhyuk Choi et al., Examining Identity Drift in Conversations of LLM Agents, https://arxiv.org/abs/2412.00804
[3] J. Huang et al., On the Humanity of Conversational AI: Evaluating the Psychological Portrayal of LLMs, ICLR 2024
[4] R. Chen et al., Learning to Memorize Entailment and Discourse Relations for Persona-Consistent Dialogues, AAAI 2023

## Question C. Other

C1. In terms of presentation, Figure 3 is not appropriate if the authors want to compare between different capabilities of the same model. Redrawing the figure by grouping them with model, instead of capabilities might be more suitable; especially for lines 363-364.

C2. I think the authors should warn the readers that their framework might be used for replicating a human behavior and can be used for fraud. This statement should be added in the ethics statement.
soundness: 2
strengths: - The paper could provide a framework for detailed analysis on digital twin (or human persona replication). 
- The paper provides some discussion about the strength and weakness of present LLMs.
- The paper tested multiple models, which strengthens its generalizability.
summary: This paper suggests a new benchmark that assesses LLM's ability of replicating human persona. The authors used three different datasets to model human persona in terms of three aspects: social, interpersonal, and narrative. To ensure the validity and generalizability of experiment, the authors measured three aspects with three different methods: discriminative, generative(scoring) and generative(ranking). Also, they further measured each aspect in terms of six capabilities. The result found that language models have some ability to replicate human persona, but there's still room for improvement, especially on capabilities of memory recall and persona-tone alignment. And, the authors also noted that the gap between discriminative and generative evaluation protocols indicates the difficulty of open-ended persona replication.
weaknesses: - The experimental method has issues. The selection of language models is not systematic (and there's no reason specified in the paper), and the judge model overlaps with the generation model. The agreement measurement is not proper. See Question A.
- The paper's review on the previous benchmark or papers is somewhat shallow. Other researchers have been reported similar findings, especially on memory recall and persona consistency. Though the paper aims to be a unified framework, the paper should discuss the difference between it and the other previous work clearly to avoid confusion of reader. See Question B.
- The paper's discussion is not deep enough. To provide some insight for the community, this paper could provide some reason or conjecture which can guide the future research. The current depth is insufficient regarding this aspect. See Questions A and B.

---

confidence: 5
contribution: 2
initial_rating: 2
presentation: 1
questions: Please see the weakness section.
soundness: 2
strengths: Please see the weakness section.
summary: Please see the weakness section.
weaknesses: **Significant ICLR formatting violation**

This submission appears to use 1.0 inch left / right margins, significantly below the regulation of 1.5 inch (“Formatting instructions for ICLR 2026 conference submissions,” Line 30, Line 50). This expands the text width from the mandated 5.5 inch to 5.5 + 0.5 * 2 = 6.5 inch, so 9 pages × (6.5 / 5.5) = 10.64 pages of effective content, exceeding the strict 9-page limit (“At the time of submission, the main text should be 9 pages or fewer… This limit will be strictly enforced. Papers with main text beyond the page limit will be desk-rejected.” of the ICLR 2026 author guide https://iclr.cc/Conferences/2026/AuthorGuide). The “Formatting instructions for ICLR 2026 conference submissions” also warns that “tweaking the style files may be grounds for rejection.”

Historically, there has been papers desk-rejected for the exactly same problem.

- “Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models”, ICLR 2025 submission https://openreview.net/forum?id=u4XyECA6Zd)

- “EVLM: An Efficient Vision-Language Model for Visual Understanding”, ICLR 2025 submission https://openreview.net/forum?id=S7M1iqFLVm

- “Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm”, ICLR 2022 submission https://openreview.net/forum?id=NMSugaVzIT

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: * How robust is the proposed LLM-as-a-Judge pipeline against issues like position and self-preference biases?
* Can you show some failure cases that show model failure modes in a more straightforward way?
soundness: 3
strengths: * The proposed TwinVoice benchmark is more comprehensive than previous works:
  * TwinVoice has over 4,500 personas for evaluation, exceeding the size of prior works. The persona dataset is dissected into 3 categories: social persona, interpersonal persona, and narrative persona. This improves the diversity and robustness of persona fidelity evaluation.
  * TwinVoice consists of both real-world and synthetic data, resolving the issue with dominating synthetic data usage in existing works.
* The proposed TwinVoice benchmark approaches persona simulation assessment from multiple aspects, offering new perspectives of evaluating personalized LLM systems. 
  * The work proposes a novel evaluation framework for persona fidelity, targeting analysis from 2 main aspects: mindset coherence and linguistic expression.
    * The 2 aspects of persona simulation evaluation is further elaborated into the test of 6 fundamental capabilities, such as logical reasoning and opinion consistency for mindset coherence and lexical fidelity for linguistic expression.
  * Evaluation methodology includes both discriminative multiple-choice assessment and open-ended LLM-as-a-Judge paradigm.
    * The GPT5-as-a-judge evaluation framework is human-verified with decent agreement score.
* The paper conducts comprehensive evaluation experiments on persona simulation performance with multiple state-of-the-art LLMs, revealing current limitations in aspects such as syntactic style and memory recall.
summary: * This paper introduces the TwinVoice benchmark, a novel and large-scale benchmark for realistic and fine-grained LLM persona simulation.
* The work proposes a novel evaluation framework for persona fidelity, targeting analysis from 2 main aspects: mindset coherence and linguistic expression.
* The paper conducts comprehensive evaluation experiments on persona simulation performance with multiple state-of-the-art LLMs, revealing current limitations in aspects such as syntactic style and memory recall.
weaknesses: * While I appreciate the authors for conducting a human study on the proposed LLM-as-a-judge evaluation framework, I still have minor concerns about the pipeline's robustness:
  * First, for the human verification scale, 50 items per judging mode (100 total) for such a big evaluation benchmark might not be enough.
  * Additionally, previous works [1] [2] have revealed robustness issues with LLM-as-a-judge frameworks, and similar biases could lead to robustness issues of the proposed evaluation framework. 
    * For example, [1] discusses how LLM judges prefer texts that are more familiar to them (self-preference bias), like their own generations or generations from LLMs of similar architecture / training data. **This might explain the good performance of GPT-5-Chat (strongest aggregate generative performance as stated in line 374), since you are using GPT5 as the judge.**
    * Another example would be position bias [2]. **Permutating the choices in the multiple choice & ranking evaluation framework and take the aggregated result** will bring more robustness.
* Lack of qualitative analysis on experiment results, especially on aspects where models fail (e.g. syntatctic style, memory recall). Looking at the numbers and charts, I have a hard time understanding what failure pattens we can observe in models' generation outputs.

[1] Wataoka, Koki, Tsubasa Takahashi, and Ryokan Ri. "Self-Preference Bias in LLM-as-a-Judge." Neurips Safe Generative AI Workshop 2024.
[2] Thakur, Aman Singh, et al. "Judging the judges: Evaluating alignment and vulnerabilities in llms-as-judges." arXiv preprint arXiv:2406.12624 (2024).

---

confidence: 3
contribution: 2
initial_rating: 2
presentation: 2
questions: 1. What are the findings that are not available in other benchmarks but TwinVoice enables? Would it require non-trivial effort to make similar findings on other benchmarks? Do you have supporting empirical evidence? What component of TwinVoice enables it?

2. How did the authors decide on the six capabilities? Are all instances of the data appropriate for evaluating these six capabilities? If not, do you have an estimate on how much of the data is relevant to each capability? For each capability, are there any deterministic or cheaper alternative metrics that can replace LLM-as-judge?

3. Is discriminative evaluation really needed in addition to generative evaluation? How were distractors selected?
soundness: 1
strengths: 1. This paper addresses an important problem of evaluating LLMs at human simulation.

2. The scaled-up evaluation protocol of TwinVoice provides more comprehensive information on the human-simulation performance of LLMs.
summary: This paper introduces TwinVoice, a benchmark for evaluating persona simulation using LLMs. TwinVoice includes three dimensions (settings or contexts for persona simulation) and six capabilities (evaluation criteria). TwinVoice sources its ground truth from real-world human writing across multiple languages and evaluates in both discriminative and generative answer formats. The empirical evaluations on various LLMs, including state-of-the-art LLMs, reveal that six capabilities for each LLM are correlated, while strengths and weaknesses across LLMs are stable.
weaknesses: 1. The paper does not provide evidence of the value this new benchmark adds over the existing benchmarks. It is unclear whether the findings in this paper could have been obtained using existing benchmarks and, if not, which component of this benchmark enabled it.

2. There is no clear rationale behind the choices of six capabilities. They seem to overlap, and not all of them will be relevant at every turn. For example, persona tone will correlate with both lexical and syntactic choices. And not all conversations involve logical reasoning.

3. Discriminative evaluation is not well justified in its practicality and value. The real-world simulation will mostly take a generative form, so it would be helpful to see if discriminative evaluation agrees with generative evaluation instance-wise. Moreover, there are insufficient details about the choice of distractors for discriminative evaluation, which is essential for contextualizing it.