EduPersona: Benchmarking Subjective Ability Boundaries of Virtual Student Agents

## Abstract

Abstract As large language models (LLMs) are increasingly integrated into education, virtual student agents are becoming vital for classroom simulation and teacher training. Yet their classroom-oriented subjective abilities remain largely unassessed, limiting understanding of model boundaries and hindering trustworthy deployment.
We present EduPersona , a large-scale benchmark spanning two languages, three subjects, and ten persona types based on the Big Five theory. The dataset contains 1,308 authentic classroom dialogue rounds, corresponding to 12,814 teacher–student Q&A turns, and is further expanded through persona stylization into roughly 10 × \times larger scale (128k turns), providing a solid foundation for evaluation. Building on this resource, we decompose hard-to-quantify subjective performance into three progressive tasks: TASK1 basic coherence (whether behavior, emotion, expression, and voice align with classroom context), TASK2 student realism, and TASK3 long-term persona consistency, thereby establishing an evaluation framework grounded in educational theory and research value.
We conduct systematic experiments on three representative LLMs, comparing their original versions with ten persona-fine-tuned variants trained on EduPersona. Results show consistent and significant average improvements across all tasks: TASK1 33.6% ↑ \uparrow , TASK2 30.6% ↑ \uparrow , and TASK3 14.9% ↑ \uparrow . These improvements highlight the dataset’s effectiveness and research value, while also revealing the heterogeneous difficulty of persona modeling.
In summary, EduPersona delivers the first classroom benchmark centered on subjective abilities, establishes a decoupled and verifiable research paradigm, and we will open-source both the dataset and the framework to support the broader research community in advancing trustworthy and human-like AI for education.

## 1 Introduction

With the rapid proliferation of large language models (LLMs) in education (Wang et al., 2024b; Tan et al., 2025a; b), virtual student agents are emerging as key tools for classroom simulation and teacher training (Dai et al., 2024). They offer education researchers low-cost, controllable, and reproducible experimental environments, while giving the AI community new opportunities to examine human-like interaction and role-playing. Yet existing evaluation frameworks remain focused on objective tasks such as question answering and accuracy (Lu et al., 2022; Huang et al., 2023; Ang et al., 2023), overlooking the subjective abilities essential to classroom practice.
In authentic educational interactions, students must exhibit multiple layers of subjective traits Wang et al. (2024a); Seo et al. (2025). First is basic coherence, aligning observable behaviors, emotions, expressions, and voice with linguistic outputs (Hayashi, 2024). Second is student realism, reflected in naturalness, credible identity, and adherence to classroom norms such as admitting ignorance, requesting hints, or self-correcting (Sanyal et al., 2025). Third is persona consistency, sustaining stable traits and stylistic patterns across both short- and long-term dialogues (Ma et al., 2024). Together, these layers form a progression—from external behaviors to perceptual authenticity and long-term stability—that mirrors the educational logic of “classroom performance → student traits → individual stability,” while providing AI research with a systematic framework for subjective ability evaluation.

To address this challenge, we introduce EduPersona, the first large-scale benchmark spanning four dimensions: cross-lingual (Chinese and English), cross-subject (Chinese, Mathematics, and English), cross-behavior (four categories of classroom labels), and cross-persona (ten student traits extended from the Big Five Personality Theory). The dataset contains 1,308 authentic classroom dialogue rounds, corresponding to 12,814 teacher–student Q&A turns. It is further expanded through persona stylization into roughly 10$\times$ larger scale (128k turns), providing a solid foundation for evaluation. The central innovation of EduPersona lies in decoupling subjective performance into three progressive evaluation tasks: starting from basic coherence, advancing to student realism, and culminating in long-term persona consistency. This design transforms traditionally unquantifiable subjective performance into an operational and reproducible measurement scheme (see Fig. [1](https://arxiv.org/html/2510.04648v1#S1.F1)).

Figure: Figure 1: Workflow Overview of EduPersona. It consists of three steps: (i) dataset construction (cross-subject and cross-lingual classroom dialogues with persona expansion and multimodal labeling, Sec. [3](https://arxiv.org/html/2510.04648v1#S3)); (ii) a three-task evaluation framework (covering coherence, realism, and consistency, Sec. [4](https://arxiv.org/html/2510.04648v1#S4)); and (iii) systematic experiments and analysis (comparing original and fine-tuned models with cross-model comparisons and case studies, Sec. [5](https://arxiv.org/html/2510.04648v1#S5)). Together, these steps establish the first classroom benchmark focused on subjective abilities, systematically outlining the capability boundaries of virtual student agents.
Refer to caption: x1.png

We conduct systematic experiments on three representative open-source LLMs—Qwen3 (Yang et al., 2025), InternLM3 (Cai et al., 2024), and DeepSeek-R1 (Guo et al., 2025)—comparing their original versions with ten EduPersona fine-tuned variants. Results show significant gains across all tasks: +33.6% in basic coherence, +30.6% in student realism, and +14.9% in persona consistency. These findings validate EduPersona’s effectiveness and research value while highlighting the heterogeneous difficulty of persona modeling. Importantly, subjective abilities do not scale monotonically with model size or reasoning capacity but reveal independent challenges and capability gaps for virtual student agents in educational contexts.

Contributions.
(1) We construct the first large-scale classroom dataset covering two languages, three subjects, four categories of classroom behaviors, and ten persona traits based on extensions of the Big Five Personality Theory (Sec. [3](https://arxiv.org/html/2510.04648v1#S3));
(2) We propose a progressive three-task evaluation framework that decomposes subjective abilities into coherence, realism, and consistency, transforming them into operational and comparable measures (Sec. [4](https://arxiv.org/html/2510.04648v1#S4));
(3) We conduct systematic experiments on multiple mainstream LLMs, showing both the improvements from fine-tuning and the persistent bottlenecks, while revealing that these abilities are not monotonically correlated with model scale or reasoning power (Sec. [5](https://arxiv.org/html/2510.04648v1#S5)).
We will release EduPersona to support reproducibility and extension, advancing trustworthy and human-like virtual student research in both education and AI communities.

## 2 Related Work

We review prior research through the lens of our three evaluation tasks and highlight why systematic modeling of virtual student agents requires going beyond existing approaches.
(1) Basic Behavioral Coherence:
Existing educational datasets (e.g., ScienceQA (Lu et al., 2022), C-Eval (Huang et al., 2023), SocraticQ (Ang et al., 2023), MathQA (Amini et al., 2019)) have advanced knowledge assessment but remain largely single-turn or exam-oriented, lacking modeling of the IRF (Initiation–Response–Feedback) structure central to classrooms. Recent multimodal efforts explore VQA (Lee et al., 2025; Xiao & Zhang, 2025), emotion recognition (Song, 2025), and engagement detection (Xie et al., 2025), yet they focus on perception rather than coherence across verbal and non-verbal dimensions. Task 1 (Sec. [4.1](https://arxiv.org/html/2510.04648v1#S4.SS1)) addresses this gap.
(2) Student Realism:
Persona-driven dialogue studies such as PersonaChat (Zhang et al., 2018), PersonalDialog (Zheng et al., 2019), and MBTI-based generation (Kar & Kar, 2025) illustrate role-conditioned generation, but they rely on simplified tags and are situated in open-domain settings. They cannot answer the classroom-specific question: does a model’s response resemble that of a real student? Task 2 (Sec. [5.3](https://arxiv.org/html/2510.04648v1#S5.SS3)) formalizes this evaluation.
(3) Persona Consistency:
Maintaining stable traits over long interactions remains challenging. Traditional metrics (BLEU, ROUGE) correlate poorly with persona preservation, and alignment methods (RLHF (Ouyang et al., 2022), Constitutional AI (Bai et al., 2022)) or bias detection (Chen et al., 2024) provide only partial insights. Systematic evaluation of persona stability in classroom dialogue is still absent, which Task 3 (Sec. [4.3](https://arxiv.org/html/2510.04648v1#S4.SS3)) directly operationalizes.
Overall, while prior work has progressed in knowledge testing, role-conditioned generation, and multimodal analytics, it lacks a unified, pedagogically grounded framework for jointly evaluating basic coherence, student realism, and persona consistency. EduPersona is designed to fill this gap.

Figure: Figure 2: Chinese classroom example with persona-conditioned responses. The top panel shows a real IRF snippet (with English translation), and the bottom presents virtual-student outputs under high/low extraversion with behavior–expression labels. This illustrates the EduPersona pipeline (raw dialogue $\rightarrow$ persona stylization $\rightarrow$ behavior–expression labeling) and highlights how different personas yield linguistic and non-verbal differences within the same teaching context.
Refer to caption: x2.png

## 3 Dataset Construction

This section introduces the construction and formal representation of the EduPersona dataset, which serves as the unified foundation for the subsequent evaluation tasks. We denote the final dataset as $\mathcal{D}=\{d_{i}\}$, $d_{i}=(x_{i},\,y_{i},\,p_{i},\,b_{i},\,s_{i})$, where $x_{i}$ denotes the classroom context (teacher’s initiation and feedback), $y_{i}$ the student response, $p_{i}\in P$ the persona label, $b_{i}\in B$ the behavior–expression label, and $s_{i}\in S$ the subject. This formulation explicitly separates multiple dimensions, providing a clear and unified modeling basis for the evaluation tasks. Further details on data collection, preprocessing, and the generation and verification of persona and behavior labels are provided in App. [A](https://arxiv.org/html/2510.04648v1#A1).

### 3.1 Data Collection and Preprocessing

**Table 1: Statistics for $\mathcal{D}_{base}$.**
| Subject | Rounds | Turns | Turns / Rounds |
| --- | --- | --- | --- |
| Chinese | 401 | 1,531 | 3.82 |
| Math | 395 | 3,420 | 8.66 |
| English | 512 | 7,863 | 15.36 |
| Total | 1,308 | 12,814 | 9.80 |

We denote the raw classroom corpus as $\mathcal{D}_{base}$, serving as the foundation for subsequent data construction. The subject set is $S={\text{Chinese},,\text{Mathematics},,\text{English}}$, ensuring broad curricular and linguistic coverage.
As shown in Tab. [1](https://arxiv.org/html/2510.04648v1#S3.T1), $\mathcal{D}_{base}$ contains 1,308 dialogue rounds with 12,814 teacher–student Q&A turns, averaging 9.8 turns per round, reflecting rich classroom interactions. The Chinese subset is drawn from the National Primary and Secondary Smart Education Platform (Ministry of Education of the People’s Republic of China, 2024), comprising 401 rounds from 32 verified junior secondary open-class videos. The mathematics subset integrates TIMSS-Math (Stigler et al., 2000) (spanning classrooms from seven countries, predominantly English-speaking) and the NCTE corpus (Demszky & Hill, 2023), totaling 395 rounds. The English subset comes from the TSCC v2 corpus (Caines et al., 2022), contributing 512 rounds of authentic classroom dialogues.
Overall, $\mathcal{D}_{base}$ offers cross-subject, cross-lingual, and cross-cultural coverage while strictly complying with copyright and privacy rules, providing a diverse, reliable, and pedagogically grounded resource that underpins persona modeling, behavior annotation, and evaluation tasks.

### 3.2 Persona and Behavior Annotation

Building on $\mathcal{D}_{base}$, we enrich the dataset with two additional layers of annotation: persona stylization and behavior–expression labeling, resulting in the complete dataset $\mathcal{D}$. This expansion substantially enhances stylistic and interactional diversity, while providing a unified foundation for the subsequent evaluation tasks.

Persona Stylization.
We adopt the Big Five personality theory to define the persona set as:
$P=\mathcal{F}\times\{H,L\}$,
$\mathcal{F}=\{\text{Extraversion},\,\text{Agreeableness},\,\text{Conscientiousness},\,\text{Neuroticism},\,\text{Openness}\},$
where $H$ and $L$ denote high and low levels, yielding $|P|=10$ standardized persona types (see App. [A.3.1](https://arxiv.org/html/2510.04648v1#A1.SS3.SSS1)). For each sample $(x,y)$ consisting of classroom context and student response, we define a rewriting function:
$g:(x,y,p)\mapsto y^{(p)},$
where $p\in P$ is the target persona and $y^{(p)}$ the persona-conditioned output. The function $g$ preserves semantic content while adapting expression style. Each dialogue is thus expanded into 10 persona-specific versions, while teacher feedback remains unchanged. This expansion increases the dataset size by nearly an order of magnitude, enriching stylistic diversity for persona-aware evaluation.

Behavior–Expression Labeling.
Classroom discourse is inherently multimodal, involving both verbal responses $y^{(p)}$ and non-verbal signals. We construct a four-dimensional label space (see App. [A.3.3](https://arxiv.org/html/2510.04648v1#A1.SS3.SSS3)):
$B=B_{beh}\times B_{emo}\times B_{exp}\times B_{voi},$
where $B_{beh}$ (Behavior), $B_{emo}$ (Emotion), $B_{exp}$ (Expression), and $B_{voi}$ (Voice) jointly capture learning orientation and observable signals. Each instance is denoted as $b=(beh,emo,exp,voi)\in B$.
The labeling function is defined as:
$f:(x,\,y^{(p)},\,p)\mapsto b,$
where GPT-4o infers $b$ given $(x,y^{(p)},p)$, with low-confidence or inconsistent predictions flagged for human review. This ensures high-quality alignment between verbal and non-verbal signals.

In summary, the enriched dataset is formalized as:
$\mathcal{D}=\{(x,\,y^{(p)},\,p,\,b,\,s)\},$
where $s\in S$ denotes the subject. By integrating the persona set $P$ and behavior label space $B$, $\mathcal{D}$ provides a systematic and extensible foundation for evaluating the three tasks of basic coherence, student realism, and persona consistency (see Fig. [2](https://arxiv.org/html/2510.04648v1#S2.F2)).

### 3.3 Data Quality Verification and Statistics

Figure: Figure 3: Cross-subject and persona linguistic variation. The word clouds show high-frequency token distributions across Chinese, Math, and English under high/low persona settings, revealing distinct lexical preferences and expression patterns that provide linguistic features for evaluating student realism and persona consistency.
Refer to caption: x3.png

Quality Control.
Rigorous quality control was embedded throughout the dataset construction process to guarantee authenticity, reliability, and consistency. The preprocessing pipeline involved automated crawling of classroom videos and subtitles, text cleaning to remove narration and noise, and normalization to preserve semantic completeness. We reconstructed the IRF (Initiation–Response–Feedback) structure and assigned teacher–student roles, supported by a human-in-the-loop verification loop (automatic annotation, manual sampling, and iterative correction), which achieved 100% verified role labels. For the Chinese subset, each dialogue instance was cross-checked against the corresponding public classroom video, while the mathematics and English subsets underwent multiple rounds of large-scale sampling and manual screening to filter out low-quality or inconsistent cases. All utterances were de-identified to eliminate personal information, and behavior–expression annotation achieved full coverage, with all labels strictly within the predefined vocabulary. These measures ensure the authenticity and reusability of $\mathcal{D}$, providing a reliable foundation for subsequent persona stylization and behavior modeling.

Statistical Analysis.
Beyond quality assurance, we performed linguistic analyses to examine the effects of persona stylization. As shown in Fig. [3](https://arxiv.org/html/2510.04648v1#S3.F3), word clouds illustrate cross-subject and cross-persona token distributions, revealing distinct lexical preferences and expression patterns under different Big Five settings. These differences highlight the interpretability of persona-conditioned outputs and provide observable linguistic cues. Moreover, Fig. [4](https://arxiv.org/html/2510.04648v1#S3.F4) presents a heatmap of vocabulary richness, showing that both subject domain and persona traits significantly affect lexical coverage. For example, English classes exhibit broader lexical diversity overall, while high extraversion and high openness personas consistently yield richer vocabulary across all subjects. These results demonstrate that the dataset not only achieves large scale and high consistency but also encodes quantifiable cross-subject and cross-persona linguistic variation, offering critical support for evaluation.

Figure: Figure 4: Vocabulary richness across subjects and personas. The heatmap shows virtual students’ vocabulary coverage across three subjects and persona settings, indicating that both factors significantly shape lexical diversity.
Refer to caption: image/persona-heatmap.png

The final dataset thus achieves balanced coverage across subjects, languages, persona types, and behavior–expression dimensions. Its large scale and strong consistency make it well-suited for systematic evaluation of virtual student agents. Full distributions and examples are provided in the appendix for reproducibility and further extension.

## 4 Evaluation Framework

Evaluating virtual student agents poses the fundamental challenge of transforming inherently subjective abilities into measurable and reproducible statistics. To address this, we formalize the agent as a conditional generative model:
$M:(x_{t},\,C,\,p,\,s)\mapsto Y_{t},$
where $x_{t}$ denotes the teacher’s input at turn $t$, $C$ the dialogue context (including prior turns), $p\in P$ the persona configuration of the student agent, $s\in S$ the subject domain, and $Y_{t}$ the generated response with associated labels. Equivalently, the model induces a conditional distribution $\mathbb{P}_{M}(Y_{t}\mid x_{t},C,p,s)$, which is compared against a reference distribution $\mathbb{P}^{\star}(Y_{t}\mid x_{t},C,p,s)$ derived from the dataset $\mathcal{D}$. The evaluation problem thus reduces to measuring the statistical divergence between $\mathbb{P}_{M}$ and $\mathbb{P}^{\star}$ under task-specific criteria.
For experimental consistency, we partition the dataset $\mathcal{D}$ into a fine-tuning set $\mathcal{D}_{ft}$ and a held-out test set $\mathcal{D}_{test}$, following a fixed 6:4 split. Fine-tuning is performed exclusively on $\mathcal{D}_{ft}$, while all evaluations are conducted on $\mathcal{D}_{test}$. It ensures that reported results reflect genuine generalization performance rather than memorization of training data.

To operationalize subjective ability assessment, we decompose the problem into three progressive tasks:
(Task 1) Basic coherence: evaluating whether multimodal behavior–expression labels align with contextual semantics;
(Task 2) Student realism: assessing whether responses resemble authentic student traits and classroom norms (e.g., admitting ignorance, requesting hints, or self-correcting);
(Task 3) Persona consistency: examining whether persona traits and stylistic patterns are preserved across extended dialogues.
Together, these layers form a progressive chain from observable behaviors, to perceptual authenticity, and ultimately to long-term stability. Each evaluation is stratified by subject $s$ and persona $p$, with confidence intervals and statistical tests ensuring robustness. In this way, EduPersona transforms vague notions such as “realism” or “consistency” into quantifiable distributional properties, enabling systematic and reproducible benchmarking of virtual student agents. See App. [B](https://arxiv.org/html/2510.04648v1#A2) for further details.

### 4.1 Task 1: Basic Coherence

At the most observable level, a virtual student agent should not only generate textual content but also produce behavior–expression patterns that are consistent with contextual semantics and the persona configuration. Given $(x_{t},C,p,s)$, the model outputs a response $\hat{y}_{t}$ and a predicted behavior vector:
$\hat{b}_{t}=(\hat{beh}_{t},\,\hat{emo}_{t},\,\hat{exp}_{t},\,\hat{voi}_{t})\in B=B_{beh}\times B_{emo}\times B_{exp}\times B_{voi},$
where $B_{beh}$ (ICAP-based classroom behaviors), $B_{emo}$ (emotions), $B_{exp}$ (facial expressions), and $B_{voi}$ (vocal styles) form a *closed vocabulary*. Reference labels $b_{t}=(beh_{t},\,emo_{t},\,exp_{t},\,voi_{t})$ are generated by GPT-4 under persona and context constraints and then human-audited (see Sec. [3.2](https://arxiv.org/html/2510.04648v1#S3.SS2)). We macro-average all metrics over the four dimensions, using the index set $\mathcal{I}=\{\text{beh},\text{emo},\text{exp},\text{voi}\}$ and $T$ evaluation instances; $\emptyset$ denotes no output, and $B_{i}$ the codebook of dimension $i$.

Step 1. Response rate. We first check whether any output is produced on each dimension:

$$ $\mathrm{RespRate}=\frac{1}{|\mathcal{I}|}\sum_{i\in\mathcal{I}}\frac{1}{T}\sum_{t=1}^{T}\mathds{1}[\hat{b}_{t,i}\neq\emptyset].$ (1) $$

Step 2. Validity rate. Conditional on producing outputs, we verify whether they fall within the predefined vocabulary:

$$ $\mathrm{ValidRate}=\frac{1}{|\mathcal{I}|}\sum_{i\in\mathcal{I}}\frac{\sum_{t=1}^{T}\mathds{1}[\hat{b}_{t,i}\in B_{i}]}{\max\!\left(1,\sum_{t=1}^{T}\mathds{1}[\hat{b}_{t,i}\neq\emptyset]\right)}.$ (2) $$

Step 3. Label prediction quality. Restricting to dimensions with outputs, we evaluate agreement with the reference annotations. To disentangle different error sources, we define three complementary accuracies (same numerator—the number of correct labels—but different denominators):

$$ $\mathrm{RawAcc}=\frac{1}{|\mathcal{I}|}\sum_{i\in\mathcal{I}}\frac{\sum_{t=1}^{T}\mathds{1}[\hat{b}_{t,i}=b_{t,i}]}{\max\!\left(1,\sum_{t=1}^{T}\mathds{1}[\hat{b}_{t,i}\neq\emptyset]\right)},$ (3) $$

$$ $\mathrm{ValAcc}=\frac{1}{|\mathcal{I}|}\sum_{i\in\mathcal{I}}\frac{\sum_{t=1}^{T}\mathds{1}[\hat{b}_{t,i}=b_{t,i}]}{\max\!\left(1,\sum_{t=1}^{T}\mathds{1}[\hat{b}_{t,i}\in B_{i}]\right)},$ (4) $$

$$ $\mathrm{OverallAcc}=\frac{1}{|\mathcal{I}|}\sum_{i\in\mathcal{I}}\frac{1}{T}\sum_{t=1}^{T}\mathds{1}[\hat{b}_{t,i}=b_{t,i}].$ (5) $$

These three metrics satisfy the inequality
$\mathrm{OverallAcc}\leq\mathrm{RawAcc}\leq\mathrm{ValAcc}$,
reflecting a progressive tightening from *availability* $\rightarrow$ *validity* $\rightarrow$ *end-to-end correctness*.
Task 1 thus provides the most objective baseline for behavioral alignment and supplies interpretable low-level signals upon which higher-level evaluations of student realism (Task 2) and persona consistency (Task 3) can be built.

### 4.2 Task 2: Student Realism

Student realism evaluates whether a model’s response looks like a real student. This concept goes beyond linguistic fluency, requiring identity credibility (e.g., admitting ignorance, requesting hints, self-correction) and adherence to classroom interaction norms.
To ground the evaluation, we consulted ten experts from education and AI, who reviewed a sampled subset of responses and distilled a set of core dimensions $\mathcal{R}=\{r_{1},\dots,r_{m}\}$, including linguistic naturalness, identity credibility, strategy appropriateness, and coordination with teacher feedback. These dimensions form the foundation for subsequent large-scale evaluation.
We then encode $\mathcal{R}$ into prompts to construct an evaluation function:
$G_{\mathcal{R}}:(x_{t},\,C,\,p,\,s,\,\hat{y}_{t})\ \longmapsto\ \{\hat{h},\,\hat{\mathbf{z}}\},$
where $\hat{h}\in\{0,1\}$ denotes the overall student-likeness decision and $\hat{\mathbf{z}}\in\{0,1\}^{m}$ the dimension-wise outcomes. In this setup, GPT does *not* serve as an independent judge but as a scalable extension of the expert-derived criteria, ensuring interpretability and reproducibility.
Results are aggregated across subjects $s$ and personas $p$, with macro-averaged scores reported at both overall and dimension levels, yielding a systematic, interpretable, and scalable evaluation of student realism.

### 4.3 Task 3: Persona Consistency

Persona consistency requires virtual student agents to remain aligned with the target persona in both single-turn responses and extended dialogues. We define a standardized confidence function with range $[0,1]$:
$J(\hat{y}_{t},\,p)\in[0,1],$
where $J(\hat{y}_{t},\,p)=0$ indicates complete mismatch, $J(\hat{y}_{t},\,p)=1$ indicates perfect alignment, and intermediate values reflect partial consistency.
Evaluation is conducted at two scales. For short-term consistency, each generated response $\hat{y}_{t}$ on the held-out test set $\mathcal{D}_{test}$ receives a persona score, and the average across samples is reported. For long-term consistency, models engage in fixed 10-turn classroom-style interactions driven by a *Teacher-Policy* $\pi_{T}$ induced from the full dataset $\mathcal{D}$, which captures authentic instructional patterns such as IRF structures, scaffolding, and progressive difficulty. Scores are aggregated over the session to assess stability under sustained interaction.
A unified metric is applied to both settings:
$\mathrm{Cons}=\frac{1}{|\mathcal{T}|}\sum_{t\in\mathcal{T}}J(\hat{y}_{t},\,p),$
where $\mathcal{T}=\{1,\dots,N\}$ for short-term consistency (with $N$ test samples) and $\mathcal{T}=\{1,\dots,10\}$ for long-term interactive sessions (fixed at 10 turns). This formulation allows us to examine both immediate persona alignment and its persistence throughout extended interactions.

## 5 Experimental Design and Analysis

Building on EduPersona, we delineate the performance boundaries of virtual student agents across *basic coherence*, *student realism*, and *persona consistency*. We study two complementary settings in a unified protocol: baseline evaluation, where three representative foundation LLMs $\mathcal{M}^{(i)}_{\text{base}}$ are directly assessed without additional adaptation, and persona-conditioned evaluation, where each base model $\mathcal{M}^{(i)}$ is fine-tuned over the Big Five–based persona set $P=\{p_{1},\dots,p_{10}\}$ to obtain $\mathcal{M}^{(i)}_{\text{ft}}(p)$, yielding $3\times 10=30$ EduPersona-trained variants. All evaluations are run on the held-out test set $\mathcal{D}_{test}$, while fine-tuning uses only $\mathcal{D}_{ft}$; the complete dataset $\mathcal{D}$ is split $6{:}4$ (train:test) to emphasize generalization.
Within this setup, Sec. [5.1](https://arxiv.org/html/2510.04648v1#S5.SS1) outlines the model lineup and fine-tuning configuration; Secs. [5.2](https://arxiv.org/html/2510.04648v1#S5.SS2)–[5.4](https://arxiv.org/html/2510.04648v1#S5.SS4) report task-wise results. Further implementation details are provided in App. [C](https://arxiv.org/html/2510.04648v1#A3).

### 5.1 Experimental Architecture and Model Selection

We employ three open-source foundation LLMs as bases: Qwen3-8B ($\mathcal{M}^{Q}$), noted for strong Chinese–English instruction following (Yang et al., 2025); InternLM3-8B-Instruct ($\mathcal{M}^{I}$), robust in Chinese educational scenarios with broad multilingual coverage (Cai et al., 2024); and DeepSeek-R1-Distill-Qwen-14B ($\mathcal{M}^{D}$), distilled for enhanced mathematical and logical reasoning (Guo et al., 2025). For each $\mathcal{M}^{(j)}$ and persona $p\in P=\{p_{1},\dots,p_{10}\}$, we obtain a persona-specific variant $\mathcal{M}^{(j)}_{\text{ft}}(p)$, while the unadapted counterparts $\mathcal{M}^{(j)}_{\text{base}}$ serve as references.
Fine-tuning follows a consistent LoRA configuration (rank $r=16$, scaling $\alpha=32$) with AdamW, learning rate $\eta=3\times 10^{-4}$, per-device batch size $8$ and gradient accumulation $4$, for up to $5$ epochs. Each turn is encoded as a unified input $u_{t}=(x_{t},\,C,\,p,\,s,\,b_{t})$ to couple generative dialogue with closed-vocabulary behavior–expression codes. Inference settings and random seeds are aligned across models.

### 5.2 Basic Coherence: Can virtual students generate multimodal behaviors aligned with context?

Basic coherence requires virtual students not only to produce text but also to align behaviors, emotions, expressions, and vocal styles with the classroom context. We evaluate three model families using the five metrics defined in Sec. [4.1](https://arxiv.org/html/2510.04648v1#S4.SS1)—response rate, validity rate, RawAcc, ValAcc, and OverallAcc—macro-averaged across four dimensions (behavior, emotion, expression, voice).

Figure: Figure 5: Impact of persona fine-tuning on basic coherence across five metrics. Persona fine-tuning consistently improves basic coherence across all models, demonstrating the value of EduPersona. Fine-tuned Qwen and DeepSeek achieve OverallAcc above 0.62 with strong label alignment, while InternLM3 also benefits but remains constrained by a low response rate.
Refer to caption: image/Task1V4.png

Fig. [5](https://arxiv.org/html/2510.04648v1#S5.F5) shows that persona fine-tuning substantially enhances multimodal alignment. Qwen (0.9951/0.9997) and DeepSeek (1.0000/0.9995) achieve near-perfect response and validity rates, while InternLM3 retains high validity (0.9998) but suffers from low response coverage (0.4261). For label alignment, fine-tuned RawAcc and ValAcc converge in the 0.624–0.653 range, a large improvement over base versions (e.g., Qwen: 0.2894 $\rightarrow$ 0.6310). With validity nearly saturated, Raw and Val scores converge, whereas base models show wider Raw–Val gaps due to frequent OOV outputs (e.g., DeepSeek: 0.2296 $\rightarrow$ 0.4883). For end-to-end correctness, Qwen and DeepSeek reach OverallAcc of 0.6277 and 0.6242, far above their baselines (0.2775/0.1145), while InternLM3 remains at 0.2773, constrained by limited response coverage.

In summary, Task 1 shows that persona fine-tuning substantially enhances basic coherence, enabling Qwen and DeepSeek to approach usable levels of multimodal alignment, though InternLM3 still faces a bottleneck in response generation.
Beyond overall gains, two further insights emerge based on detailed per-dimension results in App. [C.1](https://arxiv.org/html/2510.04648v1#A3.SS1).
First, a stable difficulty ordering is observed across dimensions, with Emotion being the easiest and Behavior the hardest, highlighting that residual errors stem from discourse- and intent-level challenges rather than random noise.
Second, Qwen and DeepSeek converge in end-to-end accuracy after fine-tuning (OverallAcc $\approx 0.62$), suggesting that task structure and dataset design, rather than model scale alone, largely determine the performance ceiling.
With validity already near saturation, future improvements will most likely come from increasing response coverage and strengthening fine-grained behavioral guidance.

### 5.3 Student Realism: Can virtual students be like real students?

Figure: Figure 6: Persona fine-tuning markedly improves student realism. EduPersona brings all three model families to a converged level around 0.82 while revealing stable persona-specific differences (HC/HO harder, HN/LC/LO easier).
Refer to caption: x4.png

Student realism is a key criterion for evaluating whether virtual students resemble real learners. Using expert-derived dimensions operationalized through an automatic evaluator, we analyze results from both persona and model perspectives (Fig. [6](https://arxiv.org/html/2510.04648v1#S5.F6)).

Persona-wise analysis (Fig. [6](https://arxiv.org/html/2510.04648v1#S5.F6)a).
Fine-tuned models exhibit consistently stronger realism across all personas, yet heterogeneity persists. High Conscientiousness (HC) and High Openness (HO) remain lower both before and after fine-tuning, with modest gains, whereas High Neuroticism (HN), Low Conscientiousness (LC), and Low Openness (LO) achieve relatively high realism. This reflects the interaction between persona traits and model defaults: HC/HO emphasize objective, comprehensive, and teacher-aligned behaviors, overlapping with LLMs’ inherent answer-first tendency, thus appearing more machine-like. By contrast, HN/LC/LO manifest hesitation, partial responses, or self-corrections, which enhance perceived authenticity and yield higher realism.

Model-wise analysis (Fig. [6](https://arxiv.org/html/2510.04648v1#S5.F6)b).
At the baseline level, the three families differ substantially (Qwen 0.7019, InternLM3 0.6478, DeepSeek 0.556). After fine-tuning, however, all models converge to a narrow band around 0.82 (Qwen 0.8221, InternLM3 0.8265, DeepSeek 0.8176). This demonstrates that persona conditioning both elevates student realism and reduces inter-model disparities.

In summary, Task 2 shows that persona fine-tuning significantly enhances student realism and brings different models to a converged level, though persona-specific bottlenecks remain. Additional analysis in the App. [C.2](https://arxiv.org/html/2510.04648v1#A3.SS2) further reveals that the absolute gains vary across models, with DeepSeek benefiting most, and that cross-task consistency emerges: the easiest Emotion dimension in Task 1 corresponds to high realism personas (HN/LN/LO), while the hardest Behavior dimension aligns with low realism personas (HC/HO). These findings establish EduPersona’s effectiveness in enhancing high-level perception while also highlighting areas where persona-specific improvements are needed, providing a solid foundation for subsequent evaluations of persona consistency.

### 5.4 Persona Consistency: Can virtual students maintain stable personas during interactions?

Task 3 evaluates whether virtual students can maintain stable persona traits throughout interactions. Results show that fine-tuned models achieve substantially higher and more stable consistency at both persona and model levels, confirming the effectiveness of persona conditioning.

Persona-wise analysis (Fig. [7](https://arxiv.org/html/2510.04648v1#S5.F7)a).
While fine-tuning consistently improves all ten personas, heterogeneity persists. High Conscientiousness (HC, 0.731) and High Openness (HO, 0.779) remain the most difficult to sustain, even after adaptation, whereas High Neuroticism (HN, 0.901), Low Conscientiousness (LC, 0.887), and Low Openness (LO, 0.873) achieve the highest stability. This echoes Task 2’s findings on realism, suggesting that structured, “idealized” personas are both less authentic and less consistent, while personas reflecting hesitation or partial responses are easier to maintain. Gains also vary across personas: Low Extraversion (+0.146), Low Openness (+0.133), and Low Agreeableness (+0.130) benefit most, whereas High Openness (+0.062) and High Conscientiousness (+0.073) improve the least.

Figure: Figure 7: Persona fine-tuning enhances persona stability. Fine-tuned models converge around 0.84 while showing consistent persona-specific differences (HC/HO harder, HN/LC/LO easier).
Refer to caption: x5.png

Model-wise analysis (Fig. [7](https://arxiv.org/html/2510.04648v1#S5.F7)b).
At the baseline level, Qwen, InternLM3, and DeepSeek differ significantly (0.795, 0.723, and 0.677). After fine-tuning, however, all three converge to a narrow range of 0.833–0.841, showing that persona conditioning boosts consistency while reducing cross-family disparities.

Multi-turn consistency. To compare the persona retention ability of fine-tuned versus closed-source models, we conducted a 10-turn English classroom experiment, involving three LoRA-fine-tuned models (Qwen3, InternLM3, DeepSeek) and GPT-4o. LoRA-fine-tuned models retained persona traits even after explicit prompts disappeared, with Qwen3-LoRA achieving the highest overall score ($0.920\pm 0.042$). All three models maintained $\geq 0.8$ in later turns (6–10), whereas GPT-4o quickly drifted once persona instructions vanished, averaging only $0.480\pm 0.262$ and dropping below 0.50 in later turns. This highlights the importance of fine-tuning for long-horizon persona stability.

In summary, Task 3 demonstrates that persona consistency is more demanding than student realism, but persona fine-tuning significantly enhances stability while narrowing both model- and persona-level gaps. Notably, HC and HO personas remain persistent bottlenecks across realism and consistency, whereas HN, LC, and LO are consistently easier to model. More detailed distributions and analyses are provided in App. [C.3](https://arxiv.org/html/2510.04648v1#A3.SS3).

### 5.5 Summary

The three experiments depict the landscape of virtual students’ subjective abilities. In basic coherence (Task 1), Qwen and DeepSeek converge around 0.62 after fine-tuning (0.6277/0.6242), while InternLM3 remains lower at 0.2773 due to limited response coverage (RespRate 0.4261; App. [C.1](https://arxiv.org/html/2510.04648v1#A3.SS1)). For student realism (Task 2), baseline gaps (Qwen 0.7019, InternLM3 0.6478, DeepSeek 0.5560) narrow after fine-tuning, with all three converging near 0.82 (App. [C.2](https://arxiv.org/html/2510.04648v1#A3.SS2)). In persona consistency (Task 3), models rise from 0.795/0.723/0.677 to  0.84 (0.841/0.834/0.837), again showing significant gains and convergence (App. [C.3](https://arxiv.org/html/2510.04648v1#A3.SS3)). Thus, EduPersona boosts absolute performance while aligning cross-family outcomes.

The tasks also reveal a layered progression: Task 1 captures structural compliance, Task 2 addresses human-perceived realism, and Task 3 tests long-horizon persona stability. Persona-level trends are consistent—HC/HO remain bottlenecks, while HN/LC/LO are easier to emulate, echoing Task 1’s “Emotion easier, Behavior harder” pattern. A 10-turn study further shows LoRA-fine-tuned models retain traits after prompts vanish (e.g., Qwen3-LoRA 0.920±0.042), while GPT-4o drifts (0.480±0.262), underscoring that long-horizon stability depends on fine-tuning rather than model scale. In sum, performance ceilings are shaped more by persona type and interaction length than by parameter size, pointing to future work on coverage, behavior supervision, and embedding natural hesitation–repair–collaboration strategies into idealized personas.

## 6 Conclusion

This paper introduced EduPersona, a framework for evaluating virtual students across three subjective abilities: basic coherence, student realism, and persona consistency. Experiments show that persona fine-tuning markedly improves performance, with models converging at 0.62, 0.82, and 0.84. Stable persona-specific gaps persist: high conscientiousness and openness are harder to emulate, while high neuroticism, low conscientiousness, and low openness are easier to sustain. These results validate the framework and reveal structural links between persona traits and model behaviors. Overall, EduPersona demonstrates a progression from structural compliance to perceived realism and long-horizon stability, underscoring the importance of the “behavior” dimension and sustained dialogue. While fine-tuning narrows disparities, challenges remain in response coverage, idealized personas, and long-term stability. Future work should strengthen behavioral supervision, foster natural interaction, and extend to interdisciplinary contexts, positioning EduPersona as a pathway toward trustworthy educational agents.

## Ethics Statement

This work complies with ethical standards and data usage regulations. The EduPersona dataset is constructed from publicly available classroom resources, with all data anonymized to remove IDs and other identifiers. Persona configurations are derived from the well-established Big Five personality theory and serve only as abstract stylistic constructs for modeling language and behavior; they do not encode demographic or sensitive attributes. Thus, this study introduces no new ethical risks and instead provides a safe, controlled, and reproducible framework for evaluating virtual students in educational contexts.

## Reproducibility Statement

To ensure reproducibility, we will open-source the complete codebase, annotation guidelines, preprocessing pipeline, and trained models. The released EduPersona resources will include the full data schema (prompts, vocabularies, labels, and evaluation metrics). Third-party raw media will not be redistributed due to licensing restrictions; instead, we provide scripts to re-derive processed text from public resources. Appendix [A](https://arxiv.org/html/2510.04648v1#A1) details data collection and preprocessing, Appendix [A.3.1](https://arxiv.org/html/2510.04648v1#A1.SS3.SSS1) and [A.3.2](https://arxiv.org/html/2510.04648v1#A1.SS3.SSS2) describe persona configurations and stylization prompts, and Appendix [B](https://arxiv.org/html/2510.04648v1#A2) provides the evaluation prompts. Additional fine-grained analyses are reported in Appendix [C.1](https://arxiv.org/html/2510.04648v1#A3.SS1)–[C.3](https://arxiv.org/html/2510.04648v1#A3.SS3) to further substantiate our conclusions. All code, data, and models will be released upon acceptance to foster long-term reproducibility and extensibility in educational agent research.

## References

- Amini et al. (2019)
Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi.
Mathqa: Towards interpretable math word problem solving with operation-based formalisms.
*arXiv preprint arXiv:1905.13319*, 2019.
- Ang et al. (2023)
Beng Heng Ang, Sujatha Das Gollapalli, and See Kiong Ng.
Socratic question generation: A novel dataset, models, and evaluation.
In *Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics*, pp. 147–165, 2023.
- Bai et al. (2022)
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al.
Constitutional ai: Harmlessness from ai feedback.
*arXiv preprint arXiv:2212.08073*, 2022.
- Cai et al. (2024)
Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, Shuaibin Li, Wei Li, Yining Li, Hongwei Liu, Jiangning Liu, Jiawei Hong, Kaiwen Liu, Kuikun Liu, Xiaoran Liu, Chengqi Lv, Haijun Lv, Kai Lv, Li Ma, Runyuan Ma, Zerun Ma, Wenchang Ning, Linke Ouyang, Jiantao Qiu, Yuan Qu, Fukai Shang, Yunfan Shao, Demin Song, Zifan Song, Zhihao Sui, Peng Sun, Yu Sun, Huanze Tang, Bin Wang, Guoteng Wang, Jiaqi Wang, Jiayu Wang, Rui Wang, Yudong Wang, Ziyi Wang, Xingjian Wei, Qizhen Weng, Fan Wu, Yingtong Xiong, Chao Xu, Ruiliang Xu, Hang Yan, Yirong Yan, Xiaogui Yang, Haochen Ye, Huaiyuan Ying, Jia Yu, Jing Yu, Yuhang Zang, Chuyu Zhang, Li Zhang, Pan Zhang, Peng Zhang, Ruijie Zhang, Shuo Zhang, Songyang Zhang, Wenjian Zhang,
Wenwei Zhang, Xingcheng Zhang, Xinyue Zhang, Hui Zhao, Qian Zhao, Xiaomeng Zhao, Fengzhe Zhou, Zaida Zhou, Jingming Zhuo, Yicheng Zou, Xipeng Qiu, Yu Qiao, and Dahua Lin.
Internlm2 technical report, 2024.
- Caines et al. (2022)
Andrew Caines, Helen Yannakoudakis, Helen Allen, Pascual Pérez-Paredes, Bill Byrne, and Paula Buttery.
The teacher-student chatroom corpus version 2: more lessons, new annotation, automatic detection of sequence shifts.
In David Alfter, Elena Volodina, Thomas François, Piet Desmet, Frederik Cornillie, Arne Jönsson, and Evelina Rennes (eds.), *Proceedings of the 11th Workshop on NLP for Computer Assisted Language Learning*, pp. 23–35, Louvain-la-Neuve, Belgium, December 2022. LiU Electronic Press.
URL [https://aclanthology.org/2022.nlp4call-1.3/](https://aclanthology.org/2022.nlp4call-1.3/).
- Chen et al. (2024)
Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, and Yanghua Xiao.
From persona to personalization: A survey on role-playing language agents.
*Transactions on Machine Learning Research*, 2024.
ISSN 2835-8856.
Survey Certification.
- Dai et al. (2024)
Chih-Pu Dai, Fengfeng Ke, Nuodi Zhang, Alex Barrett, Luke West, Saptarshi Bhowmik, Sherry A Southerland, and Xin Yuan.
Designing conversational agents to support student teacher learning in virtual reality simulation: a case study.
In *Extended Abstracts of the CHI Conference on Human Factors in Computing Systems*, pp. 1–8, 2024.
- Demszky & Hill (2023)
Dorottya Demszky and Heather Hill.
The ncte transcripts: A dataset of elementary math classroom transcripts, 2023.
URL [https://arxiv.org/abs/2211.11772](https://arxiv.org/abs/2211.11772).
- Guo et al. (2025)
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.
*arXiv preprint arXiv:2501.12948*, 2025.
- Hayashi (2024)
Yugo Hayashi.
Modeling synchronization for detecting collaborative learning process using a pedagogical conversational agent: Investigation using recurrent indicators of gaze, language, and facial expression.
*International Journal of Artificial Intelligence in Education*, 34(3):1206–1247, 2024.
- Huang et al. (2023)
Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Yao Fu, et al.
C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models.
*Advances in Neural Information Processing Systems*, 36:62991–63010, 2023.
- Kar & Kar (2025)
Parijat Kar and Ridhima Kar.
Convergence of chatbot personalities using reinforcement learning and text generation.
In *Proceedings of the AAAI Symposium Series*, volume 5, pp. 327–333, 2025.
- Lee et al. (2025)
Gyeonggeon Lee, Lehong Shi, Ehsan Latif, Yizhu Gao, Arne Bewersdorff, Matthew Nyaaba, Shuchen Guo, Zhengliang Liu, Gengchen Mai, Tianming Liu, et al.
Multimodality of ai for education: Towards artificial general intelligence.
*IEEE Transactions on Learning Technologies*, 2025.
- Lu et al. (2022)
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan.
Learn to explain: Multimodal reasoning via thought chains for science question answering.
*Advances in Neural Information Processing Systems*, 35:2507–2521, 2022.
- Ma et al. (2024)
Yiping Ma, Shiyu Hu, Xuchen Li, Yipei Wang, Yuqing Chen, Shiqing Liu, and Kang Hao Cheong.
When llms learn to be students: The soei framework for modeling and evaluating virtual student agents in educational interaction.
*arXiv preprint arXiv:2410.15701*, 2024.
- Ministry of Education of the People’s Republic of China (2024)
Ministry of Education of the People’s Republic of China.
National primary and secondary smart education platform.
[https://www.smartedu.cn/](https://www.smartedu.cn/), 2024.
Accessed: 2025-09-25.
- Ouyang et al. (2022)
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.
Training language models to follow instructions with human feedback.
*Advances in neural information processing systems*, 35:27730–27744, 2022.
- Sanyal et al. (2025)
Debdeep Sanyal, Agniva Maiti, Umakanta Maharana, Dhruv Kumar, Ankur Mali, C Lee Giles, and Murari Mandal.
Investigating pedagogical teacher and student llm agents: Genetic adaptation meets retrieval augmented generation across learning style.
*arXiv preprint arXiv:2505.19173*, 2025.
- Seo et al. (2025)
Kyoungwon Seo, Mina Yoo, Samuel Dodson, and Sung-Hee Jin.
Augmented teachers: K–12 teachers’ needs for artificial intelligence’s complementary role in personalized learning.
*Journal of Research on Technology in Education*, 57(4):876–893, 2025.
- Song (2025)
Xiaohuan Song.
Emotional recognition and feedback of students in english e-learning based on computer vision and face recognition algorithms.
*Entertainment Computing*, 52:100847, 2025.
- Stigler et al. (2000)
James W. Stigler, Ronald Gallimore, and James Hiebert.
Using video surveys to compare classrooms and teaching across cultures: Examples and lessons from the timss and timss-r video studies.
*Educational Psychologist*, 35(2):87–100, 2000.
doi: 10.1207/S15326985EP3502_3.
- Tan et al. (2025a)
Le Ying Tan, Shiyu Hu, Darren J Yeo, and Kang Hao Cheong.
Artificial intelligence-enabled adaptive learning platforms: A review.
*Computers and Education: Artificial Intelligence*, pp. 100429, 2025a.
- Tan et al. (2025b)
Le Ying Tan, Shiyu Hu, Darren J Yeo, and Kang Hao Cheong.
A comprehensive review on automated grading systems in stem using ai techniques.
*Mathematics*, 13(17):2828, 2025b.
- Wang et al. (2024a)
Shanshan Wang, Zhen Zeng, Xun Yang, Ke Xu, and Xingyi Zhang.
Boosting neural cognitive diagnosis with student’s affective state modeling.
In *Proceedings of the AAAI conference on artificial intelligence*, volume 38, pp. 620–627, 2024a.
- Wang et al. (2024b)
Shen Wang, Tianlong Xu, Hang Li, Chaoli Zhang, Joleen Liang, Jiliang Tang, Philip S Yu, and Qingsong Wen.
Large language models for education: A survey and outlook.
*arXiv preprint arXiv:2403.18105*, 2024b.
- Xiao & Zhang (2025)
Jiongen Xiao and Zifeng Zhang.
Eduvqa: A multimodal visual question answering framework for smart education.
*Alexandria Engineering Journal*, 122:615–624, 2025.
- Xie et al. (2025)
Nan Xie, Zhengxu Li, Haipeng Lu, Wei Pang, Jiayin Song, and Beier Lu.
Msc-trans: A multi-feature-fusion network with encoding structure for student engagement detecting.
*IEEE Transactions on Learning Technologies*, 2025.
- Yang et al. (2025)
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al.
Qwen3 technical report.
*arXiv preprint arXiv:2505.09388*, 2025.
- Zhang et al. (2018)
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston.
Personalizing dialogue agents: I have a dog, do you have pets too?
In Iryna Gurevych and Yusuke Miyao (eds.), *Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 2204–2213, Melbourne, Australia, July 2018. Association for Computational Linguistics.
doi: 10.18653/v1/P18-1205.
URL [https://aclanthology.org/P18-1205/](https://aclanthology.org/P18-1205/).
- Zheng et al. (2019)
Yinhe Zheng, Guanyi Chen, Minlie Huang, Song Liu, and Xuan Zhu.
Personalized dialogue generation with diversified traits.
*arXiv preprint arXiv:1901.09672*, 2019.

## Appendix

This appendix provides implementation details of the EduPersona corpus and the processing steps that support the evaluation framework in the main paper, along with additional fine-grained experimental results that further substantiate our conclusions.

## Appendix A Dataset Details

### A.1 Detailed Data Collection and Preprocessing

The EduPersona corpus is constructed from publicly accessible classroom resources licensed for research, covering three subjects (Chinese, Mathematics, English) across two languages (Chinese/English). Sources include: (i) curated Chinese open-class videos and transcripts from a national education platform, (ii) international mathematics discourse corpora, and (iii) English classroom transcripts. Only segments with clear instructional intent are retained; all personally identifiable information is masked, and collection complies with the original platforms’ terms of use. No redistribution of third-party raw media is performed.

Preprocessing and structuring.
A unified pipeline ensures cross-subject comparability. Videos and subtitles (or ASR transcripts where subtitles are unavailable) are aligned at the utterance level with normalized punctuation and casing. Teacher–student roles and IRF structures are reconstructed, narration and meta-comments are removed, and turn boundaries are adjusted to avoid truncation. Identifiers such as IDs are masked at source to guarantee non-attributability. A human-in-the-loop process (automatic tagging, sampled review, manual correction) yields verified role labels.

Persona expansion and behavior labeling.
To enable persona-conditioned dialogue modeling, each student response is expanded into ten variants using the Big Five dimensions $\mathcal{F}={\text{Extraversion},\ \text{Agreeableness},\ \text{Conscientiousness},\ \text{Neuroticism},\ \text{Openness}}$ with high/low polarity, giving $P=\mathcal{F}\times{H,L}$. Each original $(x,y)$ is rewritten into $y^{(p)}$ that preserves semantics while adapting to persona $p$. In addition, each $(x,y^{(p)})$ is labeled with a four-dimensional vector $b=(beh,emo,exp,voi)$ covering *Behavior*, *Emotion*, *Expression*, and *Voice*, constrained to a fixed vocabulary (App. [A.3.3](https://arxiv.org/html/2510.04648v1#A1.SS3.SSS3)). Low-confidence or contradictory cases are resolved through human auditing.

Split, statistics, and quality control.
The full dataset $\mathcal{D}$ is stratified by subject and persona into fine-tuning ($\mathcal{D}{ft}$) and test sets ($\mathcal{D}{test}$) with a 6:4 ratio. All evaluation tasks (Tasks 1–3) are performed exclusively on $\mathcal{D}{test}$.
Before persona expansion, the base corpus $\mathcal{D}{base}$ contains 1,308 dialogue rounds with 12,814 teacher–student Q&A turns. By subject: Chinese contributes 401 rounds from 32 lessons (1,531 Q&A turns; avg. 3.82/round), Mathematics 395 rounds (3,420 Q&A turns; avg. 8.66/round), and English 512 rounds (7,863 Q&A turns; avg. 15.36/round). After stylization, each turn yields ten persona-conditioned variants, expanding the dataset tenfold.
Quality control is embedded at every stage: role integrity verified against source materials, persona fidelity checked by independent judges, label validity enforced with 100% in-vocabulary coverage, and near-duplicates removed via semantic similarity. These measures ensure that EduPersona is reliable, diverse, and ethically compliant, providing a solid foundation for the evaluation framework and subsequent experiments.

### A.2 Data Access and Ethics

We will release full preprocessing and labeling code, along with a legally redistributable subset that mirrors the complete schema (including prompts, vocabularies, and metric definitions), thereby supporting reproducibility of all reported experiments. Third-party raw media are not redistributed; instead, scripts are provided to re-derive processed text from publicly available resources where permitted.

All dialogues are fully de-identified, with names, IDs, and any sensitive information removed. Persona variants are abstract stylistic constructs derived from the Big Five framework and do not encode demographic attributes. We explicitly caution against any attempt at re-identification or demographic inference. The EduPersona corpus is designed under principles of compliance, transparency, and responsible use, ensuring safe deployment in educational and AI research contexts.

### A.3 More Information about Annotation

#### A.3.1 Ten Persona Configurations Based on Big Five Theory

Our persona framework is grounded in the Big Five personality theory (BFAS scale), extending it into ten standardized configurations that represent authentic student behavioral patterns in classroom settings. Each persona is defined through multiple dimensions to ensure comprehensive and scientifically-based characterization.

**Table A1: Ten persona configurations based on Big Five personality theory.**
| Persona Type | Detailed Characteristics |
| --- | --- |
| High Extraversion | Core traits: Take charge, have a strong personality, warm up quickly to others |
|  | Behavioral traits: Active participation, strong social skills, comfortable expression, high exhibition, obvious dominance |
|  | Language style: Fluent and confident, detailed elaboration, positive attitude, clear structure, likes to lead conversations |
|  | Classroom behavior: Strong desire to participate, actively answers questions, proactively shows themselves, frequent interaction with teachers |
|  | Response patterns: Detailed answers with additional explanations, fluent language but may include repetitions and filler words |
| Low Extraversion | Core traits: Do not have an assertive personality, hard to get to know, keep others at a distance |
|  | Behavioral traits: Introverted and cautious, doesn’t like to show off, socially conservative, low participation |
|  | Language style: Concise and conservative, cautious responses, avoids excessive expression, passive responses |
|  | Classroom behavior: Less proactive participation, speaks softly, doesn’t want to be center of attention |
|  | Response patterns: Brief answers, sometimes needs teacher encouragement to speak, lighter tone |
| High Agreeableness | Core traits: Sympathize with others’ feelings, take an interest in other people’s lives, feel others’ emotions |
|  | Behavioral traits: Cooperative and caring, considerate and patient, positive feedback, understanding, high empathy |
|  | Language style: Gentle and friendly, detailed elaboration, accurate expression, caring for others |
|  | Classroom behavior: Happy to help classmates, actively participates in discussions, pays attention to others’ feelings |
|  | Response patterns: Gentle language, accurate expression, rarely makes mistakes, considers others’ feelings |
| Low Agreeableness | Core traits: Can’t be bothered with others’ needs, take advantage of others, not interested in other people’s problems |
|  | Behavioral traits: Competitive, direct and frank, self-focused, less compromising, lacks empathy |
|  | Language style: Direct and clear, sometimes slightly harsh, focuses on personal views |
|  | Classroom behavior: More focused on own performance, may lack patience with others’ opinions |
|  | Response patterns: Direct answers, sometimes slightly harsh, more focused on expressing own views |
| High Conscientiousness | Core traits: Keep things tidy, like order, carry out plans |
|  | Behavioral traits: Serious and responsible, organized, clear goals, persistent, highly self-disciplined |
|  | Language style: Clear and logical, accurate and error-free, strong logic, complete expression |
|  | Classroom behavior: Listens carefully, well-prepared when answering questions, solid grasp of content |
|  | Response patterns: Accurate and complete answers, clear logic, rarely makes mistakes |
| Low Conscientiousness | Core traits: Waste time, find it difficult to get down to work, leave belongings around |
|  | Behavioral traits: Careless, poor organization, easily distracted, lacks persistence |
|  | Language style: Simple and direct, occasional errors, incomplete expression, sometimes inconsistent |
|  | Classroom behavior: Easily distracted, unprepared when answering questions |
|  | Response patterns: Unstable answers, sometimes right sometimes wrong, incomplete expression |
| High Neuroticism | Core traits: Get upset easily, get angry easily, get easily agitated |
|  | Behavioral traits: Nervous and anxious, emotional fluctuations, sensitive and irritable, emotionally unstable |
|  | Language style: Hesitant and indecisive, repetitive backtracking, incoherent expression, full of uncertainty |
|  | Classroom behavior: Easily nervous, sensitive to classroom environment, shows worry when answering |
|  | Response patterns: Full of ’um’, ’uh’ filler words, repeats and backtracks, incoherent expression |
| Low Neuroticism | Core traits: Rarely get irritated, not easily annoyed, feel comfortable with self |
|  | Behavioral traits: Emotionally stable, calm and composed, strong stress adaptability, high self-acceptance |
|  | Language style: Stable and natural, clear logic, calm expression, few emotional fluctuations |
|  | Classroom behavior: Remains calm when facing problems, peaceful attitude when answering |
|  | Response patterns: Stable and natural answers, clear logic, shows inner calm and confidence |
| High Openness | Core traits: Quick to understand things, believe in the importance of art, can handle a lot of information |
|  | Behavioral traits: Strong curiosity, imaginative, accepts new things, flexible thinking, values aesthetics |
|  | Language style: Creative and rich expression, good at association, broad thinking |
|  | Classroom behavior: Curious about new knowledge, good at asking questions, creative answers |
|  | Response patterns: Creative answers, makes associations and extensions, can handle complex information |
| Low Openness | Core traits: Have difficulty understanding abstract ideas, do not like poetry, seldom notice emotional aspects of art |
|  | Behavioral traits: Conservative and traditional, relies on experience, low acceptance, rigid thinking |
|  | Language style: Simple and direct, lacks extension, difficulty going deep, prefers concrete descriptions |
|  | Classroom behavior: Tends to rely on existing knowledge, low acceptance of new content |
|  | Response patterns: Simple and direct answers, lacks extension, prefers concrete answers over abstract analysis |

#### A.3.2 Prompt Template for Persona Stylization

To implement personality-driven dialogue generation, we design a structured prompt template that formalizes the subjective task of persona stylization into a systematic workflow. The system prompt defines the model as an “expert in Big Five personality theory” and establishes rules that govern three major components:

First, student speech processing is grounded in the five BFAS dimensions, with placeholders specifying core traits, behavioral tendencies, linguistic patterns, classroom manifestations, and response styles. This multi-dimensional mapping ensures that student responses preserve semantic meaning while being rewritten into styles consistent with the target persona.

Second, teacher speech processing follows a conservative strategy: original content is preserved whenever possible, with minimal modifications applied only in four predefined cases (e.g., coherence or disambiguation). This guarantees that instructional intent remains intact.

Finally, quality and output requirements enforce strict formatting rules and consistency checks, balancing stylistic fidelity with semantic preservation. By integrating these layers, EduPersona transforms persona-conditioned dialogue generation into a reproducible and scalable process, enabling consistent expansion and evaluation in large-scale educational applications.

#### A.3.3 Behavior–Expression Label Space

The behavior–expression annotation system employs a controlled vocabulary across four dimensions.

**Table A2: Behavior–expression controlled vocabulary with operational definitions.**
| Dimension | Label | Operational Definition |
| --- | --- | --- |
| Behavior | Simple Response | Answers with “yes/no”, “I don’t know”. |
| Mechanical Repetition | Repeats the teacher’s question or content verbatim. |  |
| Standing Answer | Independently provides a complete answer to the teacher’s question. |  |
| Example Explanation | Actively uses examples to explain knowledge. |  |
| Summary Generalization | Summarizes the learned content; expresses personal understanding. |  |
| Active Questioning | Asks questions to express confusion or reflective thoughts. |  |
| Supplementary Speech | Expands or supplements others’ viewpoints. |  |
| Opinion Expression | Refutes or negotiates with others’ statements. |  |
| Emotion | Positive | Positive/engaged/confident state. |
| Confused | Unclear thinking or partial understanding but still trying. |  |
| Negative | Disappointed, frustrated, helpless emotions. |  |
| Expression | Smile | Pleasant, satisfied facial expression. |
| Frown | Confused or dissatisfied facial expression. |  |
| Nod | Agreeing/understanding/affirming action. |  |
| Shake Head | Disagreeing or confused action. |  |
| Silence | No obvious facial change; remains calm. |  |
| Voice | Hesitant | “um/uh/I think/maybe”, disfluent. |
| Confident | Clear, continuous, steady tone. |  |
| Enthusiastic | Higher pitch, emotional emphasis. |  |
| Low-tone | Low and weak voice. |  |
| Neutral | Medium rate and tone; no strong affect. |  |

## Appendix B Evaluation Prompt Template for Task 2 & 3

Our evaluation adopts a dual-task framework combining automatic language detection with adaptive prompting to systematically measure *dialogue realism* (Task 2) and *persona consistency* (Task 3). This design ensures comparability and reproducibility across multilingual datasets.

This framework ensures robustness and reproducibility through three mechanisms. First, language is automatically detected via character-level statistics, enabling adaptive switching between Chinese and English prompts with corresponding persona descriptions. Second, a retry mechanism with exponential backoff and strict JSON parsing with fallback patterns safeguard evaluation reliability. Third, the dual-task design balances realism and persona consistency, scoring only the student’s final response to maintain contextual grounding while ensuring comparability across scenarios.

Overall, this evaluation framework transforms subjective persona assessment into a structured, replicable process, providing the methodological foundation for large-scale experiments and statistical analysis.

## Appendix C Detailed Experimental Results

### C.1 Additional Analysis for Task 1

Figure: (a) Behavior
Refer to caption: x6.png

Fig. [A1](https://arxiv.org/html/2510.04648v1#A3.F1) presents a comprehensive view of Task 1 across three model families, covering both base and fine-tuned settings, four behavioral dimensions, and five evaluation metrics. This integrated view makes it possible to trace how fine-tuning improves performance at both the overall and dimension-specific levels.

The end-to-end gains can be understood as the combined effect of three factors: whether the model produces an output at all (response coverage), whether the output is structurally valid (vocabulary compliance), and whether the valid outputs align correctly with the reference labels (in-vocabulary discrimination). For Qwen and DeepSeek, structural validity is already saturated after fine-tuning, so their improvements mainly come from higher label alignment together with strong response coverage. By contrast, InternLM3 also benefits from improved label alignment, but its relatively low response rate continues to limit its overall performance.

A consistent difficulty ordering is observed across dimensions: Emotion is the easiest, Behavior the hardest, and Expression/Voice fall in between. For example, under fine-tuning, Emotion reaches a mean overall accuracy above 0.65, while Behavior lags at around 0.35. This pattern indicates that residual errors are not random but stem from the inherent challenge of classifying pedagogical acts in Behavior, which requires stronger discourse- and intent-level modeling.

Family-wise, Qwen and DeepSeek converge after fine-tuning, both achieving strict end-to-end accuracy around 0.62. InternLM3, despite showing the largest relative improvement, remains constrained by its limited response coverage. These results suggest that, in this task, the performance ceiling is determined more by dataset design and task structure than by model scale alone.

Looking forward, since structural validity is already near perfect, the most promising directions for further improvement are: (i) increasing response coverage to reduce empty outputs, and (ii) strengthening fine-grained guidance and exemplars for the Behavior dimension to improve label discrimination. All of these trends are clearly reflected in the 24 subplots of Fig. [A1](https://arxiv.org/html/2510.04648v1#A3.F1).

### C.2 Additional Analysis for Task 2

Figure: (a) High Agreeableness
Refer to caption: x30.png

Fig. [A2](https://arxiv.org/html/2510.04648v1#A3.F2) reports Task 2 results for the ten Big Five–based personas across Qwen, InternLM3, and DeepSeek, under both base and fine-tuned conditions. Overall, fine-tuning consistently improves realism across all models, with scores converging around 0.82 (DeepSeek: 0.556$\rightarrow$0.822, InternLM3: 0.648$\rightarrow$0.821, Qwen: 0.702$\rightarrow$0.826).

At the persona level, the best post-tuning alignment is observed for High Neuroticism (HN, 0.891), Low Neuroticism (LN, 0.879), and Low Openness (LO, 0.871), reflecting their ability to naturally capture uncertainty and emotional variation. By contrast, High Conscientiousness (HC, 0.748) and High Openness (HO, 0.764) remain the most challenging personas, yielding the lowest realism scores even after adaptation. This suggests that highly structured traits often overlap with LLMs’ default answer-first tendency, making outputs appear more machine-like than student-like.

In terms of gains, the largest improvements occur for Low Openness (+0.176), High Neuroticism (+0.166), and Low Neuroticism (+0.156), while High Conscientiousness (+0.090) and High Openness (+0.093) improve the least. Model-wise, DeepSeek shows the most dramatic increase (+0.266) despite its low baseline, Qwen maintains a stable advantage with strong pre-tuning performance, and InternLM3 improves moderately but converges with the others in the end. Taken together, these results indicate that student realism is most effectively enhanced through personas reflecting natural uncertainty or variability, while structured or idealized personas remain difficult to simulate authentically.

Cross-task linkage with Task 1.
Relating Task 1 (basic coherence) and Task 2 (student realism) reveals complementary insights. Task 1 captures low-level observable alignment signals, while Task 2 evaluates higher-level subjective perception.

First, response coverage from Task 1 directly constrains realism in Task 2. For instance, InternLM3 maintains a low post-tuning response rate (0.4261), which limits its realism score compared to Qwen and DeepSeek, despite improvement.

Second, the dimension-level difficulty ordering in Task 1 aligns with persona-level differences in Task 2: Emotion is the easiest in Task 1 (OverallAcc=0.65), matching the high realism scores of High/Low Neuroticism and Low Openness personas; Behavior is the hardest (OverallAcc=0.355), consistent with the poor realism of High Conscientiousness and High Openness personas, which demand strict adherence to classroom norms.

Finally, both tasks exhibit post-tuning convergence across families but at different levels: Qwen and DeepSeek converge at OverallAcc=0.62 in Task 1, with InternLM3 trailing due to limited response coverage; in Task 2, all three converge tightly around 0.82. This indicates that EduPersona exerts stronger corrective effects on high-level perception, while low-level structural bottlenecks remain.

In summary, the additional analysis of Task 2 demonstrates that basic coherence is a prerequisite but not sufficient for student realism; achieving realism further depends on the authentic reproduction of persona-specific behaviors and classroom dynamics. Future improvements should emphasize training data and strategies that incorporate “imperfect, human-like student behaviors” to simultaneously enhance observable alignment and perceived authenticity.

### C.3 Additional Analysis for Task 3

Figure: (a) High Agreeableness
Refer to caption: x40.png

Fig. [A3](https://arxiv.org/html/2510.04648v1#A3.F3) presents persona consistency results across the ten personas and three model families, comparing base and fine-tuned conditions. Overall, fine-tuning consistently improves consistency scores and brings the three families to a converged range around 0.84 (DeepSeek: 0.677$\rightarrow$0.837, InternLM3: 0.723$\rightarrow$0.834, Qwen: 0.795$\rightarrow$0.841). This mirrors the findings from Task 1 and Task 2: in Task 1, Qwen and DeepSeek converged in basic coherence after fine-tuning, while in Task 2, all models reached similar levels of student realism. Together, these results confirm that EduPersona fine-tuning reliably improves model performance across layers of subjective ability while reducing inter-model disparities.

At the persona level, a stable difficulty hierarchy emerges. High Neuroticism (0.901), Low Conscientiousness (0.887), and Low Openness (0.873) achieve the highest post-tuning consistency, reflecting hesitation, partial answers, or self-corrections that align well with authentic student behavior. In contrast, High Conscientiousness (0.731) and High Openness (0.779) remain the most challenging to sustain, even after fine-tuning. This pattern echoes Task 2, where the same personas also scored lowest in student realism, indicating that structured, idealized personas are consistently difficult for models to simulate both authentically and consistently.

In terms of improvement magnitude, the largest gains occur for Low Extraversion (+0.146), Low Openness (+0.133), and Low Agreeableness (+0.130), while High Openness (+0.062) and High Conscientiousness (+0.073) improve the least. This highlights that EduPersona fine-tuning is particularly effective for enhancing “non-idealized” student traits, whereas idealized personas remain a persistent challenge.

At the model level, baseline disparities are substantial (Qwen 0.795, InternLM3 0.723, DeepSeek 0.677), but after fine-tuning they narrow dramatically to a range of 0.833–0.841. This again parallels Tasks 1 and 2, reinforcing that EduPersona fine-tuning not only boosts absolute performance but also reduces variance across both models and personas.

In summary, Task 3 demonstrates that persona consistency is more demanding than student realism, yet EduPersona fine-tuning significantly enhances overall stability while reducing inter-model and inter-persona variance. Cross-task comparisons reveal consistent bottlenecks (High Conscientiousness and High Openness) as well as easier-to-model traits (High Neuroticism, Low Conscientiousness, Low Openness), providing a coherent picture of how virtual student agents can be systematically improved.

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 2
presentation: 1
questions: None
soundness: 2
strengths: 1. This paper introduces EduPersona, a benchmark for virtual student agents that is multi-lingual (2 languages), multi-subject (3 subjects), and features ten distinct persona types based on the Big Five theory.
2. This paper proposes an evaluation framework that decomposes abstract subjective abilities into three measurable tasks: Basic Coherence, Student Realism, and Persona Consistency.
summary: This paper introduces EduPersona, a large-scale benchmark designed to evaluate the subjective abilities of virtual student agents powered by Large Language Models (LLMs). The framework assesses performance across three progressive tasks: basic coherence, student realism, and long-term persona consistency. Experiments show that fine-tuning representative LLMs on the EduPersona dataset significantly improves their performance in these areas, validating the benchmark's effectiveness for creating more realistic and consistent AI agents for educational simulations and teacher training.
weaknesses: 1. The paper claims to be "grounded in educational theory" but fails to elaborate on or integrate specific pedagogical or psychological theories that justify its framework.
2. Imprecise Terminology: The use of "Basic Coherence" for Task 1 is questionable. "Coherence" may not be the appropriate term to unify the distinct concepts of behavior, emotion, expression, and vocal style.
3. Overclaims: The paper overlooks highly relevant work in role-play evaluation (e.g., CharacterChat, RoleLLM, CharacterBench), undermining its claims of novelty regarding the evaluation dimensions.
4. The paper asserts that subjective performance can be decoupled into its three proposed tasks (Coherence, Realism, Consistency) without providing a strong theoretical and empirical justification for why this specific decomposition is a comprehensive or appropriate measure.
5. Key dataset details are missing, such as the rationale for subject selection (Chinese, English, etc.). Furthermore, despite "human-audited" claims, the paper provides no quantitative metrics (e.g., expert inspection data, inter-annotator agreement) to substantiate the dataset's quality.
6. Lack of Human Validation: The evaluations for realism (Task 2) and consistency (Task 3) rely on an automatic evaluator (GPT-4), but the paper presents no study correlating the evaluator's judgments with human experts to validate its reliability.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 3
questions: * The evaluation of student realism (Task 2) and persona consistency (Task 3) is central to your contribution, and it relies entirely on scores from a GPT-4o-based evaluator. How can you be sure that this evaluator is reliable? What experiments could you run to validate that its scores for these subjective traits correlate well with human judgments?
* The main experimental result is that fine-tuning on EduPersona improves performance on the EduPersona benchmark. How do you propose to break this circularity? For example, could you show that models fine-tuned on your data also improve on a different, existing benchmark that measures related social or interactional skills?
soundness: 2
strengths: * Originality: It is one of the first works to attempt a systematic, large-scale benchmarking of subjective abilities for AI agents in an educational context. The conceptual framework, which decomposes the broad notion of subjective ability into a progressive hierarchy of three distinct tasks (Basic Coherence, Student Realism, Persona Consistency), is a novel and thoughtful way to operationalize a subjective concept.
* Quality: The authors have constructed a large-scale resource spanning two languages, three subjects, ten personas, and four behavioral dimensions. The detailed process described involved collecting from real classrooms, annotating, and a tenfold expansion via persona stylization, which shows quite some effort in data engineering.
* Clarity: The paper is structured clearly. Figure 1 explains the entire process from data collection to experimental analysis. The detailed descriptions of the dataset and persona configurations in the appendix further enhance the clarity.
* Significance: The work attempts to address the challenge of evaluating AI agents' subjective, human-like abilities in the domain of education. While the paper's evaluation has flaws, it points out an understudied direction for future research.
summary: This paper introduces EduPersona, a large-scale benchmark dataset designed to evaluate the subjective abilities of virtual student agents in educational settings. The authors argue that existing benchmarks focus on objective metrics, such as accuracy, while neglecting crucial subjective traits, including coherence, realism, and persona consistency.

The EduPersona dataset is built from authentic classroom dialogues in two languages (Chinese, English) and three subjects. It is then expanded tenfold by stylizing student responses to align with ten different personas derived from the Big Five personality theory. The dataset also includes multimodal behavior/expression labels. The paper proposes a framework that decomposes subjective ability into three progressive tasks: Task 1 (Basic Coherence—aligning multimodal behaviors with text), Task 2 (Student Realism—assessing whether responses resemble a real student), and Task 3 (Persona Consistency—maintaining a stable persona over time).

The authors benchmark three open-source LLMs and their 30 fine-tuned variants on the EduPersona tasks. The primary finding is that models fine-tuned on their dataset show significant average improvements across all three tasks.
weaknesses: * Circular evaluation: The main finding is that "training on our data makes models better at our tasks," which does not validate the benchmark's utility or relevance to any external ground truth.
* LLM as a Subjective Judge: Using GPT-4o to score realism and consistency is a methodological flaw. The entire evaluation rests on the unproven assumption that the LLM judge is a reliable measure of these complex subjective human-centric concepts.
* Missing human validation: The paper makes claims about modeling human-like traits without presenting any correlation studies demonstrating that its metrics align with human judgments. This is a major omission for a benchmark of this nature.
* Limited scientific insight: The paper does not deliver a strong, novel insight beyond the dataset itself. The results largely confirm the intuitive expectation that fine-tuning works.
* Readability: It is pretty hard to read some texts in Figures 2, 4, 6, and 7 (small font and crowded content).

---

confidence: 5
contribution: 1
initial_rating: 4
presentation: 3
questions: Please see weakness
soundness: 2
strengths: Strengths
- Timely topic focusing on the AI in education and  LLM application
- Reals world class room dataset
summary: This paper presents an study to evaluate the quality of benchmarks of large language models (LLMs) deployed for online education and their subjective abilities. The authors present a benchmark spanning two languages, three sub- jects, and ten persona types based on the Big Five theory.
weaknesses: The claim in the introduction  is not correct “Yet existing evaluation frameworks remain focused on objective tasks such as question answering and accuracy (Lu et al., 2022; Huang et al., 2023; Ang et al., 2023), overlooking the subjective abilities essential to classroom practice.”

The paper misses recent literature. The proposed scenarios by the authors can be done by the Zhou and colleagues approach by simulating a student and giving personalities [1] . Please also see work by Dr. Ashok Goel and Dr. Noboru Matsuda.

References
1. Zhou, X., Zhu, H., Mathur, L., Zhang, R., Yu, H., Qi, Z., Morency, L.P., Bisk, Y., Fried, D., Neubig, G. and Sap, M., SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents. In The Twelfth International Conference on Learning Representations.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 2
questions: * Evaluation validity: How do you mitigate circularity when GPT-style models are used both for label generation and evaluation? Have you assessed agreement with independent human raters at scale?

* Persona realism: Since persona stylization is synthetic how confident are you that the personas reflect real classroom behaviour?
soundness: 2
strengths: The paper addresses a timely problem of evaluating subjective abilities of classroom-oriented virtual students. The dataset is motivated, and includes persona and multimodal annotation, providing coverage across subjects and languages. The proposed three-task progression is clear and seemingly grounded in educational theory.
summary: This paper introduces EduPersona, a large-scale benchmark designed to evaluate subjective abilities of virtual student agents in classroom settings. The dataset spans two languages (Zh/En), three subjects, and ten Big-Five-based personas, built from 1,308 authentic classroom dialogue rounds. The authors propose a three-task evaluation framework: (1) basic multimodal coherence, (2) student realism, and (3) persona consistency. Experiments using LoRA fine-tuning on three open-source LLM families showimprovements across all tasks and reveal persona-specific difficulty patterns.
weaknesses: The reliance on GPT-based annotation and evaluation raises concerns about circularity and unclear inter-annotator reliability. 

The persona data is automatically generated, limiting ecological validity. 

Educational motivation is plausible, but actual downstream utility for teachers or pedagogy is not demonstrated. 

It is unclear whether performance gains generalize beyond the benchmark itself. 

Analysis could better quantify statistical significance and reporting of evaluation robustness.