Venue: NAACL 2025

# CharacterGPT: A Persona Reconstruction Framework for Role-Playing Agents

Jeiyoon Park\({}^{1}\),  Chanjun Park\({}^{2\dagger}\),  Heuiseok Lim\({}^{2\dagger}\)

\({}^{1}\) SOOP,  \({}^{2}\) Korea University

naruto@sooplive.com

{bcj1210, limhseok}@korea.ac.kr

Corresponding Author

###### Abstract

The recent introduction of the Assistants API highlights its potential for large language models (LLMs) in role-playing agents (RPA). However, maintaining consistent character personas remains a significant challenge due to variability in information extraction, which frequently omits critical elements such as backstory or interpersonal relationships. To address this limitation, we introduce CharacterGPT, a framework designed to dynamically reconstruct character personas through Character Persona Training (CPT). This approach incrementally updates personas by extracting traits from chapter-wise novel summaries, reflecting the progression of the narrative. Our framework is evaluated through Big Five personality evaluations and creative tasks, in which characters generate original narratives, demonstrating the efficacy of CharacterGPT in preserving persona consistency. The code and results are available at [https://github.com/Jeiyoon/charactergpt](https://github.com/Jeiyoon/charactergpt)

## 1 Introduction

The rapid advancements in large language models (LLMs) have positioned them as the core module of various AI systems OpenAI (2023a, c; Anthropic, 2023; Google, 2024; DeepSeek-AI et al., 2024, 2025), enabling a wide range of applications. Building on this progress, the recent introduction of the Assistants API OpenAI (2023b), a tool designed for document-based information retrieval, demonstrates the potential of LLM in multiple domains, especially in role-playing agents (RPA) Kim et al. (2019); Yu et al. (2023); Jiang et al. (2023); Park et al. (2023); Wang et al. (2023); Zhang et al. (2024); Kong et al. (2024); Wang et al. (2025). However, RPAs that rely solely on documents as input often face problems of inconsistent information extraction, where key personality traits or background knowledge are omitted, leading to degraded persona coherence Sadeq et al. (2024). For example, as illustrated in Figure 1, when the Assistants API is provided with an unstructured Wiki document about the novel _God's Blessing on This Wonderful World!_, it often fails to provide accurate responses, while

Figure 1: Comparison of response accuracy between persona-based GPT-4 assistants utilizing unstructured versus structured character traits as input. When provided with unstructured traits, the assistant demonstrates limited success in generating accurate responses. In contrast, the use of structured traits significantly improves the correctness of the assistant’s responses.



structured character traits produce more reliable, role-specific answers.

In this paper, we propose a novel framework called _CharacterGPT_, which addresses this challenge through a structured persona reconstruction process. Drawing inspiration from cognitive memory models, we introduce _Character Persona Training (CPT)_, a method that incrementally updates character personas by extracting traits from chapter-wise summaries of novels. This approach mirrors how human memory consolidates information into schemas over time van Kesteren and Meeter (2020), enabling more consistent and contextually appropriate responses from RPAs.

_CPT_ operates by identifying eight essential traits--_personality, physical description, motivations, backstory, emotions, relationships, growth and change_, and _conflict_--based on character analysis literature Forster (1927); Reams (2015). For each chapter of a novel, these traits are extracted from summaries and appended to a character's evolving persona, forming a document that reflects the character's development in chronological order. Note that the extracted traits are updated separately to ensure they remain distinct and are not coalesced. This reconstructed persona document is then used as input to the Assistants API, allowing it to generate more contextually accurate and coherent responses based on the character's evolving identity. This framework minimizes information loss and computational cost associated with traditional document-based retrieval methods, as it systematically organizes and updates persona traits over time. Moreover, by generating personas at different narrative points, _CharacterGPT_ enables users to interact with characters at specific moments within the novel (e.g., a user can experience a hero's thoughts just before confronting the _Devil King!_).

We evaluate the effectiveness of _CharacterGPT_ through human assessments, examining how well our method captures role-specific knowledge. Each character undergoes the Big Five Inventory (BFI) personality test Barrick and Mount (1991) to evaluate personality consistency, and characters are tasked with generating short narratives to assess creative capabilities. 7 crowd-workers evaluate these narratives across six metrics using a 5-point Likert scale. Our results demonstrate that _CharacterGPT_ significantly improves persona consistency, controllability, and role-specific knowledge compared to standard document-based systems.

## 2 Proposed Method

The goal of _CharacterGPT_ is to build a personal-based assistant, denoted as \(f\), which takes as input a persona document \({\cal D}\) and an inference prompt \({\cal P}_{f}\), and generates a character response \({\cal R}\). Let \({\cal D}=\left\{s_{1},s_{2},\ldots,s_{N}\right\}\) represent a persona document with \(N\) sentences. A naive approach using the Assistant s API would treat the entire sampled document as input. However, as illustrated in Figure 1, this method often fails to capture essential character traits, leading to inconsistent and unnatural responses. To address this, we reorganize the persona document into a refined version \({\cal D}_{r}\) and define the assistant's output as:

\[{\cal R}=f({\cal D}_{r},{\cal P}_{f}) \tag{1}\]

### Preliminaries

**Character Traits.** We identify eight key traits that define each character Forster (1927); Reams (2015):

* _Personality_: Core personality traits such as bravery, introversion, or wit.
* _Physical Description_: The character's physical appearance.
* _Motivations_: The character's goals and desires driving their actions.
* _Backstory_: Historical background shaping the character's personality and motivations.
* _Emotions_: The range of emotions that influence the character's responses.
* _Relationships_: Interactions and relationships with other characters.
* _Growth and Change_: The character's development over the course of the narrative.
* _Conflict_: Internal or external conflicts faced by the character.

**Persona Document.** We analyze four distinct characters: _Megumin, Anya Forger, Frieren_, and _Hitori Gotoh_ (Figure 6), gathering character information and story summaries from Namuwiki1,2. Table 1 summarizes the data collected, including chapter counts, token statistics, and character dialogues. _(info)_ refers to detailed character information, _(dialogue)_ refers to collected lines, and _(trained)_ refers to novel summaries used for CPT.

 



### Persona Initialization

Simply providing a sampled document for trait extraction is insufficient. To address this limitation, we propose a two-stage persona reconstruction process: (i) _Initialization_ and (ii) _CPT_.

During the _Initialization Phase_, we assume no significant narrative progression (i.e., prior to CPT) and remove all content tied to the story's progress. To optimize the persona document, we organize the collected character information into five core traits: _Personality_, _Physical Description_, _Motivations_, _Backstory_, and _Relationships_. These form the initialization persona:

\[{\cal D}_{init}=\left\{{\cal D}_{per},{\cal D}_{phy},{\cal D}_{mot},{\cal D}_{ back},{\cal D}_{Rel}\right\} \tag{2}\]

Traits such as _emotions, growth and change_, and _conflict_ are excluded at this stage, as they are more relevant to narrative progression and are addressed in the CPT phase.

### Character Persona Training

**Trait Classification.** Intuitively, human knowledge can be broadly categorized into internal and external attributes. Internal attributes (Type A) define the character's intrinsic traits (e.g., personality), while external attributes (Type B) are accumulated through interactions with the environment (e.g., relationships). Inspired by Park et al. (2023), we classify the eight traits into two types:

* **Type A**: Personality, Physical Description, Motivations
* **Type B**: Backstory, Emotions, Relationships, Growth and Change, Conflict

During CPT, Type A traits are generalized to refine the protagonist's core attributes, while Type B traits accumulate role-specific external knowledge.

**Training Phase.** CPT updates the character persona at each epoch by extracting role-specific traits from chapter summaries (Figure 7):

\[{\cal T}^{i}_{t}=\left\{\begin{array}{ll}h(g({\cal D}_{i},{\cal P}_{g}),{ \cal P}_{h}),&\mbox{if}\quad t\in\text{Type A}\\ g({\cal D}_{i},{\cal P}_{g}),&\mbox{otherwise}\end{array}\right. \tag{3}\]

, where \(i\) represents the epoch, \({\cal D}_{i}\) is the chapter summary, \(g\) refers to the Assistants API with prompt \({\cal P}_{g}\), \(h\) is an LLM-based generalization function with prompt \({\cal P}_{h}\), \(t\) is the trait, and \({\cal T}^{i}_{t}\) is the extracted trait. For Type A traits, generalization refines internal attributes, while Type B traits are appended to the persona document.

### CharacterGPT

In Section 2.3, we leverage Character Persona Training (CPT) to iteratively build each character's persona. This method offers two key advantages: (i) CharacterGPT minimizes information loss and computational cost by aligning persona accumulation with narrative progression, and (ii) CharacterGPT is the first system to store and update a protagonist's persona at each epoch, allowing users to engage with characters at specific narrative points.

Figure 2 illustrates how the final persona \({\cal D}_{r}\) is composed, including the initialized persona \({\cal D}_{init}\), the trained persona \({\cal D}_{train}\), and tone \({\cal T}_{v}\):

\[{\cal D}_{r}={\cal D}_{init}+{\cal D}_{train}+{\cal T}_{v} \tag{4}\]

While \({\cal T}_{v}\) can enhance dialogue naturalness, the collected data mainly includes character information and summaries with limited dialogue. Further work can explore this area in more detail.

\begin{table}
\begin{tabular}{l c c c c} \hline  & Megumian & Anya & Frieren & Hitori \\ \hline \# Chapters & 16 & 30 & 11 & 12 \\ \# Tokens (novel) & 27,200 & 16,096 & 12,191 & 8,647 \\ \# Tokens (info) & 12,868 & 17,026 & 19,290 & 20,555 \\ \# Tokens (info)† & 4,015 & 2,498 & 9,236 & 1,572 \\ \# Tokens (dialogue) & 1,131 & 681 & 87 & 301 \\ \# Tokens (trained) & 31,917 & 52,207 & 32,328 & 24,039 \\ \hline \end{tabular}
\end{table}
Table 1: Statistics of the number of collected tokens and chapters for each character. † refers to the number of refined character information tokens in Section 2.2.

Figure 2: An example of CharacterGPT (_Anya Forger_). (Top) Character Personsa Training process. (Bottom) CharacterGPT generating responses that align with the character’s persona.





## 3 Experiments

### Setup

We implement _CharacterGPT_ using the Assistants API alongside GPT-4 Turbo (version "gpt-4-1106-preview"). To verify model compatibility, we also conduct experiments, including ablation studies, using ChatGTP (version "gpt-3.5-turbo-1106"). Note that ChatGPT supports the Retrieval functionality of the Assistant's API solely for this model version. The generalization function \(h\) is configured with a maximum token length of 4096 and a temperature setting of 0.7.

### Evaluation Protocols

**Tasks.** We address the primary research question (RQ) in two key tasks: _1) How to better exploit character persona_, and _2) How to encourage characters to use imagination for generating new ideas._

**Task for RQ1: Persona Evaluation.** For persona evaluation, we compare the personality traits analyzed by one of the authors, who has read all

\begin{table}
\begin{tabular}{c c c c c c} \hline \multirow{2}{*}{**Test**} & **Friends** & **CandiderGT** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-4** & **CP1-4** \\  & & **Friends** & **CandiderGT-Home** & **GP1-

four novels multiple times, with the traits generated by LLMs under various settings. For fairness, we average the experimental results across the four characters for each model.

**Task for RQ2: Story Generation.** The story generation task is evaluated based on common aspects in generated story assessment (Wen et al., 2023; Chiang and Lee, 2023a; Karpinska et al., 2021): (i) _Grammar_, (ii) _Coherence_, (iii) _Likability_, (iv) _Relevance_, (v) _Complexity_, and (vi) _Creativity_. Although automatic evaluation methods using LLMs are being actively developed (Sottana et al., 2023; Chiang and Lee, 2023b; Liu et al., 2023; Zheng et al., 2023; Samuel et al., 2024), metrics and benchmarks for assessing human preferences are still inadequate. Therefore, we conduct extensive human evaluations using 7 crowd-workers instead of relying on LLM-based evaluations.

**Case Study.** We further investigate the performance of CharacterGPT in interacting with users at specific points in the story. Additionally, we examine how role-specific attributes (Type A and Type B) evolve through CPT.

### Results for Persona Evaluation

In Section 2.4, we created four distinct characters to assess how well models capture their personas. Following evaluation protocols similar to (Wang et al., 2024; Jiang et al., 2023), we conducted the Big Five Inventory (BFI) personality test (Barrick and Mount, 1991), which consists of 24 questions for each of the five traits (_Openness to experience_, _Conscientiousness_, _Extraversion_, _Agreeableness_, and _Neuroticism_), totaling 120 questions. The test results were then converted into facet values for each trait. For example, in the _Agreeableness_ (AGR) trait, as shown in Table 2, humans perceive Megumin as trusting others' intentions (_Trust_), making judgments based on emotions (_Tendermindedness_), but being less direct (_Straightforwardness_) and somewhat arrogant or self-agrandizing (_Modesty_).

In Table 2, Table 3, Table 4, and Table 5, we compare model predictions against human-predicted values by calculating the gap for each facet. Two metrics are reported: the number of facets where a model has the smallest gap with human predictions (# Wins), and the sum of the absolute gaps (\(\Sigma|d|\)). A higher # Wins indicates better performance, while a lower \(\Sigma|d|\) reflects closer alignment with human judgment. Our method demonstrates improvements in both metrics when applied to ChatGPT and GPT-4, indicating that utilizing a structured character persona significantly enhances a model's ability to capture a character's personality compared to using an unstructured document input. For instance, in Megumin's _Neuroticism_, GPT-4 with unstructured traits predicted that Megumin would be prone to depression, while both our method and human concluded otherwise.

### Results for Story Generation

To evaluate the models' controllability and their utilization of role-specific knowledge (Alabdulkarim et al., 2021; Wen et al., 2023), we provided each character with the following prompt: _"Based on the given text file, imagine an engaging and specific future episode about what will happen to you, and write it as a novel of approximately 2000 words."_ A total of 32 stories were generated, with four stories created for each character's setting.

We employed 7 crowd-workers to evaluate the quality of the generated stories based on six metrics using a 5-point Likert scale, as outlined in Section 3.2. The participants were informed that the stories were generated by an AI, as prior research suggests that awareness of whether a text is human-generated or AI-generated may influence the evaluation results (Jiang et al., 2023). Table 6 presents the human evaluation results for story

Figure 4: Total sum of \(|d|\) for each character in ChatGPT and GPT-4 settings (\(\Sigma\Sigma|d|\)). The smaller value, the better.

Figure 3: Total sum of # Wins for each character in ChatGPT and GPT-4 settings (\(\Sigma\) # Wins). The larger value, the better.

 

* [25] M. C. Bader, R. D. Kiefer, and A. J. Brox, "A new approach to the design of a multi-objective optimization problem in a multi-objective optimization framework," _Journal of Machine Learning Research_**106**(4):379-483, 2009.
* [26] S. G. Barnett, P. N. Torr, and L. E. Oliva, "The role of deep learning for large scale data recognition," _IEEE Transactions on Pattern Analysis and Machine Intelligence_**10**(1):1-10, 2010.
* [27] T. H. Nguyen, "Learning multiple layers with high probability spaces," _arXiv preprint arXiv:1902.02788v1_ (2019).
* [28] Y. Bengio, I. Guyon, and J. Schmidhuber, "Deep residual networks for image classification," _CoRR_**106**(4):e130902, 2010.
* [29] X. Chen, Z. Liu, W. Li, and Q. Weinberger, "ImageNet: A simple way to improve the performance of deep neural networks," _Neural Networks_**10**(2):e130902, 2010.
* [30] Y. Bengio, V. LeCun, and O. Vitells, "Generative adversarial nets for large scale data recognition," _Advances in Neural Information Processing Systems_**10**(1):e130902, 2010.

[MISSING_PAGE_POST]

 

generation under different GPT-4 settings. Our approach demonstrates improved performance across all six metrics, with particularly notable improvements in Likability, Complexity, and Creativity. The experimental results indicate that, while GPT-4 exhibits strong baseline performance, integrating structured personas through our method yields significantly higher human preferences compared to using unstructured document inputs alone. Further detailed information can be found in Appendix E

### Case Study

**Points in Time.** A notable advantage of our proposed method is its ability to allow users to interact with characters at specific points in the narrative. For instance, as discussed in Section 2.3, we trained the model using summaries of the novel featuring Megumin, which is divided into 16 chapters. Consequently, our method generates 16 separate models, one for each epoch. Figure 5 shows that CharacterGPT can vividly express the character's thoughts and emotions at specific moments, leveraging the character persona created through the Initialization and CPT processes.

**Ablation Study.** Figure 9 presents the results of our ablation study, comparing models with and without CharacterGPT. As anticipated, characters not utilizing CharacterGPT fail to accurately capture the nuances of their personas. For example, Hitori, who is typically shy and struggles with fluent speech, is not properly represented by GPT-4 without CharacterGPT. Similarly, Frieren without CharacterGPT exhibits inconsistencies in persona, including awkward and unnatural dialogue, as well as hallucinations (e.g., Frieren is interested in "magic" rather than her canonical interest in "arcane arts"). These findings demonstrate that CharacterGPT is significantly more effective at preserving the integrity of a character's persona.

**Type A and Type B.** Figure 9 further illustrates how each character evolves through the CPT process. For example, Frieren, who begins as a character indifferent to human emotions and solely focused on magic, gradually becomes more empathetic towards human emotions as she embarks on her journey with her companions (Type A). Likewise, Hitori, initially portrayed as a loner with no friends, eventually forms close bonds with her bandmates, particularly with Ikuyo Kita, demonstrating her growth and development (Type B). These results highlight the potential of our method for applications in novel generation, role-playing, and more complex agent-level tasks.

## 4 Conclusion

We introduce CharacterGPT, a persona-based assistant designed to enhance persona consistency by utilizing structured character traits as input. The proposed framework consists of two primary phases: initialization and training. In the initialization phase, we treat the character as if the narrative has not yet advanced, thus excluding any content related to story progression. During the training phase, the character persona is incrementally refined at each epoch by extracting relevant traits from chapter summaries, emulating the natural development of a character throughout a novel. Our approach has been rigorously evaluated through human assessments and case studies, demonstrating its effectiveness in preserving persona coherence and retaining character-specific knowledge. Future directions include extending this framework to enable deeper reasoning and decision-making capabilities, supported by more comprehensive personality models.

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Model** & **Grammar** & **Coherence** & **Likability** & **Relevance** & **Complexity** & **Creativity** \\ \hline
**Megumnia** & 3.79 & 3.82 & 3.11 & **4.21** & 2.46 & 2.86 \\
**Mgemuin + Ours** & **4.11** & **4.00** & **3.71** & 4.11 & **3.46** & **3.29** \\ \hline
**Anya** & **4.29** & 3.82 & 3.39 & 3.86 & **3.61** & 3.68 \\
**Anya + Ours** & 4.25 & **4.00** & **3.79** & **4.00** & 3.43 & **3.89** \\ \hline
**Frieren** & 4.29 & 3.89 & 3.50 & 3.86 & 3.93 & 3.79 \\
**Frieren + Ours** & **4.32** & **3.96** & **3.71** & **4.21** & **4.04** & **3.86** \\ \hline
**Hitori** & **4.36** & 4.04 & 3.57 & **4.18** & 3.43 & 3.50 \\
**Hitori + Ours** & **4.36** & **4.39** & **3.82** & **4.18** & **3.96** & **3.93** \\ \hline \hline
**GPT-4 (avg)** & 4.18 & 3.89 & 3.39 & 4.03 & 3.36 & 3.46 \\
**GPT-4 + Ours (avg)** & **4.26** & **4.09** & **3.76** & **4.13** & **3.72** & **3.74** \\ \hline \hline \end{tabular}
\end{table}
Table 6: Human evaluation of generated stories. The backbone model is the same as GPT-4, and four stories for each setting, a total of 32 stories are generated and evaluated by 7 crowd-workers using a 5-point Likert scale.

 



### Limitations

This study presents three key limitations that can be addressed in future work. First, **Key Traits**: Although CharacterGPT demonstrates strong performance in terms of persona consistency and knowledge retention, the selection of key traits was not formally validated beyond empirical results. For instance, traits such as _Cultural and Social Context_, which were not included in this study, may be essential for character modeling (e.g., a character's diplomatic situation). Further exploration is needed to investigate the importance and necessity of these traits. Additionally, while _Voice and Speech Pattern_ is recognized as a critical trait, the dataset used in this study lacked substantial dialogue, limiting our ability to fully explore this dimension. Future work should focus on identifying how much dialogue is necessary to effectively model a character's speech patterns.

Second, **Reasoning Ability**: While CharacterGPT shows significant improvements in persona consistency and knowledge utilization, its reasoning capabilities remain underexplored. In Table 6, we tasked models with imagining future scenarios and writing stories. Despite outperforming GPT-4 on metrics such as Likability, Complexity, and Creativity, these scores did not exceed 4 points, indicating room for improvement in reasoning abilities. Further research is necessary to enhance the depth of reasoning in persona-based models.

Third, **Hallucinations**: Although ongoing research has made strides in understanding and reducing hallucinations in LLM responses, few studies have addressed hallucinations in persona-based tasks. This is likely due to the fictional nature of persona knowledge, which often diverges from realworld facts (e.g., a image using flame magic). Developing cost-effective benchmarks for each novel is a challenge, and future work should focus on creating efficient methods to handle persona-related hallucinations.

## Ethics Statement

This research adheres to ethical guidelines aimed at ensuring the integrity and fairness of all experiments. We took measures to avoid bias by selecting a diverse group of evaluators and conducting human evaluations in a fair and transparent manner. All data collected from Namuwiki complied with usage permissions and did not contain any personally identifiable information. The dataset, written in Korean, was solely used for academic purposes. Furthermore, our approach to using the Assistants API was transparent, with no modifications that could obscure the model's functionality.

Our experimental setup followed strict ethical standards, ensuring data privacy and protection. All persona documents and prompts used were either publicly available, anonymized, or ethically created. Human participants in the study were fully informed about the nature and purpose of the research, and they had the right to withdraw at any time without any penalty. By adhering to these principles, we aim to contribute to AI research in a manner that is not only innovative but also ethically responsible, ensuring that our work respects privacy, intellectual property, and the well-being of all participants.

## Acknowledgments

This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(NRF-2021R1A6A1A03045425)and supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) (RS-2024-00398115, Research on the reliability and coherence of outcomes produced by Generative AI)

## References

* Abbasian et al. [2023] Mahyar Abbasian, Iman Azimi, Amir M Rahmani, and Ramesh Jain. 2023. Conversational health agents: A personalized llm-powered agent framework. _arXiv preprint arXiv:2310.02374_.
* Abu-Rasheed et al. [2024] Hasan Abu-Rasheed, Mohamad Hussam Abdulsalam, Christian Weber, and Majid Fathi. 2024. Supporting student decisions on learning recommendations: An llvm-based chatbot with knowledge graph contextualization for conversational explainability and mentoring. _arXiv preprint arXiv:2401.08517_.
* Alabdulkarim et al. [2021] Amal Alabdulkarim, Siyan Li, and Xiangyu Peng. 2021. Automatic story generation: Challenges and attempts. In _Proceedings of the Third Workshop on Narrative Understanding_, pages 72-83, Virtual. Association for Computational Linguistics.
* Anthropic [2023] Anthropic. 2023. Introducing Claude.
* Barrick and Mount [1991] Murray R Barrick and Michael K Mount. 1991. The big five personality dimensions and job performance: a meta-analysis. _Personnel psychology_, 44(1):1-26.
* Chen et al. [2023] Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Sesay Jaward, Karlsson Borje, Jie Fu, and Yemin Shi. 2023.



Autoagents: The automatic agents generation framework. _arXiv preprint_.
* Chiang and Lee (2023a) Cheng-Han Chiang and Hung-yi Lee. 2023a. Can large language models be an alternative to human evaluations? In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 15607-15631, Toronto, Canada. Association for Computational Linguistics.
* Chiang and Lee (2023b) Cheng-Han Chiang and Hung-yi Lee. 2023b. Can large language models be an alternative to human evaluations? In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 15607-15631, Toronto, Canada. Association for Computational Linguistics.
* DeepSeek-AI et al. (2021) DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuang Wang, Bochao Wu, Bei Feng, Chenga Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Delhi Chen, Dongile Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fului Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han geo, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junpie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexi Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qianchene Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wandgine Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjin Gao, Wenbin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangye Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyu Zhou, Xianzwal Wang, Xinna Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, YYyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuhenda You, YuJuan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhehan Xu, Zhenda Xie, ZhengaYan Zhong, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zhibui Gu, Zijia Zhu, Zijiu Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, Zhen Zhang, Zheheng Tu, Zijiang Yu, Zhibong Ma, Zijia Zhu, Zijium Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, Zhen Zhang, Zheheng Ta, Zijiang Yu, Zhibong Ma, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, Zhen Zhang, Zheheng Tu, Zijiang Yu, Zhibong Ma, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, Zheheng Tu, Zijiang Yu, Zhibong Ma, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, Zheheng Tu, Zijiang Yu, Zhibong Ma, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijium Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Zijing Liu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Ziling Li, Ziwei Xie, Ziyang Song, Zijia Zhu, Zijia Zhu, Ziling Li, Ziwei Xie 

Conference on Knowledge Discovery and Data Mining_, KDD '23, New York, NY, USA. Association for Computing Machinery.
* Feizi-Derakhshi et al. (2022) Ali-Reza Feizi-Derakhshii, Mohammad-Reza Feizi-Derakhshi, Majid Ramezani, Narjes Nikzad-Khasmakhi, Meysam Asgari-Chenaghlu, Taymaz Akan, Mehrdad Ranjbar-Khadivi, Elnaz Zafarni-Moutatar, and Zoleikha Jahanbakhsh-Naghadeh. 2022. Text-based automatic personality prediction: A bibliographic review. _Journal of Computational Social Science_, 5(2):1555-1593.
* Forster (1927) Edward Morgan Forster. 1927. _Aspects of the Novel_. Harcourt, Brace.
* Gallotta et al. (2024) Roberto Gallotta, Graham Todd, Marvin Zammit, Sam Earle, Antonios Liapis, Julian Togelius, and Georgios N. Yannakakis. 2024. Large language models and games: A survey and roadmap. _Preprint_, arXiv:2402.18659.
* Google (2024) Google. 2024. Our next-generation model: Gemini 1.5.
* Jayaraman et al. (2023) Ashok Kumar Jayaraman, Gayathri Ananthakrishnan, Tina Esther Trueman, and Erik Cambria. 2023. Text-based personality prediction using xlnet. _Advances in Computers_.
* Jiang et al. (2023) Hang Jiang, Xiajie Zhang, Xubo Cao, and Jad Kabbara. 2023. Personallm: Investigating the ability of large language models to express big five personality traits. _Preprint_, arXiv:2305.02547.
* Karpinska et al. (2021) Marzena Karpinska, Nader Akoury, and Mohit Iyyer. 2021. The perils of using Mechanical Turk to evaluate open-ended text generation. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 1265-1285, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Kim et al. (2019) Hannah Kim, Denys Katerenchuk, Daniel Billet, Jun Huan, Haesun Park, and Boyang Li. 2019. Understanding actors and evaluating personae with gaussian embeddings. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 33, pages 6570-6577.
* Kong et al. (2024) Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruqi Sun, Xin Zhou, Enzhi Wang, and Xiaohang Dong. 2024. Better zero-shot reasoning with role-play prompting. In _Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 4099-4113, Mexico City, Mexico. Association for Computational Linguistics.
* Li et al. (2023a) Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi MI, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, and Haozhen Sun. 2023a. Chatharuhi: Reviving anime character in reality via large language model. _Preprint_, arXiv:2308.09597.
* Li et al. (2023b) Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitriu Khizbulin, and Bernard Ghanem. 2023b. Camel: Communicative agents for "mind" exploration of large language model society. In _Thirty-seventh Conference on Neural Information Processing Systems_.
* Li et al. (2018) Jingjing Li, A Abbasi, F Ahmad, and Hsinchun Chen. 2018. Deep learning for psychometric nlp. In _the 28th Workshop on Information Technologies and Systems (WITS)_, pages 15-16.
* Liu et al. (2023) Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Rucochen Xu, and Cheng Guang Zhu. 2023. G-eval: NLG evaluation using gpt-4 with better human alignment. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 2511-2522, Singapore. Association for Computational Linguistics.
* Ma et al. (2020) Yukun Ma, Khanh Linh Nguyen, Frank Z Xing, and Erik Cambria. 2020. A survey on empathetic dialogue systems. _Information Fusion_, 64:50-70.
* Maas et al. (2023) Carey Maas, Saatchi Wheeler, Shamash Billington, et al. 2023. To infinity and beyond: Show-1 and showrunner agents in multi-agent simulations. _To infinity and beyond: Show-1 and showrunner agents in multi-agent simulations_.
* Mo et al. (2018) Kaixiang Mo, Yu Zhang, Shuangyin Li, Jiajun Li, and Qiang Yang. 2018. Personalized a dialogue system with transfer reinforcement learning. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 32.
* Onorati et al. (2023) Teresa Onorati, Alvaro Castro-Gonzalez, Javier Cruz del Valle, Paloma Diaz, and Jose Carlos Castillo. 2023. Creating personalized verbal human-robot interactions using llm with the robot mini. In _International Conference on Ubiquitous Computing and Ambient Intelligence_, pages 148-159. Springer.
* OpenAI (2023a) OpenAI. 2023a. Gpt-4 technical report. _Preprint_, arXiv:2303.08774.
* OpenAI (2023b) OpenAI. 2023b. no date provided. introducing assistants api.
* OpenAI (2023c) OpenAI. 2023c. no date provided. introducing chatgpt.
* Park et al. (2023) Joan Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In _Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology_, pages 1-22.
* Reams (2015) Jack Reams. 2015. Characterization in fiction.
* Sadeq et al. (2024) Nafis Sadeq, Zhouhang Xie, Byungkyu Kang, Prrait Lamba, Xiang Gao, and Julian McAuley. 2024. Mitating hallucination in fictional character role-play. In _Findings of the Association for Computational Linguistics: EMNLP 2024_, pages 14467-14479, Miami, Florida, USA. Association for Computational Linguistics 

Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2024. Lamp: When large language models meet personalization. _Preprint_, arXiv:2304.11406.
* Samuel et al. (2024) Vinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari, Ashwin Kalyan, Tanmay Rajpurohit, Ameet Deshpande, Karthik Narasimhan, and Vishvak Murahari. 2024. Personagym: Evaluating persona agents and llms. _Preprint_, arXiv:2407.18416.
* Shanahan et al. (2023) Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role-play with large language models. _Preprint_, arXiv:2305.16367.
* Sottana et al. (2023) Andrea Sottana, Bin Liang, Kai Zou, and Zheng Yuan. 2023. Evaluation metrics in the era of GPT-4: Reliably evaluating large language models on sequence to sequence tasks. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 8776-8788, Singapore. Association for Computational Linguistics.
* Tanwar et al. (2024) Henansh Tanwar, Kunal Shrivastava, Rahul Singh, and Dhruv Kumar. 2024. Opinebot: Class feedback reimagined using a conversational llvm. _arXiv preprint arXiv:2401.15589_.
* Uludaglin and Oguz (2023) Muhtar Cagkan Uludaglin and Kaya Oguz. 2023. Non-player character decision-making in computer games. _Artif. Intell. Rev._, 56(12):14159-14191.
* van Kesteren and Meeter (2020) Marlieke Tina Renee van Kesteren and Martijn Meeter. 2020. How to optimize knowledge construction in the brain. _npj Science of Learning_, 5(1):5.
* Wang et al. (2025) Xiaoyang Wang, Hongming Zhang, Tao Ge, Wenhao Yu, Dian Yu, and Dong Yu. 2025. Opencharacter: Training customizable role-playing llms with large-scale synthetic personas. _Preprint_, arXiv:2501.15427.
* Wang et al. (2024) Xintao Wang, Yunze Xiao, Jen tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, and Yanghua Xiao. 2024. Incharacter: Evaluating personality fidelity in role-playing agents through psychological interviews. _Preprint_, arXiv:2310.17976.
* Wang et al. (2023a) Zekun Wang, Ge Zhang, Kexin Yang, Ning Shi, Wangchunshu Zhou, Shaochun Hao, Guangzheng Xiong, Yizhi Li, Mong Yuan Sim, Xiuying Chen, Qingqing Zhu, Zhenzhu Yang, Adam Nik, Qi Liu, Chenghua Lin, Shi Wang, Ruibo Liu, Wenhu Chen, Ke Xu, Dayiheng Liu, Yike Guo, and Jie Fu. 2023a. Interactive natural language processing. _Preprint_, arXiv:2305.13246.
* Wang et al. (2023b) Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Man Zhang, et al. 2023b. Rolellum: Benchmarking, eliciting, and enhancing role-playing abilities of large language models. _arXiv preprint arXiv:2310.00746_.
* Wei et al. (2023) Jimmy Wei, Kurt Shuster, Arthur Szlam, Jason Weston, Jack Urbanek, and Mojtaba Romeili. 2023. Multi-party chat: Conversational agents in group settings with humans and models. _Preprint_, arXiv:2304.13835.
* Wen et al. (2023) Zhihua Wen, Zhiliano Tian, Wei Wu, Yukin Yang, Yanqi Shi, Zhen Huang, and Dongsheng Li. 2023. GROVE: A retrieval-augmented complex story generation framework with a forest of evidence. In _Findings of the Association for Computational Linguistics: EMNLP 2023_, pages 3980-3998, Singapore. Association for Computational Linguistics.
* Yu et al. (2023) Mo Yu, Jiangnan Li, Shunya Yao, Wenjie Pang, Xiaochen Zhou, Zhou Xiao, Fandong Meng, and Jie Zhou. 2023. Personality understanding of fictional characters during book reading. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 14784-14802, Toronto, Canada. Association for Computational Linguistics.
* Zhang et al. (2024) Wenyuan Zhang, Jiawei Sheng, Shuaiyi Nie, Zefeng Zhang, Xinghua Zhang, Yongquan He, and Tingwen Liu. 2024. Revealing the challenge of detecting character knowledge errors in llvm role-playing. _Preprint_, arXiv:2409.11726.
* Zheng et al. (2023)Linamin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with MT-bench and chatbot arena. In _Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track_.

 

Character Profiles

Figure 6 presents the information we collected on four distinct characters, each exhibiting a unique personality, along with summaries of the novels in which they appear. This figure highlights the diversity in character design, showcasing the varied attributes and traits that define each character's persona.

## Appendix B Character Persona Training (CPT)

Figure 7 visualizes the overall process of _Character Persona Training (CPT)_, which involves updating a character's persona at each epoch by extracting key traits from chapter summaries. This ensures that the character's persona evolves consistently with the progression of the story, maintaining coherence and depth.

### Change in the Number of Tokens for Each Trait

Figure 8 reveals the dynamic redistribution of tokens across Megumin's traits throughout the CPT process. This visualization not only captures the evolving focus on specific character attributes but also highlights how critical aspects of the character's persona are refined and developed over time. The shifting token allocation provides a tangible measure of how different traits gain prominence or recede during the training, offering deep insights into the model's capacity to mirror character growth and complexity as the narrative unfolds.

## Appendix C Additional Case Study

In this section, we provide additional case studies to illustrate the effectiveness of CharacterGPT in maintaining persona consistency and capturing character evolution over time. Specifically, we examine how characters respond to queries at different points in a narrative and how their personalities and relationships evolve through Character Persona Training (CPT).

As shown in Figure 5, the responses of Megumin at different points in the novel reveal varying perspectives and emotions in response to the same query. This demonstrates how CharacterGPT is able to model the progression of a character's persona over time, providing more contextually accurate and natural responses.

Figure 9 shows the changes in Hitori's relationships and Frieren's personality as a result of CPT. Hitori, who initially struggles with social interactions, gradually forms meaningful relationships, while Frieren, who starts out indifferent to human emotions, becomes more empathetic. These examples underscore the ability of CharacterGPT to dynamically capture both internal and external attributes of characters as they evolve throughout astory.

## Appendix D Prompt Design

Figure 10 presents the input prompts used for both the generalization function \(h\) and the inference stage. To enhance user immersion, the inference prompt instructs the model to prioritize the character's _Voice and Speech Pattern_. Additionally, the model is directed to first assess whether the user's utterance is a request for information or part of a regular conversation, thereby optimizing the efficiency of the search process.

Figure 6: Character profiles and novel summaries of four popular fictional characters. (a) Megumin: Protagonist of _KONOSUBA: God’s Blessing on This Wonderful World!_, known for her eccentric and explosive personality. (b) Anya Forger: A central figure in _SPY \(\times\) FAMILY_, characterized by her mischievous and telepathic abilities. (c) Frieren: The titular character of _Frieren: Beyond Journey’s End_, a reserved elf image grappling with the meaning of life after a long journey. (d) Hitori Gotoh: The main character of _Bocchi the Rock_†, portrayed as an introverted and socially anxious guitarist.

 



## Appendix E Human Evaluation: Details of Story Generation

For the human evaluation in this study, participants were recruited through an online community consisting of undergraduate and graduate students. A total of 7 crowd-workers were selected, five of whom were male and two female, all aged in their 20s or 30s. The detailed instructions provided to all participants are shown in Figure 11. Participants were informed that the experiment results would be used to assess performance, and all compensation was provided in accordance with standard practices. It is important to note that participants were not coerced at any stage of the experiment, and all choices were made voluntarily.

## Appendix F Related Work

**Role-Playing.** Role-playing enables immersive and memorable interactions, and large language models (LLMs) have recently shown significant potential as role-playing agents Li et al. (2023); Wang et al. (2024); Wei et al. (2023); Jiang et al. (2023); Shanahan et al. (2023); Li et al. (2023); Salemi et al. (2024); Maas et al. (2023); Chen et al. (2023); Park et al. (2023). Despite its growing importance in Human-AI interaction, current methods primarily focus on personalization Abbasian et al. (2023); Dong et al. (2023); Tanwar et al. (2024); Abu-Rasheed et al. (2024); Salemi et al. (2024), evaluation Wang et al. (2024); Jiang et al. (2023), and interaction Wang et al. (2023); Maas et al. (2023); Li et al. (2023), leaving a fundamental research question unanswered: _"How can we effectively construct a persona-based assistant that mirrors the brain's memory storage process?"_. Although pre

Figure 8: Change in the number of tokens for each trait during CPT (Megumin).

Figure 7: Visualization of the Character Persona Training (CPT) process, showing how character traits are updated and refined with each chapter.



vious work (Park et al., 2023) utilizes a memory stream consisting of an agent's observations, the approach often relies on general descriptions and lacks the depth needed for more specific personalities, such as motivations or detailed backstories of iconic characters like _Naruto_ or _Son Goku_.

An assistant burdened by an extensive character persona faces two key challenges: (i) difficulty in retrieving role-specific knowledge, such as a protagonist's backstory, personality, and relationships, leading to unstable persona consistency, and (ii) excessive computational costs due to the need to search across fragmented persona documents. To address these challenges, we introduce a novel persona-rebuilding framework that consolidates extracted trait information into a cohesive narrative, structured chronologically within the persona document. Moreover, CharacterGPT, to the best of our knowledge, is the first approach to store each trained protagonist's persona at every training epoch. This feature is particularly beneficial in dynamic domains such as non-player characters (NPCs) in games (Uludagli and Oguz, 2023; Gallotta et al., 2024; Park et al., 2023), where the NPC's personality must adapt to the evolving storyline, enabling natural interaction with users.

**Psychology in NLP.** In the interdisciplinary space between psychology and computational linguistics, the application of personality theories, such as the Big Five Inventory (BFI) (Barrick and Mount, 1991), 16Personalities (16P)3, and the Myers-Briggs Type Indicator (MBTI)4, has significantly advanced our understanding of human traits and their relevance in natural language processing. These foundational frameworks have led to the development of psychometric tools (Li et al., 2018) that assess individual differences across a wide range of contexts. Simultaneously, the NLP community has applied these psychological insights to diverse areas, such as automatic personality prediction from text (Feizi-Derakhshi et al., 2022; Jayaraman et al., 2023) and personalized dialogue systems (Mo et al., 2018; Ma et al., 2020). The convergence of psychology and NLP has been further strengthened by the advent of LLMs, which enhance the potential for personality assessment and personalized interaction through advanced benchmarking and prompting methodologies (Wang et al., 2023b; Park et al., 2023; Onorati et al., 2023).

Footnote 3: [https://www.16personalities.com/](https://www.16personalities.com/)

Footnote 4: [https://www.myersbriggs.org/](https://www.myersbriggs.org/) 

Figure 9: Case study of the evolution of Hitori’s relationships (left) and Frieren’s personality (right) through Character Persona Training (CPT). These results demonstrate how the method captures shifts in both external relationships and internal character development.

 



**Figure 10: Actual example of our prompts: (Top) Generalization function, (Bottom) Inference.



2.3 The \(k_{1}\)-thrust