Reasoning Does Not Necessarily Improve Role-Playing Ability
Authors: Xiachong Feng, Longxu Dou, Lingpeng Kong
Accepted at: acl25

Reasoning Does Not Necessarily Improve Role-Playing Ability

Xiachong Feng♡♣, Longxu Dou♣*, Lingpeng Kong♡*
♡The University of Hong Kong, ♣Sea AI Lab
fengxc@hku.hk, doulx@sea.com, lpk@cs.hku.hk

Abstract
The application of role-playing large language models (LLMs) is rapidly expanding in both academic and commercial domains, driving an increasing demand for high-precision role-playing models. Simultaneously, the rapid advancement of reasoning techniques has continuously pushed the performance boundaries of LLMs. This intersection of practical role-playing demands and evolving reasoning capabilities raises an important research question: “Can reasoning techniques enhance the role-playing capabilities of LLMs?” To address this, we conduct a comprehensive study using 6 role-playing benchmarks, 24 LLMs, and 3 distinct role-playing strategies, comparing the effectiveness of direct zero-shot role-playing, role-playing with Chain-of-Thought (CoT), and role-playing using reasoning-optimized LLMs. Our findings reveal that CoT may reduce role-playing performance, reasoning-optimized LLMs are unsuitable for role-playing, reasoning ability disrupts the role-playing scaling law, large models still lack proficiency in advanced role-playing, and Chinese role-playing performance surpasses English role-playing performance. Furthermore, based on extensive experimental results, we propose two promising future research directions: Role-aware CoT for improving role-playing LLMs and Reinforcement Learning for role-playing LLMs, aiming to enhance the adaptability, consistency, and effectiveness of role-playing LLMs for both research and real-world applications1.

1 Introduction
Large language models (LLMs), leveraging their strong foundational capabilities and advanced instruction-following abilities, have become the cornerstone of role-playing AI models (Chen et al., 2024b; Tseng et al., 2024). These role-playing LLMs present new opportunities in both research and commercial applications. In the academic domain, they offer novel possibilities for social simulation experiments (Mou et al., 2024). In industry, role-specific assistants, emotional companionship AI, and intelligent game NPCs (Park et al., 2023; Xu et al., 2024) hold significant potential for further development and commercialization. At the same time, reasoning capabilities in LLMs have been further enhanced through Chain-of-Thought (CoT) reasoning (Wei et al., 2022) and reinforcement learning (Guo et al., 2025), enabling them to tackle increasingly complex problems.

This situation prompts us to explore a previously unanswered research question: “Can reasoning techniques enhance the role-playing capabilities of large language models?” The answer to this question could provide valuable insights for the future application of reasoning techniques in role-playing LLMs, potentially paving a new path for their development and advancement.

To address this question, we carefully select six role-playing datasets and conduct experiments us-

<table>
  <tr>
    <th>Context</th>
    <th></th>
  </tr>
  <tr>
    <td colspan="2">Tom: Captain, your boat has been stolen.</td>
  </tr>
  <tr>
    <th colspan="2">Zero-shot role-playing with GPT-4-Turbo</th>
  </tr>
  <tr>
    <td colspan="2">Not stolen, mate—temporarily misplaced. And I always get my ship back, savvy?</td>
  </tr>
  <tr>
    <th colspan="2">Role-playing with Chain-of-thought using TPT-4-Turbo</th>
  </tr>
  <tr>
    <td colspan="2">[Reasoning process omitted] So the Pearl’s decided to go on a little adventure without me—no matter, I’ll just steal her back.</td>
  </tr>
  <tr>
    <th colspan="2">Role-playing with OpenAI 01-mini</th>
  </tr>
  <tr>
    <td colspan="2">[Reasoning process omitted] Stolen, you say, mate—why, I’d be far more worried for the soul foolish enough to filch the Pearl, savvy?</td>
  </tr>
</table>

Table 1: Jack Sparrow role-playing example. (Jack Sparrow is a cunning and eccentric pirate captain known for his flamboyant, witty, and often rambling speech that conceals sharp intelligence beneath a layer of drunken charm.) It can be observed that direct zero-shot prompting of GPT yields better results, aligning more closely with Jack Sparrow’s character traits.
ing 24 widely used open-source and proprietary models. These models include API-based closed-source models such as GPT-4-Turbo (Achiam et al., 2023), popular open-source models like Qwen2.5 (Yang et al., 2024), and reinforcement learning-optimized reasoning models such as DeepSeek-R1 (Guo et al., 2025). All experiments are conducted using OpenCompass (Contributors, 2023) to ensure evaluation consistency, stability, and reproducibility. Specifically, we utilize six role-playing benchmarks: RoleBench (Wang et al., 2023b), InCharacter (Wang et al., 2024), SocialBench (Chen et al., 2024a), CharacterEval (Tu et al., 2024), HPD (Chen et al., 2023), and CroSS-MR (Yuan et al., 2024). These benchmarks cover both multiple-choice and text generation tasks and include English and Chinese, making the evaluation linguistically diverse. For evaluation, we employ both traditional metrics (e.g., Accuracy, ROUGE, and Exact Match) and LLM-as-a-Judge methods (e.g., prompt-based evaluation and reward model scoring), ensuring a broad assessment scope to enhance the robustness of our conclusions. Furthermore, we define three role-playing approaches: direct zero-shot role-playing, role-playing with Chain-of-Thought (CoT), and role-playing using reasoning-optimized LLMs. This comprehensive setup allows us to systematically investigate the impact of reasoning techniques on role-playing performance. Table 1 presents a specific example, demonstrating that the first setting, direct zero-shot role-playing, achieves the best performance.

Through extensive experiments, our findings reveal several key insights that can inform the future design of role-playing LLMs: (1) CoT may reduce role-playing performance; (2) Reasoning-optimized LLMs are unsuitable for role-playing; (3) Reasoning ability disrupts the role-playing scaling law; (4) The Qwen series is well-suited for role-playing; (5) Chinese role-playing performance surpasses English role-playing performance; (6) Large models still lack proficiency in advanced role-playing.

Our study makes three key contributions. First, through extensive experiments on six role-playing datasets and 24 models, we reveal that reasoning techniques do not necessarily enhance the role-playing capabilities of LLMs. Second, our results provide a comprehensive analysis of the current state and limitations of role-playing LLMs, leading to two promising research directions: Role-Aware CoT for Improving Role-Playing LLMs and Reinforcement Learning for Role-Playing LLMs. Third, to address the fragmented nature of prior work with inconsistent evaluation standards, we integrate all datasets and experimental methods into OpenCompass, enabling one-click execution and providing a standardized experimental framework and codebase for future research2.

2 Evaluation Framework

The evaluation framework for LLM-based role-playing agents primarily consists of the following components: the LLM itself (\( \mathcal{M} \)), character profile (\( \mathcal{P} \)), evaluation task (\( \mathcal{T} \)), evaluation metric (\( \mathcal{E} \)), and, when applicable, a reference standard answer (\( \mathcal{A} \)). In this study, we focus on whether reasoning techniques (\( \mathcal{R} \)) can enhance the role-playing capabilities of LLMs. Therefore, the overall evaluation process can be formalized as follows:

Given an LLM \( \mathcal{M} \), a predefined character profile \( \mathcal{P} \), an evaluation task \( \mathcal{T} \), and an evaluation metric \( \mathcal{E} \), the role-playing performance of \( \mathcal{M} \) is assessed by generating responses conditioned on \( \mathcal{P} \) and \( \mathcal{T} \). The generated outputs are then evaluated using \( \mathcal{E} \), which quantifies alignment with the intended role. Additionally, reasoning techniques \( \mathcal{R} \) are incorporated into \( \mathcal{M} \) to examine their impact on enhancing role-playing abilities. Formally, the evaluation process can be expressed as: \( S = \mathcal{E}(\mathcal{M}(\mathcal{P}, \mathcal{T}, \mathcal{R}), \mathcal{A}) \) where \( S \) represents the final performance score, capturing the effectiveness of \( \mathcal{M} \) in role-playing under the given conditions. A higher \( S \) indicates stronger role-playing capabilities. If \( \mathcal{A} \) is unavailable or unnecessary, the evaluation metric \( \mathcal{E} \) can be adapted to rely on alternative assessment criteria, such as LLM-as-a-Judge.

3 Experimental Setting

In this section, we introduce the benchmarks, models, metrics, and reasoning methods used in our experiments. All relevant code has been integrated into the OpenCompass (Contributors, 2023) library to facilitate efficient replication by researchers.

3.1 Role-playing Benchmarks

We select six role-playing benchmarks to ensure the reliability of our experimental results, including: RoleBench (Wang et al., 2023b): Generates responses to both general and role-specific questions based on role information. HPD (Chen et al.,

2All code will be open-sourced upon publication.
<table>
  <tr>
    <th>Dataset</th>
    <th>Task Type</th>
    <th>Evaluation Metric</th>
    <th>Task</th>
    <th>Language</th>
    <th>#Characters</th>
    <th>#Samples</th>
  </tr>
  <tr>
    <td rowspan="2">RoleBench (Wang et al., 2023b)</td>
    <td rowspan="2">Generation</td>
    <td>Automatic (ROUGE)</td>
    <td>General Question Answering</td>
    <td>EN</td>
    <td>95</td>
    <td>32833</td>
  </tr>
  <tr>
    <td></td>
    <td>Role-specific Question Answering</td>
    <td>ZH</td>
    <td>5</td>
    <td>1690</td>
  </tr>
  <tr>
    <td rowspan="2">HPD (Chen et al., 2023)</td>
    <td rowspan="2">Generation</td>
    <td>Automatic (ROUGE)</td>
    <td>Response Generation</td>
    <td>EN</td>
    <td>1</td>
    <td>149</td>
  </tr>
  <tr>
    <td></td>
    <td>Response Generation</td>
    <td>ZH</td>
    <td>1</td>
    <td>167</td>
  </tr>
  <tr>
    <td>CharacterEval (Tu et al., 2024)</td>
    <td>Generation</td>
    <td>LLM-as-a-Judge (Pretrained Reward Model)</td>
    <td>Response Generation</td>
    <td>ZH</td>
    <td>77</td>
    <td>4564</td>
  </tr>
  <tr>
    <td rowspan="4">CroSS-MR (Yuan et al., 2024)</td>
    <td rowspan="2">Multiple-choice</td>
    <td>Automatic (Accuracy)</td>
    <td>Motivation Recognition</td>
    <td>EN</td>
    <td>126</td>
    <td>445</td>
  </tr>
  <tr>
    <td></td>
    <td>Role Knowledge Understanding</td>
    <td>EN</td>
    <td>23</td>
    <td>988</td>
  </tr>
  <tr>
    <td rowspan="2">Multiple-choice</td>
    <td>Automatic (Accuracy)</td>
    <td>Behavioral Style Understanding</td>
    <td>EN</td>
    <td>8</td>
    <td>740</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>ZH</td>
    <td>8</td>
    <td>323</td>
  </tr>
  <tr>
    <td rowspan="2">Socialbench (Chen et al., 2024a)</td>
    <td rowspan="2">Generation</td>
    <td></td>
    <td>Social Preference Recognition</td>
    <td>EN</td>
    <td>79</td>
    <td>731</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>ZH</td>
    <td>101</td>
    <td>1185</td>
  </tr>
  <tr>
    <td rowspan="2"></td>
    <td rowspan="2">Generation</td>
    <td>Automatic (Exact Match)</td>
    <td>Long Conversation Memorization</td>
    <td>EN</td>
    <td>47</td>
    <td>728</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>ZH</td>
    <td>35</td>
    <td>620</td>
  </tr>
  <tr>
    <td rowspan="2"></td>
    <td rowspan="2">Generation</td>
    <td></td>
    <td>Short Conversation Memorization</td>
    <td>EN</td>
    <td>46</td>
    <td>463</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>ZH</td>
    <td>27</td>
    <td>310</td>
  </tr>
  <tr>
    <td>InCharacter (Wang et al., 2024)</td>
    <td>Generation</td>
    <td>LLM-as-a-Judge (Prompting)</td>
    <td>16Personalities Identification</td>
    <td>EN</td>
    <td>32</td>
    <td>32</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td></td>
    <td>BFI Identification</td>
    <td>EN</td>
    <td>32</td>
    <td>32</td>
  </tr>
</table>

Table 2: Benchmark statistics.

2023): Produces responses by leveraging the provided Harry Potter character information and related character details. CharacterEval (Tu et al., 2024): Generates responses based on detailed character information, including background experiences and personality traits. CroSS-MR (Yuan et al., 2024): Recognizes a role’s behavioral motivations based on the provided character profile. SocialBench (Chen et al., 2024a): Understands a role’s knowledge, behavioral style, social preferences, and fine-grained memory based on the given role information. InCharacter (Wang et al., 2024): Assesses a character’s personality through questionnaire-based interviews3. Detailed benchmark statistics are provided in Table 2.

3.2 Backbone LLMs

Our experiments include a total of 24 models, comprising 2 closed-source models and 22 open-source models. These models originate from six different companies, including state-of-the-art models such as GPT-4-Turbo and DeepSeek-R1. The detailed list of models is provided in Appendix A.

3.3 Evaluation Metrics

Our evaluation incorporates two types of metrics: automated metrics and LLM-as-a-Judge, both of which are widely recognized and scalable evaluation approaches. Automated metrics include ROUGE, Accuracy, and Exact Match, which are well-established and widely accepted in the field as classical evaluation benchmarks. LLM-as-a-Judge further encompasses two methodologies: the Pretrained Reward Model and Prompting-based Scoring. The Pretrained Reward Model is trained on annotated datasets and assigns scores to specified dimensions during evaluation. In contrast, Prompting-based Scoring leverages prompt engineering techniques and the extensive knowledge base of LLMs to dynamically and adaptively generate relevant scores, providing a more flexible and responsive evaluation mechanism. In Appendix B, we provide a detailed explanation of the evaluation metrics used for each benchmark.

3.4 Role-playing Methods

In this study, we employ three approaches to guide LLMs in performing role-playing tasks: \( \mathcal{R}_1 \): Direct zero-shot role-playing using an LLM, where the model generates responses without reasoning steps. \( \mathcal{R}_2 \): Role-playing with chain-of-thought (CoT) reasoning, where an LLM explicitly engages in step-by-step reasoning before executing the role-playing task. \( \mathcal{R}_3 \): Role-playing using reasoning-optimized LLMs, such as QwQ-32B-Preview and DeepSeek-R1, which autonomously engage in deep reasoning before generating responses.

3 InCharacter benchmark comprises 14 types of assessment questionnaires. In this study, we select the most classic ones: 16Personalities and BFI for reporting results. Meanwhile, all other questionnaires are also implemented in the code.
Figure 1: Performance comparison of 17 Models using two role-playing methods across six benchmarks. The horizontal axis ranks models in descending order of scale, while the vertical axis represents the unique metric for each dataset. Notably, darker colors indicate that zero-shot role-playing outperforms CoT role-playing, whereas lighter colors signify that zero-shot role-playing underperforms compared to CoT role-playing.

4 Results and Findings

In this section, we present our experimental analysis and key findings.

4.1 CoT May Reduce Role-Playing Performance

To investigate whether reasoning techniques enhance the role-playing capabilities of LLMs, we conduct extensive experiments. Specifically, we select 17 models and evaluate their role-playing performance using both zero-shot and chain-of-thought (CoT) approaches across six standardized and widely used benchmarks. All experimental results are presented in the Appendix C. Figure 1 provides an aggregated overview of the results. Specifically, for each model, its final performance on a given benchmark is computed as the average of its performance across all sub-datasets.

As shown in Figure 1, employing CoT reasoning is more likely to degrade role-playing performance on four benchmarks: CroSS-MR, HPD, SocialBench, and CharacterEval. In contrast, on InCharacter and RoleBench, CoT reasoning enhances the role-playing capabilities of LLMs. To further understand why CoT reduces the role-playing capabilities of LLMs, we select Qwen2.5-7B-Instruct as the experimental model. We then sample 50 test cases from each of the six benchmarks where CoT performance is lower than Zero-shot performance for detailed analysis.

Our findings indicate that the primary reasons for CoT-induced degradation in role-playing are: (1) “Attention Diversion”: The model must simultaneously engage in reasoning and role-playing modes, which dilutes its focus on the role-playing task. (2) “Linguistic Style Drift”: Reasoning responses tend to be structured, logical, and formal, whereas effective role-playing requires a vivid, expressive, and character-consistent linguistic style.

Finding 1:
CoT may reduce the role-playing capabilities of LLMs, primarily due to attention diversion and linguistic style drift induced by the reasoning process.

4.2 Reasoning-optimized LLMs Are Unsuitable for Role-Playing

The most advanced models available today are undoubtedly reasoning-optimized models, including OpenAI’s o series, DeepSeek’s R1 model, and various distilled versions derived from DeepSeek-R1\footnote{https://github.com/deepseek-ai/DeepSeek-R1}. Compared to CoT reasoning, these models leverage pretraining and reinforcement learning techniques to cultivate intrinsic reasoning capabilities, making them inherently more adept at reflection, verification, and other cognitive processes.

To investigate whether reasoning-optimized
Figure 2: Experimental results of various models across 6 benchmarks. Models of similar sizes are represented using the same color scheme, with each employing different types of reasoning techniques. The vertical axis denotes the evaluation metrics specific to each dataset.

models are better suited for role-playing tasks, we conduct experiments using OpenAI o1-mini, QwQ-32B-Preview, DeepSeek-R1, and its various distilled versions. The experimental results are presented in Figure 2.

It is evident that reasoning-optimized LLMs generally perform poorly and are not well-suited for role-playing tasks, even for state-of-the-art models such as OpenAI o1-Mini and DeepSeek-R1. Furthermore, we observe that models refined through reasoning distillation exhibit even worse role-playing performance compared to their original counterparts. This finding aligns with the conclusion drawn in the previous subsection: enhancing rational reasoning capabilities tends to undermine the emotional and intuitive aspects essential for effective role-playing. We provide more discussion in Appendix E.

Finding 2:
Reasoning-optimized LLMs are less suitable for role-playing tasks.

4.3 Reasoning Ability Disrupts the Role-Playing Scaling Law

Figure 3 presents the results of our three experimental settings: direct zero-shot role-playing, role-playing with Chain-of-Thought (CoT), and role-playing using reasoning-optimized LLMs. First, we observe that, with the exception of the InCharacter benchmark, the other five benchmarks generally follow the scaling law, where larger models exhibit stronger role-playing capabilities. However, we also find that the role-playing scaling law is not particularly pronounced—the performance gains from increasing model size remain relatively modest and inconsistent across benchmarks. Furthermore, introducing reasoning capabilities, whether through CoT or reasoning-optimized LLMs, further weakens the benefits of scaling, leading to more pronounced fluctuations, increased instability, and greater variability in model performance across different role-playing tasks and datasets, making the scaling trend less predictable and consistent.

Finding 3:
The role-playing scaling law exists but is not pronounced, and the introduction of reasoning capabilities disrupts this scaling law.

4.4 The Qwen Series Is Well-Suited for Role-Playing Tasks

To provide guidance on model selection for future role-playing tasks, we conduct a comprehensive and systematic comparative analysis across different model scales (e.g., 1B, 3B, 7B, 14B, 32B, 72B). The results are shown in Figure 3. Our findings indicate that the Qwen2.5 series consistently out-
(a) Results of direct zero-shot role-playing.

(b) Results of role-playing with chain-of-thought (CoT) reasoning.

(c) Results of role-playing using reasoning-optimized LLMs.

Figure 3: Performance comparison of different models across six benchmarks. The horizontal axis represents model size, arranged from smallest to largest, while the vertical axis denotes benchmark-specific evaluation metrics, where higher values indicate better role-playing performance. Within each benchmark, different color gradients represent the performance curves for its respective sub-datasets.

performs the Llama, Gemma, and Mistral series across various size ranges in terms of role-playing accuracy, persona consistency, linguistic expressiveness, and contextual coherence. Notably, we recommend Qwen2.5-7B-Instruct as the most cost-effective and well-balanced model for role-playing applications, as it offers a strong trade-off between performance, computational efficiency, and adaptability across diverse role-playing scenarios.

Finding 4:
The Qwen2.5 series excels in role-playing tasks, with Qwen2.5-7B-Instruct being the most cost-effective model.
Figure 4: Fine-grained performance of the Qwen2.5 series on the CharacterEval benchmark. The radar chart illustrates multiple evaluation dimensions, with metrics computed using a pretrained reward model. Higher scores indicate stronger capabilities.

4.5 Chinese Role-Playing Performance Surpasses English Role-Playing Performance

The role-playing benchmarks HPD, SocialBench, InCharacter, and RoleBench provide both Chinese and English versions, enabling a direct comparison of multilingual role-playing performance. Interestingly, an analysis of Figure 3 reveals that current LLMs exhibit stronger role-playing capabilities in Chinese than in English across multiple benchmarks and evaluation metrics. This observation contradicts the common conclusion that LLMs generally have a stronger foundation in English compared to other languages, particularly in reasoning and knowledge-intensive tasks. We hypothesize that this phenomenon may be attributed to the following factor: Due to the extensive training on English data, models have to some extent internalized generalized character information. When performing precise role-playing tasks, this internalized generalization can interfere with context-sensitive role-playing, leading to performance degradation.

Finding 5:
Large models exhibit superior role-playing capabilities in Chinese compared to English to some extent.

4.6 Large Models Still Lack Proficiency in Advanced Role-Playing

To further investigate the limitations of current LLM-based role-playing models, we conduct an analysis using the CharacterEval benchmark. Specifically, CharacterEval provides a pretrained reward model that assigns scores across 12 role-playing evaluation dimensions5. We analyze the performance of the Qwen2.5 series, with the results presented in Figure 4. All detailed results are presented in Appendix D. First, consistent with our previous findings, direct zero-shot role-playing outperforms other approaches in most cases. Furthermore, a fine-grained comparison of evaluation metrics reveals that current LLMs still exhibit deficiencies in advanced role-playing dimensions, such as character knowledge exposition, personality expression, and linguistic diversity. These aspects require further enhancement in future research to improve the overall role-playing capability of LLMs.

Finding 6:
Current LLMs still underperform in advanced role-playing capabilities, such as knowledge exposition, personality expression, and diversity.

5https://huggingface.co/morecry/BaichuanCharRM
5 Related Works

In recent years, large language models (LLMs) have revolutionized the paradigm of artificial intelligence, achieving human-like performance in mathematics (Liu et al., 2023; Shao et al., 2024), coding (Luo et al., 2023; Roziere et al., 2023; Guo et al., 2024; Zhu et al., 2024), embodied intelligence (Wang et al., 2023a; Liu et al., 2024), game intelligence (Hu et al., 2024; Feng et al., 2024), and advanced reasoning (Chu et al., 2023; Xu et al., 2025). The comprehensive enhancement of foundational capabilities in these models has opened new opportunities for the development of role-playing agents (Chen et al., 2024b), resulting in a proliferation of such applications based on LLMs. On one hand, this has catalyzed innovations in fields such as psychological counseling, anthropomorphic companionship, and game NPCs (Tseng et al., 2024). On the other hand, it has provided fresh avenues for advancing social simulations (Mou et al., 2024). Concurrently, reasoning techniques in LLMs have flourished, with methods like chain-of-thought reasoning (Wei et al., 2022), tree-of-thought reasoning (Yao et al., 2024), and o1-style reasoning (Jaech et al., 2024) progressively maximizing the potential of these models, particularly achieving remarkable outcomes in tasks involving mathematics and coding (Guo et al., 2025; Team et al., 2025). While prior research has predominantly focused on role-playing applications and their evaluation, this study is positioned at the intersection of LLM-based role-playing agents and reasoning techniques. It seeks to address the scientific question of whether reasoning techniques enhance the role-playing capabilities of LLMs. By doing so, this work aims to provide guidance for developing more realistic and reliable role-playing agents powered by LLMs.

6 Future Directions

Based on our experimental findings, we propose two potential research directions that integrate role-playing and reasoning techniques for future exploration.

Role-aware Chain-of-Thought (CoT) for Improving Role-playing LLMs. One promising direction for enhancing the role-playing ability of LLMs is Role-aware CoT reasoning. While standard CoT enables step-by-step logical inference, it often disregards the persona-specific constraints that are essential for consistent role-playing. A role-aware CoT approach would integrate persona attributes, narrative constraints, and character-specific perspectives into the reasoning process, ensuring that logical deductions align with the character’s predefined traits. For instance, a historical figure simulated in an LLM should reason within the knowledge and biases of their era rather than applying contemporary logic. This requires incorporating dynamic memory structures that retain persona information throughout multi-turn interactions, mitigating the risk of breaking character or adopting inconsistent reasoning patterns.

Reinforcement Learning for Role-playing LLMs. DeepSeek-R1 has demonstrated that by defining precise rule-based rewards, reinforcement learning alone can induce emergent reasoning and cognitive capabilities. This suggests an important research direction: investigating whether carefully designed role-playing task rewards can enable models to autonomously develop intrinsic, role-specific reasoning and thinking abilities, thereby enhancing their role-playing performance. A critical consideration in reward design is ensuring that it simultaneously accounts for the accuracy of the role-playing task while effectively guiding the model to develop role-specific reasoning abilities. The reward function should prevent the model from relying on shortcuts to generate final responses, instead encouraging it to engage in authentic character-driven reasoning and decision-making.

7 Conclusion

This study aims to address the research question: “Can reasoning techniques enhance the role-playing capabilities of large language models (LLMs)?” To this end, we conduct extensive experiments using 6 role-playing benchmarks, 24 LLMs, and 3 distinct role-playing methods. Our experimental results lead to the following key findings: CoT may reduce role-playing performance, reasoning-optimized LLMs are unsuitable for role-playing, reasoning ability disrupts the role-playing scaling law, the Qwen series is well-suited for role-playing tasks, Chinese role-playing performance surpasses English role-playing performance, and large models still lack proficiency in advanced role-playing. All code is integrated into OpenCompass, ensuring reproducibility and facilitating further research. We hope that our findings provide new perspectives for future studies on role-playing LLMs.
Limitations

Although we have made every effort to comprehensively cover the most commonly used models and role-playing benchmarks, our selection may still be incomplete. Certain niche models or specialized benchmarks may not have been included, which could introduce potential biases in our findings. As a result, while our conclusions provide valuable insights into the role-playing capabilities of large language models, their robustness could be further enhanced through broader model coverage, additional benchmark evaluations, and more extensive experimental validations. Future studies should explore a wider range of datasets and model architectures to ensure greater generalizability and reliability of the findings.

Acknowledgments

This work was supported by Hong Kong Innovation and Technology Support Programme Platform Research Project fund (ITS/269/22FP).

References

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shaymal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.

Hongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu, Gao Xing, Weizhou Shen, Xiaojun Quan, Chenliang Li, Ji Zhang, and Fei Huang. 2024a. Socialbench: Sociality evaluation of role-playing conversational agents. In Findings of the Association for Computational Linguistics ACL 2024, pages 2108–2126.

Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, et al. 2024b. From persona to personalization: A survey on role-playing language agents. arXiv preprint arXiv:2404.18231.

Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, and Jia Li. 2023. Large language models meet harry potter: A dataset for aligning dialogue agents with characters. In Findings of the Association for Computational Linguistics EMNLP 2023, pages 8506–8520.

Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu, Bing Qin, and Ting Liu. 2023. A survey of chain of thought reasoning: Advances, frontiers and future. arXiv preprint arXiv:2309.15402.

OpenCompass Contributors. 2023. Opencompass: A universal evaluation platform for foundation models. https://github.com/open-compass/opencompass.

Xiachong Feng, Longxu Dou, Ella Li, Qinghao Wang, Haochuan Wang, Yu Guo, Chang Ma, and Lingpeng Kong. 2024. A survey on large language model-based social agents in game-theoretic scenarios. arXiv preprint arXiv:2412.03920.

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.

Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. Deepseek-coder: When the large language model meets programming—the rise of code intelligence. arXiv preprint arXiv:2401.14196.

Sihao Hu, Tiansheng Huang, Fatih Ilhan, Selim Tekin, Gaowen Liu, Ramana Kompella, and Ling Liu. 2024. A survey on large language model-based game agents. arXiv preprint arXiv:2404.02039.

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720.

Wentao Liu, Hanglei Hu, Jie Zhou, Yuyang Ding, Junsong Li, Jiayi Zeng, Mengliang He, Qin Chen, Bo Jiang, Aimin Zhou, et al. 2023. Mathematical language models: A survey. arXiv preprint arXiv:2312.07622.

Yang Liu, Weixing Chen, Yongjie Bai, Xiaodan Liang, Guanbin Li, Wen Gao, and Liang Lin. 2024. Aligning cyber space with physical world: A comprehensive survey on embodied ai. arXiv preprint arXiv:2407.06886.

Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizarddecoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568.

Xinyi Mou, Xuanwen Ding, Qi He, Liang Wang, Jingcong Liang, Xinnong Zhang, Libo Sun, Jiayu Lin, Jie Zhou, Xuanjing Huang, et al. 2024. From individual to society: A survey on social simulation driven by large language model-based agents. arXiv preprint arXiv:2412.03563.

Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22.
Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950.

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.

Kim Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. 2025. Kimi kl. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599.

Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Yu-Ching Hsu, Jia-Yin Foo, Chao-Wei Huang, and Yun-Nung Chen. 2024. Two tales of persona in llms: A survey of role-playing and personalization. arXiv preprint arXiv:2406.01171.

Quan Tu, Shilong Fan, Zihang Tian, and Rui Yan. 2024. Charactereval: A chinese benchmark for role-playing conversational agent evaluation. arXiv preprint arXiv:2401.01275.

Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023a. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291.

Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, et al. 2024. Incharacter: Evaluating personality fidelity in role-playing agents through psychological interviews. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1840–1873.

Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, et al. 2023b. Rolellm: Benchmarking, eliciting, and enhancing role-playing abilities of large language models. arXiv preprint arXiv:2310.00746.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.

Fengl Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. 2025. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.09686.

Zhenyu Xu, Hailin Xu, Zhouyang Lu, Yingying Zhao, Rui Zhu, Yujiang Wang, Mingzhi Dong, Yuhu Chang, Qin Lv, Robert P Dick, et al. 2024. Can large language models be good companions? an llm-based eyewear system with conversational common ground. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8(2):1–41.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115.

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36.

Xinfeng Yuan, Siyu Yuan, Yuhuan Cui, Tianhe Lin, Xintao Wang, Rui Xu, Jiangjie Chen, and Deqing Yang. 2024. Evaluating character understanding of large language models via character profiling from fictional works. arXiv preprint arXiv:2404.12726.

Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al. 2024. Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence. arXiv preprint arXiv:2406.11931.

A Evaluated Models

Table 3 lists all the large language models (LLMs) used in this study.

B Detailed Metrics

• Accuracy: Used for multiple-choice evaluation in Cross-MR and the SocialBench multiple-choice task.

• ROUGE: Used for text generation evaluation in HPD and RoleBench.

• Exact Match: Used in the SocialBench Generation Task to assess the alignment of key memory points.

• LLM-as-a-Judge (Prompting): Used in In-Character to convert model responses into standardized answers through analytical transformation.

• LLM-as-a-Judge (Reward Model): Used in CharacterEval, where a reward model assigns scores across 12 evaluation dimensions for model responses.

C Detailed Results on Role-palying Benchmarks

Tables 4, 5, 6, 7, and 8 present the experimental results of various models using three role-playing methods across six benchmarks.
<table>
  <tr>
    <th>Team</th>
    <th>Model</th>
  </tr>
  <tr>
    <th colspan="2">Closed-source</th>
  </tr>
  <tr>
    <td rowspan="2">OpenAI</td>
    <td>GPT-4 Turbo</td>
  </tr>
  <tr>
    <td>OpenAI o1-mini</td>
  </tr>
  <tr>
    <th colspan="2">Open-source</th>
  </tr>
  <tr>
    <td rowspan="7">Qwen</td>
    <td>Qwen2.5-0.5B-Instruct</td>
  </tr>
  <tr>
    <td>Qwen2.5-1.5B-Instruct</td>
  </tr>
  <tr>
    <td>Qwen2.5-3B-Instruct</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B-Instruct</td>
  </tr>
  <tr>
    <td>Qwen2.5-14B-Instruct</td>
  </tr>
  <tr>
    <td>Qwen2.5-32B-Instruct</td>
  </tr>
  <tr>
    <td>Qwen2.5-72B-Instruct</td>
  </tr>
  <tr>
    <td>QwQ-32B-Preview</td>
  </tr>
  <tr>
    <td rowspan="2">Google</td>
    <td>gemma-2-2b-it</td>
  </tr>
  <tr>
    <td>gemma-2-9b-it</td>
  </tr>
  <tr>
    <td rowspan="2">Meta</td>
    <td>gemma-2-27b-it</td>
  </tr>
  <tr>
    <td>Llama-3.2-1B-Instruct</td>
  </tr>
  <tr>
    <td rowspan="2">Mistral AI</td>
    <td>Llama-3.2-3B-Instruct</td>
  </tr>
  <tr>
    <td>Llama-3.1-8B-Instruct</td>
  </tr>
  <tr>
    <td rowspan="2">Deepseek</td>
    <td>Llama-3.3-70B-Instruct</td>
  </tr>
  <tr>
    <td>Minstral-8B-Instruct-2410</td>
  </tr>
  <tr>
    <td rowspan="2">DeepSeek-R1</td>
    <td>Mistral-7B-Instruct-v0.3</td>
  </tr>
  <tr>
    <td>DeepSeek-R1-Distill-Llama-8B</td>
  </tr>
  <tr>
    <td rowspan="3">Deepseek</td>
    <td>DeepSeek-R1-Distill-Qwen-14B</td>
  </tr>
  <tr>
    <td>DeepSeek-R1-Distill-Qwen-32B</td>
  </tr>
  <tr>
    <td>DeepSeek-R1-Distill-Llama-70B</td>
  </tr>
</table>

Table 3: List of models evaluated in the experiment.

D Fine-Grained Results on CharacterEval

Table 9, 10, 11, 12, and 13 present the fine-grained performance of different models on CharacterEval using three role-playing methods. The evaluation consists of 12 dimensions, with all scores provided by a pretrained reward model.

E Limitations of Reinforcement Learning for Role-Playing Tasks

To address the potential trade-offs between reasoning-optimized models and role-playing tasks, we conducted a focused analysis using DeepSeek-R1 as a representative case, leveraging its publicly available technical details. DeepSeek-R1-Zero is trained exclusively through reinforcement learning, without supervised fine-tuning, employing a rule-based reward system designed to enhance reasoning accuracy and structural compliance. While this training paradigm achieves substantial gains in formal reasoning tasks, it introduces several critical limitations that hinder performance in persona-grounded interactions. First, the reasoning–role conflict arises because the model is explicitly optimized for factual correctness and logical precision, often at the expense of the creativity, empathy, and subjectivity necessary for maintaining in-character responses. Second, style drift emerges as the structured reasoning formats (e.g., encapsulated reasoning tags) promote rigid and formal outputs, undermining the expressive and stylistically diverse language required for immersive role-play. Third, reward misalignment is evident, as the reinforcement signals focus narrowly on deterministic correctness and formatting adherence, neglecting essential dimensions such as persona consistency, emotional tone, and conversational adaptability. Finally, the model exhibits over-optimization for test-time computation, favoring extended reasoning chains and reflection loops (e.g., “aha moments”) that, while useful for complex problem-solving, lead to verbose, unnatural, and character-inconsistent outputs during role-play. These observations highlight a fundamental misalignment between reasoning-focused optimization and the adaptive, expressive requirements of role-playing, offering important insights for future efforts to harmonize reasoning capabilities with character fidelity in large language models.
<table>
  <tr>
    <th>Dataset</th>
    <th>Sub Dataset</th>
    <th colspan="2">OpenAI GPT4-Turbo</th>
    <th colspan="2">Qwen2.5-0.5B -Instruct</th>
    <th colspan="2">Qwen2.5-1.5B -Instruct</th>
    <th colspan="2">Qwen2.5-3B-Instruct</th>
    <th colspan="2">Qwen2.5-7B -Instruct</th>
    <th colspan="2">Qwen2.5-14B -Instruct</th>
    <th colspan="2">Qwen2.5-32B -Instruct</th>
    <th colspan="2">Qwen2.5-72B -Instruct</th>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>Cross</td>
    <td></td>
    <td>cross</td>
    <td>35.93</td>
    <td>26.29</td>
    <td>45.39</td>
    <td>55.96</td>
    <td>56.40</td>
    <td>59.10</td>
    <td>61.80</td>
    <td>62.92</td>
    <td>HPD</td>
    <td>hpd_en</td>
    <td>17.21</td>
    <td>10.18</td>
    <td>11.71</td>
    <td>12.18</td>
    <td>13.28</td>
    <td>12.85</td>
    <td>14.12</td>
    <td>14.29</td>
    <td>hpd_zh</td>
    <td>20.80</td>
    <td>6.13</td>
    <td>7.65</td>
    <td>17.24</td>
    <td>18.37</td>
    <td>19.83</td>
    <td>20.90</td>
    <td>20.83</td>
  </tr>
  <tr>
    <td rowspan="10">Socialbench</td>
    <td>socialbench_sap_zh</td>
    <td>73.08</td>
    <td>39.04</td>
    <td>64.22</td>
    <td>75.78</td>
    <td>79.41</td>
    <td>82.36</td>
    <td>86.92</td>
    <td>83.54</td>
    <td>socialbench_sap_en</td>
    <td>81.04</td>
    <td>36.99</td>
    <td>61.70</td>
    <td>80.98</td>
    <td>89.19</td>
    <td>92.34</td>
    <td>92.89</td>
    <td>89.33</td>
    <td>socialbench_sa_rolestyle_zh</td>
    <td>81.59</td>
    <td>27.84</td>
    <td>40.41</td>
    <td>58.51</td>
    <td>68.93</td>
    <td>71.24</td>
    <td>77.71</td>
    <td>74.70</td>
    <td>socialbench_sa_rolestyle_en</td>
    <td>81.62</td>
    <td>44.96</td>
    <td>50.68</td>
    <td>71.35</td>
    <td>80.27</td>
    <td>83.24</td>
    <td>87.43</td>
    <td>90.81</td>
    <td>socialbench_sa_roleknowledge_zh</td>
    <td>89.07</td>
    <td>53.09</td>
    <td>65.08</td>
    <td>85.02</td>
    <td>88.16</td>
    <td>94.03</td>
    <td>96.26</td>
    <td>96.30</td>
    <td>socialbench_sa_roleknowledge_en</td>
    <td>83.13</td>
    <td>44.03</td>
    <td>37.63</td>
    <td>76.63</td>
    <td>79.62</td>
    <td>85.91</td>
    <td>80.27</td>
    <td>96.96</td>
    <td>socialbench_mem_short_zh</td>
    <td>83.16</td>
    <td>13.35</td>
    <td>48.03</td>
    <td>63.83</td>
    <td>76.50</td>
    <td>88.41</td>
    <td>76.33</td>
    <td>82.53</td>
    <td>socialbench_mem_short_en</td>
    <td>83.15</td>
    <td>16.35</td>
    <td>48.40</td>
    <td>72.78</td>
    <td>76.82</td>
    <td>80.24</td>
    <td>87.21</td>
    <td>86.90</td>
    <td>socialbench_mem_long_zh</td>
    <td>69.04</td>
    <td>11.20</td>
    <td>58.46</td>
    <td>61.46</td>
    <td>75.30</td>
    <td>87.78</td>
    <td>75.52</td>
    <td>82.32</td>
    <td>socialbench_mem_long_en</td>
    <td>69.01</td>
    <td>14.01</td>
    <td>58.46</td>
    <td>61.46</td>
    <td>75.30</td>
    <td>87.78</td>
    <td>75.52</td>
    <td>83.92</td>
  </tr>
  <tr>
    <td>InCharacter</td>
    <td>incharacter_16personalities_en</td>
    <td>0.48</td>
    <td>0.44</td>
    <td>0.56</td>
    <td>0.45</td>
    <td>0.53</td>
    <td>0.54</td>
    <td>0.41</td>
    <td>0.39</td>
    <td>incharacter_16personalities_zh</td>
    <td>0.50</td>
    <td>0.43</td>
    <td>0.50</td>
    <td>0.39</td>
    <td>0.36</td>
    <td>0.68</td>
    <td>0.32</td>
    <td>0.45</td>
    <td>incharacter_BFI_en</td>
    <td>0.50</td>
    <td>0.50</td>
    <td>0.50</td>
    <td>0.57</td>
    <td>0.47</td>
    <td>0.47</td>
    <td>0.53</td>
    <td>0.46</td>
    <td>incharacter_BFI_zh</td>
    <td>0.53</td>
    <td>0.57</td>
    <td>0.57</td>
    <td>0.53</td>
    <td>0.60</td>
    <td>0.60</td>
    <td>0.63</td>
    <td>0.57</td>
  </tr>
  <tr>
    <td>CharacterEval</td>
    <td>charactereval</td>
    <td>2.52</td>
    <td>1.70</td>
    <td>2.05</td>
    <td>2.45</td>
    <td>2.77</td>
    <td>3.08</td>
    <td>2.96</td>
    <td>3.21</td>
  </tr>
  <tr>
    <td>RoleBench</td>
    <td>instruction_generalization_en</td>
    <td>18.57</td>
    <td>20.41</td>
    <td>20.67</td>
    <td>20.73</td>
    <td>21.15</td>
    <td>21.89</td>
    <td>13.28</td>
    <td>3.21</td>
    <td>instruction_generalization_zh</td>
    <td>19.45</td>
    <td>16.63</td>
    <td>15.25</td>
    <td>18.18</td>
    <td>16.88</td>
    <td>16.68</td>
    <td>17.36</td>
    <td>20.39</td>
    <td>role_generalization_en</td>
    <td>23.93</td>
    <td>19.04</td>
    <td>19.18</td>
    <td>21.19</td>
    <td>21.68</td>
    <td>23.08</td>
    <td>22.29</td>
    <td>17.20</td>
  </tr>
</table>

Table 4: Direct zero-shot role-playing performance of different LLMs on various role-playing benchmarks (part1).

<table>
  <tr>
    <th>Dataset</th>
    <th>Sub Dataset</th>
    <th>Gemma-2 -2B-it</th>
    <th>Gemma-2 -9B-it</th>
    <th>Gemma-2 -27B-it</th>
    <th>Llama-3-2.1B -Instruct</th>
    <th>Llama-3-2.1B -Instruct</th>
    <th>Llama-3.1-8B -Instruct</th>
    <th>Llama-3.1-8B -Instruct</th>
    <th>Mistral-8B -Instruct-2410</th>
    <th>Mistral-7B -Instruct-v0.3</th>
  </tr>
  <tr>
    <td></td>
    <td>Cross</td>
    <td>36.40</td>
    <td>62.92</td>
    <td>65.62</td>
    <td>30.79</td>
    <td>51.91</td>
    <td>56.63</td>
    <td>65.17</td>
    <td>47.64</td>
    <td>52.81</td>
  </tr>
  <tr>
    <td>HPD</td>
    <td>hpd_en</td>
    <td>12.30</td>
    <td>12.13</td>
    <td>10.66</td>
    <td>6.78</td>
    <td>11.10</td>
    <td>14.33</td>
    <td>12.47</td>
    <td>15.54</td>
    <td>14.74</td>
    <td>hpd_zh</td>
    <td>11.50</td>
    <td>13.08</td>
    <td>14.20</td>
    <td>9.82</td>
    <td>9.63</td>
    <td>16.93</td>
    <td>19.10</td>
    <td>21.12</td>
    <td>19.20</td>
  </tr>
  <tr>
    <td rowspan="10">SocialBench</td>
    <td>socialbench_sap_zh</td>
    <td>75.86</td>
    <td>84.81</td>
    <td>88.52</td>
    <td>36.54</td>
    <td>74.77</td>
    <td>83.54</td>
    <td>93.25</td>
    <td>78.06</td>
    <td>77.22</td>
    <td>socialbench_sap_en</td>
    <td>75.92</td>
    <td>90.97</td>
    <td>91.96</td>
    <td>42.54</td>
    <td>86.05</td>
    <td>91.31</td>
    <td>85.91</td>
    <td>85.77</td>
    <td>85.77</td>
    <td>socialbench_sa_rolestyle_zh</td>
    <td>37.93</td>
    <td>28.50</td>
    <td>31.06</td>
    <td>26.01</td>
    <td>51.39</td>
    <td>64.97</td>
    <td>76.78</td>
    <td>64.40</td>
    <td>55.11</td>
    <td>socialbench_sa_rolestyle_en</td>
    <td>47.16</td>
    <td>66.35</td>
    <td>57.97</td>
    <td>40.68</td>
    <td>72.30</td>
    <td>81.49</td>
    <td>63.65</td>
    <td>84.73</td>
    <td>77.70</td>
    <td>socialbench_sa_roleknowledge_zh</td>
    <td>55.56</td>
    <td>80.00</td>
    <td>74.32</td>
    <td>39.26</td>
    <td>69.63</td>
    <td>80.49</td>
    <td>92.59</td>
    <td>79.75</td>
    <td>69.53</td>
    <td>socialbench_sa_roleknowledge_en</td>
    <td>55.87</td>
    <td>80.77</td>
    <td>77.72</td>
    <td>43.52</td>
    <td>83.66</td>
    <td>76.46</td>
    <td>78.82</td>
    <td>83.91</td>
    <td>81.68</td>
    <td>socialbench_mem_short_zh</td>
    <td>46.80</td>
    <td>80.50</td>
    <td>85.97</td>
    <td>47.42</td>
    <td>58.92</td>
    <td>81.56</td>
    <td>84.27</td>
    <td>85.70</td>
    <td>85.38</td>
    <td>socialbench_mem_short_en</td>
    <td>55.32</td>
    <td>68.97</td>
    <td>45.39</td>
    <td>39.98</td>
    <td>41.83</td>
    <td>82.03</td>
    <td>88.56</td>
    <td>76.20</td>
    <td>82.26</td>
    <td>socialbench_mem_long_zh</td>
    <td>57.68</td>
    <td>79.25</td>
    <td>80.45</td>
    <td>44.59</td>
    <td>65.73</td>
    <td>88.82</td>
    <td>91.50</td>
    <td>88.60</td>
    <td>86.75</td>
    <td>socialbench_mem_long_en</td>
    <td>59.62</td>
    <td>68.01</td>
    <td>44.13</td>
    <td>32.28</td>
    <td>65.96</td>
    <td>89.14</td>
    <td>88.87</td>
    <td>82.26</td>
    <td>80.96</td>
  </tr>
  <tr>
    <td>InCharacter</td>
    <td>incharacter_16personalities_en</td>
    <td>0.58</td>
    <td>0.54</td>
    <td>0.54</td>
    <td>0.54</td>
    <td>0.54</td>
    <td>0.44</td>
    <td>0.44</td>
    <td>0.44</td>
    <td>0.46</td>
    <td>incharacter_16personalities_zh</td>
    <td>0.56</td>
    <td>0.56</td>
    <td>0.56</td>
    <td>0.56</td>
    <td>0.56</td>
    <td>0.64</td>
    <td>0.64</td>
    <td>0.64</td>
    <td>0.68</td>
    <td>incharacter_BFI_en</td>
    <td>0.50</td>
    <td>0.52</td>
    <td>0.50</td>
    <td>0.39</td>
    <td>0.54</td>
    <td>0.50</td>
    <td>0.50</td>
    <td>0.50</td>
    <td>0.45</td>
    <td>incharacter_BFI_zh</td>
    <td>0.50</td>
    <td>0.57</td>
    <td>0.57</td>
    <td>0.53</td>
    <td>0.47</td>
    <td>0.67</td>
    <td>0.57</td>
    <td>0.63</td>
    <td>0.63</td>
  </tr>
  <tr>
    <td>CharacterEval</td>
    <td>charactereval</td>
    <td>2.23</td>
    <td>2.49</td>
    <td>2.53</td>
    <td>2.14</td>
    <td>2.24</td>
    <td>2.89</td>
    <td>2.90</td>
    <td>2.63</td>
    <td>2.26</td>
  </tr>
  <tr>
    <td>RoleBench</td>
    <td>rolebench_instruction_generalization_en</td>
    <td>20.13</td>
    <td>20.73</td>
    <td>21.95</td>
    <td>13.28</td>
    <td>15.72</td>
    <td>16.87</td>
    <td>17.93</td>
    <td>25.39</td>
    <td>18.89</td>
    <td>rolebench_instruction_generalization_zh</td>
    <td>15.89</td>
    <td>17.68</td>
    <td>17.32</td>
    <td>9.72</td>
    <td>15.95</td>
    <td>17.54</td>
    <td>18.81</td>
    <td>20.90</td>
    <td>14.38</td>
    <td>rolebench_generalization_en</td>
    <td>20.66</td>
    <td>20.70</td>
    <td>21.72</td>
    <td>12.57</td>
    <td>14.22</td>
    <td>15.29</td>
    <td>15.60</td>
    <td>24.55</td>
    <td>17.25</td>
  </tr>
</table>

Table 5: Direct zero-shot role-playing performance of different LLMs on various role-playing benchmarks (part2).

<table>
  <tr>
    <th>Dataset</th>
    <th>Sub Dataset</th>
    <th colspan="2">OpenAI GPT4-Turbo</th>
    <th colspan="2">Qwen2.5-0.5B -Instruct</th>
    <th colspan="2">Qwen2.5-1.5B -Instruct</th>
    <th colspan="2">Qwen2.5-3B-Instruct</th>
    <th colspan="2">Qwen2.5-7B -Instruct</th>
    <th colspan="2">Qwen2.5-14B -Instruct</th>
    <th colspan="2">Qwen2.5-32B -Instruct</th>
    <th colspan="2">Qwen2.5-72B -Instruct</th>
  </tr>
  <tr>
    <td></td>
    <td>COT</td>
    <td>cross</td>
    <td></td>
    <td>cross</td>
    <td>46.74</td>
    <td>26.97</td>
    <td>32.13</td>
    <td>39.78</td>
    <td>41.80</td>
    <td>58.20</td>
    <td>48.54</td>
    <td>48.76</td>
    <td>HPD</td>
    <td>hpd_en</td>
    <td>15.37</td>
    <td>7.88</td>
    <td>11.38</td>
    <td>11.45</td>
    <td>15.01</td>
    <td>14.82</td>
    <td>15.36</td>
    <td>10.03</td>
    <td>hpd_zh</td>
    <td>20.75</td>
    <td>13.49</td>
    <td>13.29</td>
    <td>15.73</td>
    <td>19.44</td>
    <td>20.66</td>
    <td>21.73</td>
    <td>20.75</td>
  </tr>
  <tr>
    <td rowspan="10">SocialBench</td>
    <td>socialbench_sap_zh</td>
    <td>70.21</td>
    <td>32.32</td>
    <td>64.22</td>
    <td>75.27</td>
    <td>77.22</td>
    <td>75.02</td>
    <td>82.53</td>
    <td>75.95</td>
    <td>socialbench_sap_en</td>
    <td>59.10</td>
    <td>29.55</td>
    <td>59.78</td>
    <td>78.25</td>
    <td>85.64</td>
    <td>87.41</td>
    <td>87.55</td>
    <td>84.13</td>
    <td>socialbench_sa_rolestyle_zh</td>
    <td>54.18</td>
    <td>26.63</td>
    <td>45.82</td>
    <td>51.39</td>
    <td>69.66</td>
    <td>70.28</td>
    <td>78.02</td>
    <td>72.45</td>
    <td>socialbench_sa_rolestyle_en</td>
    <td>54.01</td>
    <td>30.27</td>
    <td>46.49</td>
    <td>62.70</td>
    <td>63.82</td>
    <td>80.14</td>
    <td>81.76</td>
    <td>81.76</td>
    <td>socialbench_sa_roleknowledge_zh</td>
    <td>66.67</td>
    <td>32.35</td>
    <td>61.98</td>
    <td>83.70</td>
    <td>89.38</td>
    <td>95.06</td>
    <td>94.07</td>
    <td>97.53</td>
    <td>socialbench_sa_roleknowledge_en</td>
    <td>46.05</td>
    <td>37.25</td>
    <td>67.61</td>
    <td>82.39</td>
    <td>89.47</td>
    <td>95.04</td>
    <td>95.75</td>
    <td>93.52</td>
    <td>socialbench_mem_short_zh</td>
    <td>79.09</td>
    <td>13.17</td>
    <td>28.98</td>
    <td>72.04</td>
    <td>77.69</td>
    <td>87.58</td>
    <td>82.42</td>
    <td>83.98</td>
    <td>socialbench_mem_short_en</td>
    <td>76.75</td>
    <td>22.49</td>
    <td>40.18</td>
    <td>68.91</td>
    <td>78.75</td>
    <td>86.48</td>
    <td>84.26</td>
    <td>85.51</td>
    <td>socialbench_mem_long_zh</td>
    <td>59.74</td>
    <td>12.16</td>
    <td>38.22</td>
    <td>67.98</td>
    <td>78.20</td>
    <td>86.69</td>
    <td>84.07</td>
    <td>85.79</td>
    <td>socialbench_mem_long_en</td>
    <td>55.18</td>
    <td>14.44</td>
    <td>54.29</td>
    <td>67.21</td>
    <td>66.66</td>
    <td>82.15</td>
    <td>84.65</td>
  </tr>
  <tr>
    <td>InCharacter</td>
    <td>incharacter_16personalities_en</td>
    <td>0.75</td>
    <td>0.49</td>
    <td>0.47</td>
    <td>0.49</td>
    <td>0.54</td>
    <td>0.43</td>
    <td>0.52</td>
    <td>0.53</td>
    <td>incharacter_16personalities_zh</td>
    <td>0.65</td>
    <td>0.68</td>
    <td>0.50</td>
    <td>0.68</td>
    <td>0.36</td>
    <td>0.68</td>
    <td>0.55</td>
    <td>0.50</td>
    <td>incharacter_BFI_en</td>
    <td>0.40</td>
    <td>0.43</td>
    <td>0.50</td>
    <td>0.49</td>
    <td>0.52</td>
    <td>0.49</td>
    <td>0.49</td>
    <td>0.42</td>
    <td>incharacter_BFI_zh</td>
    <td>0.73</td>
    <td>0.63</td>
    <td>0.67</td>
    <td>0.60</td>
    <td>0.63</td>
    <td>0.53</td>
    <td>0.70</td>
    <td>0.70</td>
  </tr>
  <tr>
    <td>CharacterEval</td>
    <td>charactereval</td>
    <td>2.58</td>
    <td>1.81</td>
    <td>2.52</td>
    <td>2.39</td>
    <td>2.65</td>
    <td>2.99</td>
    <td>3.04</td>
    <td>3.10</td>
  </tr>
  <tr>
    <td>RoleBench</td>
    <td>instruction_generalization_en</td>
    <td>27.13</td>
    <td>16.31</td>
    <td>22.33</td>
    <td>22.66</td>
    <td>25.81</td>
    <td>25.55</td>
    <td>24.60</td>
    <td>23.61</td>
    <td>instruction_generalization_zh</td>
    <td>21.19</td>
    <td>14.12</td>
    <td>17.82</td>
    <td>16.55</td>
    <td>19.83</td>
    <td>18.95</td>
    <td>19.81</td>
    <td>18.38</td>
    <td>role_generalization_en</td>
    <td>27.42</td>
    <td>18.04</td>
    <td>22.86</td>
    <td>22.73</td>
    <td>25.99</td>
    <td>25.77</td>
    <td>25.24</td>
    <td>24.42</td>
  </tr>
</table>

Table 6: Performance of role-playing with chain-of-thought reasoning on various role-playing benchmarks (part 1).
<table>
  <tr>
    <th rowspan="2">Dataset</th>
    <th rowspan="2">Sub Dataset</th>
    <th colspan="2">Gemma-2</th>
    <th colspan="2">Gemma-2</th>
    <th colspan="2">Llama-3-2.1B</th>
    <th colspan="2">Llama-3-2.1B</th>
    <th colspan="2">Llama-3-8B</th>
    <th colspan="2">Llama-3-70B</th>
    <th colspan="2">Mistral-8B</th>
    <th colspan="2">Mistral-7B</th>
  </tr>
  <tr>
    <th>-2B-it</th><th>-0B-it</th>
    <th>-2B-it</th><th>-Instruct</th>
    <th>-Instruct</th><th>-Instruct</th>
    <th>-Instruct</th><th>-Instruct</th>
    <th>-Instruct</th><th>-Instruct</th>
    <th>-Instruct</th><th>-Instruct</th>
    <th>-Instruct</th><th>-Instruct</th>
    <th>-Instruct</th><th>-Instruct</th>
  </tr>
  <tr>
    <td>Cross</td><td>cross</td><td>35.48</td><td>50.11</td><td>42.25</td><td>27.68</td><td>40.45</td><td>48.97</td><td>62.70</td><td>39.96</td><td>41.12</td>
  </tr>
  <tr>
    <td rowspan="2">HPD</td><td>hpdl_en</td><td>7.01</td><td>8.06</td><td>7.39</td><td>6.84</td><td>11.17</td><td>4.80</td><td>6.85</td><td>15.15</td><td>10.68</td>
  </tr>
  <tr>
    <td>hpdl_zh</td><td>13.49</td><td>14.82</td><td>9.81</td><td>12.48</td><td>16.04</td><td>16.60</td><td>20.27</td><td>12.82</td><td>15.64</td>
  </tr>
  <tr>
    <td rowspan="12">SocialBench</td><td>socialbench_sap_zh</td><td>71.31</td><td>85.91</td><td>85.74</td><td>33.42</td><td>78.99</td><td>85.65</td><td>92.07</td><td>80.25</td><td>74.60</td>
  </tr>
  <tr><td>socialbench_sap_en</td><td>77.15</td><td>88.37</td><td>91.93</td><td>41.04</td><td>80.44</td><td>66.21</td><td>95.21</td><td>63.34</td><td>58.69</td></tr>
  <tr><td>socialbench_sa_rolestyle_zh</td><td>49.85</td><td>68.73</td><td>69.04</td><td>23.22</td><td>50.77</td><td>61.30</td><td>77.09</td><td>66.87</td><td>52.94</td></tr>
  <tr><td>socialbench_sa_rolestyle_en</td><td>64.19</td><td>80.41</td><td>82.70</td><td>30.95</td><td>55.14</td><td>54.46</td><td>89.19</td><td>48.65</td><td>50.95</td></tr>
  <tr><td>socialbench_sa_roboknowledge_zh</td><td>69.41</td><td>85.66</td><td>89.88</td><td>29.44</td><td>64.94</td><td>78.77</td><td>92.77</td><td>77.78</td><td>68.15</td></tr>
  <tr><td>socialbench_sa_roboknowledge_en</td><td>74.19</td><td>93.52</td><td>93.42</td><td>33.60</td><td>71.76</td><td>70.85</td><td>95.34</td><td>47.98</td><td>44.64</td></tr>
  <tr><td>socialbench_mem_short_zh</td><td>63.80</td><td>81.29</td><td>82.42</td><td>31.99</td><td>45.39</td><td>81.13</td><td>91.67</td><td>90.00</td><td>71.40</td></tr>
  <tr><td>socialbench_mem_short_en</td><td>70.49</td><td>70.57</td><td>49.78</td><td>27.71</td><td>50.84</td><td>85.37</td><td>89.23</td><td>85.54</td><td>86.62</td></tr>
  <tr><td>socialbench_mem_long_zh</td><td>63.67</td><td>83.56</td><td>82.16</td><td>29.47</td><td>35.65</td><td>74.90</td><td>89.66</td><td>87.70</td><td>73.89</td></tr>
  <tr><td>socialbench_mem_long_en</td><td>63.51</td><td>82.53</td><td>80.90</td><td>26.07</td><td>40.81</td><td>87.61</td><td>88.61</td><td>83.60</td><td></td></tr>
  <tr><td>incharacter_16Personalities_en</td><td>0.56</td><td>0.51</td><td>0.55</td><td>0.48</td><td>0.54</td><td>0.49</td><td>0.51</td><td>0.44</td><td>0.53</td></tr>
  <tr><td>incharacter_16Personalities_zh</td><td>0.59</td><td>0.68</td><td>0.55</td><td>0.64</td><td>0.64</td><td>0.73</td><td>0.77</td><td>0.45</td><td>0.45</td></tr>
  <tr><td>InCharacter</td><td>incharacter_BFI_en</td><td>0.49</td><td>0.47</td><td>0.48</td><td>0.55</td><td>0.49</td><td>0.47</td><td>0.54</td><td>0.46</td><td>0.52</td></tr>
  <tr><td></td><td>incharacter_BFI_zh</td><td>0.57</td><td>0.70</td><td>0.53</td><td>0.67</td><td>0.67</td><td>0.70</td><td>0.63</td><td>0.53</td><td>0.60</td></tr>
  <tr><td>CharacterEval</td><td>charactereval</td><td>2.29</td><td>2.48</td><td>2.56</td><td>2.15</td><td>2.11</td><td>2.67</td><td>2.70</td><td>2.55</td><td>2.21</td></tr>
  <tr><td rowspan="2">RoleBench</td><td>instruction_generalization_en</td><td>22.18</td><td>23.54</td><td>23.55</td><td>17.53</td><td>19.48</td><td>22.38</td><td>20.72</td><td>28.03</td><td>24.89</td></tr>
  <tr><td>instruction_generalization_zh</td><td>16.31</td><td>18.67</td><td>18.65</td><td>14.58</td><td>16.03</td><td>19.38</td><td>19.58</td><td>20.83</td><td>18.04</td></tr>
  <tr><td></td><td>role_generalization_en</td><td>23.52</td><td>24.33</td><td>24.73</td><td>15.59</td><td>17.17</td><td>21.74</td><td>17.79</td><td>26.70</td><td>21.00</td></tr>
</table>

Table 7: Performance of role-playing with chain-of-thought reasoning on various role-playing benchmarks (part 2).

<table>
  <tr>
    <th rowspan="2">Dataset</th>
    <th rowspan="2">Sub Dataset</th>
    <th colspan="2">OpenAI o1-mini</th>
    <th colspan="2">DeepSeek-R1</th>
    <th colspan="2">QwO-Q2B -Preview</th>
    <th colspan="2">DeepSeek-R1 -Distill-Llama-8B</th>
    <th colspan="2">DeepSeek-R1 -Distill-Qwen-14B</th>
    <th colspan="2">DeepSeek-R1 -Distill-Qwen-32B</th>
    <th colspan="2">DeepSeek-R1 -Distill-Llama-70B</th>
  </tr>
  <tr>
    <th>o1-mini</th><th>R1</th>
    <th>R1</th><th>R1</th>
    <th>R1</th><th>R1</th>
    <th>R1</th><th>R1</th>
    <th>R1</th><th>R1</th>
    <th>R1</th><th>R1</th>
    <th>R1</th><th>R1</th>
  </tr>
  <tr>
    <td>Cross</td><td>cross</td><td>45.23</td><td>44.19</td><td>62.02</td><td>45.17</td><td>46.97</td><td>50.11</td><td>51.91</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td rowspan="2">HPD</td><td>hpdl_en</td><td>14.96</td><td>13.28</td><td>3.95</td><td>9.25</td><td>12.21</td><td>12.41</td><td>12.41</td><td>12.04</td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr><td>hpdl_zh</td><td>19.87</td><td>20.87</td><td>10.14</td><td>9.22</td><td>17.48</td><td>18.58</td><td>15.85</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr>
    <td rowspan="12">SocialBench</td><td>socialbench_sap_zh</td><td>68.11</td><td>64.72</td><td>79.16</td><td>77.97</td><td>81.94</td><td>88.69</td><td>89.70</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr><td>socialbench_sap_en</td><td>57.08</td><td>56.52</td><td>85.09</td><td>83.31</td><td>88.92</td><td>81.67</td><td>94.12</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>socialbench_sa_rolestyle_zh</td><td>52.29</td><td>56.87</td><td>70.90</td><td>64.09</td><td>72.76</td><td>78.02</td><td>78.02</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>socialbench_sa_rolestyle_en</td><td>48.29</td><td>46.33</td><td>84.05</td><td>67.97</td><td>77.97</td><td>61.22</td><td>87.84</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>socialbench_sa_roboknowledge_zh</td><td>61.47</td><td>62.10</td><td>91.09</td><td>81.23</td><td>93.23</td><td>94.37</td><td>94.37</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>socialbench_sa_roboknowledge_en</td><td>64.17</td><td>47.21</td><td>95.55</td><td>85.83</td><td>82.89</td><td>80.47</td><td>97.06</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>socialbench_mem_short_zh</td><td>77.42</td><td>77.82</td><td>83.09</td><td>71.77</td><td>76.03</td><td>80.91</td><td>86.88</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>socialbench_mem_short_en</td><td>74.23</td><td>73.28</td><td>75.68</td><td>78.60</td><td>80.13</td><td>89.87</td><td>90.47</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>socialbench_mem_long_zh</td><td>37.42</td><td>39.26</td><td>82.03</td><td>73.34</td><td>79.87</td><td>85.45</td><td>86.80</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>socialbench_mem_long_en</td><td>55.71</td><td>57.45</td><td>71.46</td><td>80.40</td><td>84.14</td><td>88.69</td><td>89.29</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>incharacter_16Personalities_en</td><td>0.67</td><td>0.67</td><td>0.51</td><td>0.52</td><td>0.47</td><td>0.61</td><td>0.60</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>incharacter_16Personalities_zh</td><td>0.52</td><td>0.59</td><td>0.45</td><td>0.64</td><td>0.59</td><td>0.55</td><td>0.64</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>InCharacter</td><td>incharacter_BFI_en</td><td>0.44</td><td>0.41</td><td>0.49</td><td>0.50</td><td>0.49</td><td>0.41</td><td>0.50</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td></td><td>incharacter_BFI_zh</td><td>0.63</td><td>0.63</td><td>0.43</td><td>0.53</td><td>0.50</td><td>0.57</td><td>0.43</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>CharacterEval</td><td>charactereval</td><td>2.56</td><td>2.32</td><td>2.78</td><td>2.38</td><td>2.70</td><td>2.69</td><td>2.63</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td rowspan="2">RoleBench</td><td>instruction_generalization_en</td><td>24.91</td><td>23.64</td><td>9.54</td><td>19.54</td><td>20.45</td><td>17.66</td><td>19.54</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td>instruction_generalization_zh</td><td>20.29</td><td>18.73</td><td>17.45</td><td>12.22</td><td>18.31</td><td>15.86</td><td>14.73</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
  <tr><td></td><td>role_generalization_en</td><td>26.66</td><td>24.47</td><td>10.97</td><td>18.21</td><td>19.76</td><td>16.38</td><td>17.66</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
</table>

Table 8: Performance of role-playing using reasoning-optimized LLMs on various role-playing benchmarks.

<table>
  <tr>
    <th rowspan="2">Metric</th>
    <th colspan="2">OpenAI GPT-4 Turbo</th>
    <th colspan="2">Qwen2.5-0.5B -Instruct</th>
    <th colspan="2">Qwen2.5-1.5B -Instruct</th>
    <th colspan="2">Qwen2.5 -Instruct</th>
    <th colspan="2">Qwen2.5-7B -Instruct</th>
    <th colspan="2">Qwen2.5-14B -Instruct</th>
    <th colspan="2">Qwen2.5-32B -Instruct</th>
    <th colspan="2">Qwen2.5-72B -Instruct</th>
  </tr>
  <tr>
    <th></th><th></th>
    <th></th><th></th>
    <th></th><th></th>
    <th></th><th></th>
    <th></th><th></th>
    <th></th><th></th>
    <th></th><th></th>
    <th></th><th></th>
  </tr>
  <tr>
    <td>Accuracy</td><td>2.92</td><td>1.90</td><td>2.35</td><td>2.86</td><td>2.98</td><td>3.17</td><td>3.10</td><td>3.29</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Behavior</td><td>1.39</td><td>1.29</td><td>1.19</td><td>1.26</td><td>1.67</td><td>2.40</td><td>2.20</td><td>2.65</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Coherence</td><td>3.29</td><td>2.20</td><td>2.76</td><td>3.19</td><td>3.60</td><td>3.90</td><td>3.76</td><td>3.93</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Communication Skills</td><td>2.68</td><td>1.66</td><td>1.87</td><td>2.75</td><td>2.98</td><td>3.25</td><td>3.15</td><td>3.53</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Consistency</td><td>2.90</td><td>1.83</td><td>2.48</td><td>2.73</td><td>3.27</td><td>3.65</td><td>3.46</td><td>3.64</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Diversity</td><td>1.36</td><td>1.19</td><td>1.19</td><td>1.31</td><td>1.60</td><td>2.00</td><td>1.93</td><td>2.18</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Empathy</td><td>2.94</td><td>1.88</td><td>2.33</td><td>2.86</td><td>3.14</td><td>3.33</td><td>3.21</td><td>3.46</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Exposure</td><td>1.91</td><td>1.39</td><td>1.42</td><td>2.02</td><td>2.03</td><td>2.18</td><td>2.21</td><td>2.47</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Fluency</td><td>3.03</td><td>1.94</td><td>2.42</td><td>2.90</td><td>3.34</td><td>3.53</td><td>3.39</td><td>3.63</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Hallucination</td><td>2.55</td><td>1.70</td><td>2.08</td><td>2.46</td><td>2.78</td><td>3.05</td><td>2.94</td><td>3.17</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Humanlikeness</td><td>2.78</td><td>1.75</td><td>2.41</td><td>2.59</td><td>3.11</td><td>3.48</td><td>3.30</td><td>3.47</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>Utterance</td><td>2.48</td><td>1.67</td><td>2.11</td><td>2.41</td><td>2.74</td><td>3.04</td><td>2.90</td><td>3.09</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
  </tr>
</table>

Table 9: Fine-grained results of direct zero-shot role-playing on CharacterEval benchmark (part1).
<table>
  <tr>
    <th>Metrics</th>
    <th>Gemma-2 -2B-it</th>
    <th>Gemma-2 -9B-it</th>
    <th>Gemma-2 -27B-it</th>
    <th>Llama-3.2-1B -Instruct</th>
    <th>Llama-3.2-3B -Instruct</th>
    <th>Llama-3.1-8B -Instruct</th>
    <th>Llama-3.3-70B -Instruct</th>
    <th>Mistral-8B -Instruct-2410</th>
    <th>Mistral-7B -Instruct-v0.3</th>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>2.52</td>
    <td>2.80</td>
    <td>2.90</td>
    <td>2.37</td>
    <td>2.56</td>
    <td>2.94</td>
    <td>3.12</td>
    <td>2.85</td>
    <td>2.62</td>
  </tr>
  <tr>
    <td>Behavior</td>
    <td>1.20</td>
    <td>1.14</td>
    <td>1.16</td>
    <td>1.86</td>
    <td>1.58</td>
    <td>2.61</td>
    <td>2.04</td>
    <td>1.16</td>
    <td>1.22</td>
  </tr>
  <tr>
    <td>Coherence</td>
    <td>3.00</td>
    <td>3.42</td>
    <td>3.47</td>
    <td>2.69</td>
    <td>2.87</td>
    <td>3.55</td>
    <td>3.70</td>
    <td>3.65</td>
    <td>3.04</td>
  </tr>
  <tr>
    <td>Communication Skills</td>
    <td>2.16</td>
    <td>2.12</td>
    <td>2.10</td>
    <td>2.08</td>
    <td>2.15</td>
    <td>2.94</td>
    <td>2.99</td>
    <td>2.61</td>
    <td>2.53</td>
  </tr>
  <tr>
    <td>Consistency</td>
    <td>2.67</td>
    <td>3.27</td>
    <td>3.38</td>
    <td>2.23</td>
    <td>2.53</td>
    <td>3.31</td>
    <td>3.43</td>
    <td>3.36</td>
    <td>2.49</td>
  </tr>
  <tr>
    <td>Diversity</td>
    <td>1.25</td>
    <td>1.20</td>
    <td>1.25</td>
    <td>1.57</td>
    <td>1.47</td>
    <td>2.18</td>
    <td>1.85</td>
    <td>1.22</td>
    <td>1.26</td>
  </tr>
  <tr>
    <td>Empathy</td>
    <td>2.51</td>
    <td>2.78</td>
    <td>2.81</td>
    <td>2.36</td>
    <td>2.49</td>
    <td>3.02</td>
    <td>3.16</td>
    <td>3.06</td>
    <td>2.64</td>
  </tr>
  <tr>
    <td>Exposure</td>
    <td>1.59</td>
    <td>1.50</td>
    <td>1.51</td>
    <td>1.67</td>
    <td>1.67</td>
    <td>2.12</td>
    <td>2.06</td>
    <td>1.79</td>
    <td>1.91</td>
  </tr>
  <tr>
    <td>Fluency</td>
    <td>2.74</td>
    <td>3.09</td>
    <td>3.10</td>
    <td>2.44</td>
    <td>2.60</td>
    <td>3.28</td>
    <td>3.40</td>
    <td>3.30</td>
    <td>2.68</td>
  </tr>
  <tr>
    <td>Hallucination</td>
    <td>2.23</td>
    <td>2.56</td>
    <td>2.55</td>
    <td>2.05</td>
    <td>2.17</td>
    <td>2.73</td>
    <td>2.87</td>
    <td>2.69</td>
    <td>2.26</td>
  </tr>
  <tr>
    <td>Humanlikeness</td>
    <td>2.64</td>
    <td>3.34</td>
    <td>3.43</td>
    <td>2.30</td>
    <td>2.58</td>
    <td>3.18</td>
    <td>3.31</td>
    <td>3.20</td>
    <td>2.29</td>
  </tr>
  <tr>
    <td>Utterance</td>
    <td>2.26</td>
    <td>2.61</td>
    <td>2.67</td>
    <td>2.09</td>
    <td>2.20</td>
    <td>2.77</td>
    <td>2.92</td>
    <td>2.71</td>
    <td>2.20</td>
  </tr>
</table>

Table 10: Fine-grained results of direct zero-shot role-playing on CharacterEval benchmark (part2).

<table>
  <tr>
    <th rowspan="2">Metric</th>
    <th colspan="2">OpenAI GPT-4 Turbo</th>
    <th colspan="2">Qwen2.5 0.5B-Instruct</th>
    <th colspan="2">Qwen2.5 1.5B-Instruct</th>
    <th colspan="2">Qwen2.5 3B-Instruct</th>
    <th colspan="2">Qwen2.5 7B-Instruct</th>
    <th colspan="2">Qwen2.5 14B-Instruct</th>
    <th colspan="2">Qwen2.5 32B-Instruct</th>
    <th colspan="2">Qwen2.5 72B-Instruct</th>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>2.87</td>
    <td>2.13</td>
    <td>2.76</td>
    <td>2.81</td>
    <td>2.96</td>
    <td>3.17</td>
    <td>3.19</td>
    <td>3.26</td>
    <td>2.80</td>
    <td>2.18</td>
    <td>2.38</td>
    <td>2.25</td>
    <td>2.36</td>
    <td>2.57</td>
    <td>2.57</td>
    <td>2.57</td>
    <td>2.57</td>
  </tr>
  <tr>
    <td>Behavior</td>
    <td>2.00</td>
    <td>1.37</td>
    <td>1.52</td>
    <td>1.30</td>
    <td>1.53</td>
    <td>2.18</td>
    <td>2.38</td>
    <td>2.25</td>
    <td>1.96</td>
    <td>1.39</td>
    <td>1.34</td>
    <td>1.39</td>
    <td>1.39</td>
    <td>1.39</td>
    <td>1.39</td>
    <td>1.39</td>
    <td>1.39</td>
  </tr>
  <tr>
    <td>Coherence</td>
    <td>3.27</td>
    <td>2.28</td>
    <td>3.41</td>
    <td>3.14</td>
    <td>3.49</td>
    <td>3.76</td>
    <td>3.79</td>
    <td>3.84</td>
    <td>3.26</td>
    <td>3.00</td>
    <td>3.23</td>
    <td>3.26</td>
    <td>3.51</td>
    <td>3.51</td>
    <td>3.51</td>
    <td>3.51</td>
    <td>3.51</td>
    <td>3.51</td>
  </tr>
  <tr>
    <td>Communication Skills</td>
    <td>2.98</td>
    <td>1.73</td>
    <td>1.72</td>
    <td>2.67</td>
    <td>3.00</td>
    <td>3.23</td>
    <td>3.26</td>
    <td>3.52</td>
    <td>2.01</td>
    <td>1.89</td>
    <td>1.89</td>
    <td>2.01</td>
    <td>2.01</td>
    <td>2.01</td>
    <td>2.01</td>
    <td>2.01</td>
    <td>2.01</td>
    <td>2.01</td>
  </tr>
  <tr>
    <td>Consistency</td>
    <td>2.72</td>
    <td>1.86</td>
    <td>3.35</td>
    <td>2.63</td>
    <td>3.06</td>
    <td>3.48</td>
    <td>3.49</td>
    <td>3.52</td>
    <td>2.04</td>
    <td>1.89</td>
    <td>1.89</td>
    <td>2.04</td>
    <td>2.04</td>
    <td>2.04</td>
    <td>2.04</td>
    <td>2.04</td>
    <td>2.04</td>
    <td>2.04</td>
  </tr>
  <tr>
    <td>Diversity</td>
    <td>1.70</td>
    <td>1.31</td>
    <td>1.43</td>
    <td>1.28</td>
    <td>1.47</td>
    <td>1.89</td>
    <td>1.89</td>
    <td>2.01</td>
    <td>1.61</td>
    <td>1.39</td>
    <td>1.21</td>
    <td>2.21</td>
    <td>2.27</td>
    <td>2.50</td>
    <td>2.50</td>
    <td>2.50</td>
    <td>2.50</td>
    <td>2.50</td>
  </tr>
  <tr>
    <td>Empathy</td>
    <td>2.93</td>
    <td>1.96</td>
    <td>2.71</td>
    <td>2.82</td>
    <td>3.03</td>
    <td>3.29</td>
    <td>3.32</td>
    <td>3.44</td>
    <td>2.04</td>
    <td>1.89</td>
    <td>1.89</td>
    <td>2.04</td>
    <td>2.04</td>
    <td>2.04</td>
    <td>2.04</td>
    <td>2.04</td>
    <td>2.04</td>
    <td>2.04</td>
  </tr>
  <tr>
    <td>Exposure</td>
    <td>2.26</td>
    <td>1.61</td>
    <td>1.39</td>
    <td>2.03</td>
    <td>2.11</td>
    <td>2.21</td>
    <td>2.27</td>
    <td>2.50</td>
    <td>1.79</td>
    <td>1.50</td>
    <td>1.50</td>
    <td>1.79</td>
    <td>1.79</td>
    <td>1.79</td>
    <td>1.79</td>
    <td>1.79</td>
    <td>1.79</td>
    <td>1.79</td>
  </tr>
  <tr>
    <td>Fluency</td>
    <td>2.90</td>
    <td>2.04</td>
    <td>3.04</td>
    <td>2.82</td>
    <td>3.09</td>
    <td>3.45</td>
    <td>3.49</td>
    <td>3.50</td>
    <td>2.41</td>
    <td>1.79</td>
    <td>2.50</td>
    <td>2.66</td>
    <td>2.95</td>
    <td>2.98</td>
    <td>3.06</td>
    <td>3.06</td>
    <td>3.06</td>
    <td>3.06</td>
  </tr>
  <tr>
    <td>Hallucination</td>
    <td>2.41</td>
    <td>1.79</td>
    <td>2.50</td>
    <td>2.39</td>
    <td>2.66</td>
    <td>2.95</td>
    <td>2.98</td>
    <td>3.06</td>
    <td>2.51</td>
    <td>1.87</td>
    <td>3.69</td>
    <td>2.48</td>
    <td>2.75</td>
    <td>3.36</td>
    <td>3.32</td>
    <td>3.28</td>
    <td>3.28</td>
    <td>3.28</td>
  </tr>
  <tr>
    <td>Humanlikeness</td>
    <td>2.51</td>
    <td>1.87</td>
    <td>3.69</td>
    <td>2.48</td>
    <td>2.75</td>
    <td>3.36</td>
    <td>3.32</td>
    <td>3.28</td>
    <td>2.44</td>
    <td>1.73</td>
    <td>2.73</td>
    <td>2.32</td>
    <td>2.62</td>
    <td>2.92</td>
    <td>2.93</td>
    <td>2.99</td>
    <td>2.99</td>
    <td>2.99</td>
  </tr>
</table>

Table 11: Fine-grained results of role-playing with CoT on CharacterEval benchmark (part 1).

<table>
  <tr>
    <th>Metrics</th>
    <th>Gemma-2 -2B-it</th>
    <th>Gemma-2 -9B-it</th>
    <th>Gemma-2 -27B-it</th>
    <th>Llama-3.2-1B -Instruct</th>
    <th>Llama-3.2-3B -Instruct</th>
    <th>Llama-3.1-8B -Instruct</th>
    <th>Llama-3.3-70B -Instruct</th>
    <th>Mistral-8B -Instruct-2410</th>
    <th>Mistral-7B -Instruct-v0.3</th>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>2.63</td>
    <td>2.88</td>
    <td>2.91</td>
    <td>2.52</td>
    <td>2.51</td>
    <td>2.90</td>
    <td>3.07</td>
    <td>2.87</td>
    <td>2.57</td>
  </tr>
  <tr>
    <td>Behavior</td>
    <td>1.28</td>
    <td>1.11</td>
    <td>1.14</td>
    <td>1.66</td>
    <td>1.20</td>
    <td>1.61</td>
    <td>1.34</td>
    <td>1.23</td>
    <td>1.39</td>
  </tr>
  <tr>
    <td>Coherence</td>
    <td>3.11</td>
    <td>3.38</td>
    <td>3.50</td>
    <td>2.71</td>
    <td>2.79</td>
    <td>3.38</td>
    <td>3.58</td>
    <td>3.40</td>
    <td>2.86</td>
  </tr>
  <tr>
    <td>Communication Skills</td>
    <td>2.88</td>
    <td>2.23</td>
    <td>2.33</td>
    <td>2.06</td>
    <td>1.96</td>
    <td>2.84</td>
    <td>2.91</td>
    <td>2.71</td>
    <td>2.39</td>
  </tr>
  <tr>
    <td>Consistency</td>
    <td>2.90</td>
    <td>3.19</td>
    <td>3.30</td>
    <td>2.22</td>
    <td>2.46</td>
    <td>2.98</td>
    <td>3.20</td>
    <td>3.03</td>
    <td>2.37</td>
  </tr>
  <tr>
    <td>Diversity</td>
    <td>1.29</td>
    <td>1.18</td>
    <td>1.20</td>
    <td>1.50</td>
    <td>1.22</td>
    <td>1.76</td>
    <td>1.35</td>
    <td>1.26</td>
    <td>1.35</td>
  </tr>
  <tr>
    <td>Empathy</td>
    <td>2.53</td>
    <td>2.77</td>
    <td>2.92</td>
    <td>2.38</td>
    <td>2.41</td>
    <td>2.95</td>
    <td>3.08</td>
    <td>3.01</td>
    <td>2.55</td>
  </tr>
  <tr>
    <td>Exposure</td>
    <td>1.44</td>
    <td>1.59</td>
    <td>1.58</td>
    <td>1.77</td>
    <td>1.52</td>
    <td>2.04</td>
    <td>2.01</td>
    <td>1.93</td>
    <td>1.83</td>
  </tr>
  <tr>
    <td>Fluency</td>
    <td>2.71</td>
    <td>3.00</td>
    <td>3.20</td>
    <td>2.47</td>
    <td>2.51</td>
    <td>3.06</td>
    <td>3.30</td>
    <td>3.13</td>
    <td>2.59</td>
  </tr>
  <tr>
    <td>Hallucination</td>
    <td>2.21</td>
    <td>2.53</td>
    <td>2.60</td>
    <td>2.13</td>
    <td>2.09</td>
    <td>2.63</td>
    <td>2.76</td>
    <td>2.57</td>
    <td>2.15</td>
  </tr>
  <tr>
    <td>Humanlikeness</td>
    <td>3.06</td>
    <td>3.30</td>
    <td>3.35</td>
    <td>2.28</td>
    <td>2.54</td>
    <td>2.89</td>
    <td>3.03</td>
    <td>2.87</td>
    <td>2.32</td>
  </tr>
  <tr>
    <td>Utterance</td>
    <td>2.40</td>
    <td>2.58</td>
    <td>2.66</td>
    <td>2.12</td>
    <td>2.13</td>
    <td>2.60</td>
    <td>2.72</td>
    <td>2.52</td>
    <td>2.18</td>
  </tr>
</table>

Table 12: Fine-grained results of role-playing with CoT on CharacterEval benchmark (part 2).

<table>
  <tr>
    <th>Metrics</th>
    <th>OpenAI o1-mini -R1</th>
    <th>Deepseek -R1</th>
    <th>QwQ-32B -Preview</th>
    <th>DeepSeek-R1 -Distill-Llama-8B</th>
    <th>DeepSeek-R1 -Distill-Qwen-14B</th>
    <th>DeepSeek-R1 -Distill-Qwen-32B</th>
    <th>DeepSeek-R1 -Distill-Llama-70B</th>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>2.45</td>
    <td>2.38</td>
    <td>3.08</td>
    <td>2.65</td>
    <td>2.95</td>
    <td>3.06</td>
    <td>2.96</td>
  </tr>
  <tr>
    <td>Behavior</td>
    <td>1.29</td>
    <td>1.27</td>
    <td>1.64</td>
    <td>1.63</td>
    <td>1.62</td>
    <td>1.31</td>
    <td>1.43</td>
  </tr>
  <tr>
    <td>Coherence</td>
    <td>3.61</td>
    <td>3.19</td>
    <td>3.61</td>
    <td>3.08</td>
    <td>3.46</td>
    <td>3.55</td>
    <td>3.47</td>
  </tr>
  <tr>
    <td>Communication Skills</td>
    <td>2.42</td>
    <td>2.31</td>
    <td>2.94</td>
    <td>2.40</td>
    <td>2.93</td>
    <td>2.83</td>
    <td>2.79</td>
  </tr>
  <tr>
    <td>Consistency</td>
    <td>2.87</td>
    <td>3.02</td>
    <td>3.31</td>
    <td>2.68</td>
    <td>3.15</td>
    <td>3.25</td>
    <td>3.12</td>
  </tr>
  <tr>
    <td>Diversity</td>
    <td>1.19</td>
    <td>1.20</td>
    <td>1.53</td>
    <td>1.49</td>
    <td>1.52</td>
    <td>1.33</td>
    <td>1.40</td>
  </tr>
  <tr>
    <td>Empathy</td>
    <td>2.98</td>
    <td>3.21</td>
    <td>3.14</td>
    <td>2.61</td>
    <td>3.03</td>
    <td>3.08</td>
    <td>2.97</td>
  </tr>
  <tr>
    <td>Exposure</td>
    <td>1.94</td>
    <td>2.08</td>
    <td>2.09</td>
    <td>1.88</td>
    <td>2.06</td>
    <td>2.01</td>
    <td>1.99</td>
  </tr>
  <tr>
    <td>Fluency</td>
    <td>3.44</td>
    <td>2.29</td>
    <td>3.30</td>
    <td>2.77</td>
    <td>3.23</td>
    <td>3.23</td>
    <td>3.10</td>
  </tr>
  <tr>
    <td>Hallucination</td>
    <td>2.81</td>
    <td>2.11</td>
    <td>2.85</td>
    <td>2.35</td>
    <td>2.76</td>
    <td>2.78</td>
    <td>2.68</td>
  </tr>
  <tr>
    <td>Humanlikeness</td>
    <td>3.21</td>
    <td>2.69</td>
    <td>3.15</td>
    <td>2.68</td>
    <td>3.01</td>
    <td>3.09</td>
    <td>2.99</td>
  </tr>
  <tr>
    <td>Utterance</td>
    <td>2.54</td>
    <td>2.12</td>
    <td>2.76</td>
    <td>2.36</td>
    <td>2.66</td>
    <td>2.71</td>
    <td>2.62</td>
  </tr>
</table>

Table 13: Fine-grained results of various reasoning-enhanced models on CharacterEval.

Metareview score: 3
Metareview: This paper presents a comprehensive investigation into how reasoning techniques affect the role-playing abilities of large language models (LLMs). By evaluating 24 LLMs across 6 benchmarks and 3 role-playing strategies—zero-shot, chain-of-thought (CoT), and reasoning-optimized models—the study uncovers several key findings. First, CoT reasoning can degrade role-playing performance due to attention dispersion and stylistic shifts. Second, LLMs fine-tuned for reasoning tend to underperform in role-playing tasks. Third, models demonstrate better performance in Chinese role-playing than in English, possibly due to differences in data generalization. Lastly, even state-of-the-art models struggle with higher-level dimensions like personality expression and knowledge grounding, highlighting the limitations of current approaches and suggesting new directions for future research.
Summary of reasons to publish: - The paper is clearly structured and easy to follow, making complex findings accessible.
- Extensive experiments across 24 models and 6 benchmarks provide strong empirical support for the paper’s six key findings.
- The paper proposes promising future directions, including role-aware CoT and a role-specific reasoning generation framework based on reinforcement learning.