Enhancing Persona Consistency for LLM s’ Role-Playing using Persona-Aware Contrastive Learning (2025)

Abstract: In recent years, large language models (LLMs) have achieved breakthrough progress in many dialogue generation tasks. However, their lack of emotion and fine-grained role awareness limits the model’s ability to provide personalized and diverse interactions further. Current methods face high costs in collecting high-quality annotated data for scenarios such as role-playing, and traditional human alignment methods are difficult to deploy due to the inherent diversity of model behavior in role-playing scenarios. Inspired by the alignment of models for safety behaviors through RLHF (Reinforcement Learning from Human Feedback), in this paper, we revisit model role-playing behavior from the perspective of persona alignment and propose a novel annotation-free framework named Persona-Aware Contrastive Learning (PCL) to align LLMs’ behavior during role-playing, enhancing the model’s role consistency. Specifically, we first design a role chain method to encourage the model to self-question based on the role characteristics and dialogue context to adjust personality consistency. Then, we further enhance the model’s role-playing strategy through iterative adversarial modeling between the use of role characteristics and not. Experiments on both black-box and white-box LLMs show that LLMs equipped with PCL significantly outperform vanilla LLMs under automatic evaluation methods (CharEval & GPT-4) and human expert evaluation.

## introduction due to the success of self-supervised learning and instruction tuning, large language models (llms) have achieved remarkable success in many nlp tasks, such as text generation @xcite , instruction following @xcite , summarization @xcite , and other applications @xcite * work done during an internship at xiaobing.ai † corresponding author @xcite . with the growing demand for diversity in psychology and entertainment, more and more research is focusing on how to guide llms to cultivate personalized imitation and role-playing capabilities, rather than merely providing knowledge interaction. however, an ai assistant designed to accomplish various knowledge interaction tasks finds it difficult to provide satisfactory personalized interactions due to the lack of emotions and perception of the real world, limiting the human imagination of applying llms in diverse scenarios. compared with the utilization of language models as general assistants for answering broad knowledge questions, the main difference of role-playing lies in that the key point is to keep the llm in line with the given specific role profiles @xcite @xcite . to incorporate the character's personality into the model interaction, early studies often adopted customized designs @xcite @xcite , such as using memory networks or custom transformers to effectively fuse storage-and embeddingbased personal information. nevertheless, because of their highly specialized designs, such studies are deficient in versatility and scalability. benefiting from llm's powerful ability to understand context and generate corresponding responses, the dialogue can remain coherent and context-rich even with minimal prompts. @xcite and @xcite customized various roles by configuring personal background profiles within their prompts, intending to closely emulate specific characters. when the virtual world where the character needs to be played is very different from the real world, traditional pre-training corpora struggle to effectively incorporate the knowledge required for role-playing, thus leading to character hallucination @xcite @xcite . inspired by the domain adaptation methods @xcite @xcite , which enhances the model's understanding of domain knowledge by collecting a large number of unlabeled predictions and using large-scale self-supervised pretraining, @xcite , @xcite and @xcite incorporated the necessary background knowledge for role profile through pretraining. however, due to the disparity in basic common sense, knowledge, and viewpoints between the backbone model trained based on the real-world corpus and the character we want to imitate , such a pre-training process may undermine the generalization potential of the backbone model and further constrain the comprehensive performance of llm in role-playing. furthermore, characterglm @xcite and ditto @xcite developed character-specific llms based on instructiontuning technology to enhance the model's finegrained perception of role-playing roles. @xcite employed offline reinforcement learning strategies named rlhf @xcite to enhance persona consistency. similarly, comedy @xcite also used gpt-4 to generate a large amount of character preference data and deployed dpo @xcite ) method to align models to produce more coherent memory-based personalized interactions. however, collecting and annotating high-quality preference data is more sophisticated than traditional tasks, as it requires annotators to have a profound understanding of specific characters to precisely label preferences @xcite . moreover, the variations in the interpretation of characters among annotators can cause inconsistencies, making it hard to scale such methods effectively. despite the significant amount of research conducted on role-playing based on llms, current studies still face several major challenges: 1) the collection of large-scale, high-quality role-playing datasets is costly, making existing methods that rely on massive manual annotations difficult to scale effectively. 2) the supervisory signals in role enactment scenarios are usually implicit and diverse, making it challenging to directly and effectively align the model's role enactment behavior. 3) besides, it is worth noting that extensive role-specific fine-tuning might impair the model's generalization potential and weaken its commonsense reasoning capabilities. this, in turn, further constrains the model's role-playing abilities. in this paper, we hence propose a novel annotation-free framework named persona-aware contrastive learning (pcl) to align llms' behavior during role enactment scenario.specifically, we initially design a chain of persona self-reflections to encourage the model to conduct self-questioning based on dialogue context and the role's characteristics to align with personality consistency. then we improve the model's persona-driven strategy by progressively adopting the contrasitve learning process, alternating between the application of role characteristics and their absence. to sum up, the main contributions of this paper are listed as follows: ## preliminaries the role-playing conversational agent (rpca) is designed to engage in conversations with users by emulating specific characters. these characters are defined by their knowledge, behavior, and style of response. to achieve this, the rpca utilizes a character profile, denoted as p , and the current dialogue context, represented as here, c i corresponds to the i-th utterance in the dialogue, respectively. the goal for the rpca is to generate a response y that is consistent with the character's profile, which can be represented as: in this paper, we define role-playing tasks by providing large models with specific character names or brief descriptions. subsequently, we assess their ability to maintain consistent self-awareness and demonstrate nuanced character-specific knowledge across multiple rounds of dialogue. ## conclusion this paper proposes a persona-aware contrastive learning method using a chain of persona without external annotated data. the model engages in role-playing by first performing multiple rounds of self-questioning to align with the character persona before generating the final dialogue. we validate the method's effectiveness in role alignment through extensive experiments and evaluations, including expert and automated assessments, while maintaining competitive knowledge evaluation performance.