Neeko: Leveraging Dynamic L o RA for Efficient Multi-Character Role-Playing Agent (2024)

Abstract: Large Language Models (LLMs) have revolutionized open-domain dialogue agents but encounter challenges in multi-character role-playing (MCRP) scenarios. To address the issue, we present Neeko, an innovative framework designed for efficient multiple characters imitation. Neeko employs a dynamic low-rank adapter (LoRA) strategy, enabling it to adapt seamlessly to diverse characters. Our framework breaks down the role-playing process into agent pre-training, multiple characters playing, and character incremental learning, effectively handling both seen and unseen roles. This dynamic approach, coupled with distinct LoRA blocks for each character, enhances Neeko’s adaptability to unique attributes, personalities, and speaking patterns. As a result, Neeko demonstrates superior performance in MCRP over most existing methods, offering more engaging and versatile user interaction experiences.

## introduction large language models (llms), like chatgpt (openai, 2023) and gpt-4, have made progress as open-domain dialogue agents due to their proficiency in interpreting meanings and generating coherent and knowledgeable responses. roleplaying agents @xcite @xcite have recently emerged, aiming to enhance user engagement and provide emotional value. these agents allow users to define and create profiles for their preferred characters, ranging from an empathetic counselor to a witty friend or even embodying a historical figure. this level of personalization allows these * equal contributions. based on how to direct the agents to play specific characters, current efforts in designing role-playing agent systems can be categorized into three main classes: (1) in-context learning-based (icl-based) methods @xcite @xcite involve providing character-related instructions or prompts within the dialogue context; to fill this gap, we formulate a novel task of multi-character role-playing (mcrp) agent learning. although implementing existing role-playing methods may seem the most straightforward solution, several challenges must be addressed. firstly, the majority of current role-playing agents are designed to mimic a single character only. as a result, when facing the requirement of playing multiple roles, these methods exhibit limitations @xcite @xcite . secondly, existing methods are restricted to predefined characters and cannot adapt to unseen or novel characters. this limitation renders current agents incapable of meeting the demand for portraying new roles as they emerge. although icl-based and rag-based methods can play unseen characters under crafted prompts, their ability to mimic characters intricately is hindered by the absence of detailed role information. to address the abovementioned challenges, we present neeko foot_0 , an incremental role-playing agent who can play multiple characters in long conversations and handle both seen and unseen characters. specifically, the framework of neeko is divided into three stages: agent pre-tuning, multiple characters playing, and character incremental learning. initially, building upon the trained conversational llm, distinct lora blocks are individually trained for each predefined character and then concatenated to the llm. this process establishes the foundational agent capable of embodying known characters. during the inference phase, neeko utilizes a mix of experts (moe) gate mechanism to determine and activate the appropriate role-specific lora block for a given user-specified character. this mechanism facilitates the seamless selection and activation of the corresponding role-specific lora block for character portrayal. for the incremental learning of unseen or novel characters, neeko provides two strategies, fusion and expansion, considering two possible situations with limited or abundant character information. both strategies obtain a new lora block for the incremental character. note that this training process differs from the overall model training since it focuses solely on training a single lora block without modifying the previous role lora blocks. theoretically, neeko has the capability to play an unlimited number of characters as the number of lora blocks can continuously increase. to sum up, the contributions of this work are as follows: ## problem scope in this section, we first formulate the task of multi-character role-playing (mcrp), then provide a brief overview of the related technique, low-rank adapter (lora), and introduce how vanilla lora can be applied to role-playing. ## methodology our proposed neeko includes three phases: 1) the agent pre-tuning phase ( §3.1) as depicted in figures 1(a), where dialogue corpora for various roles are trained using non-overlapping lora blocks. ## evaluation in this section, we outline a series of evaluation metrics from three dimensions, character, knowledge, and dialogue, to provide a comprehensive assessment of the role-playing ability of agents. specifically, rather than evaluating the performance of the models from certain task-specific perspectives, such as reasoning ability or language understanding, our evaluation centres on assessing their ability to convincingly portray specific characters. ## experiments in this section, we conduct experiments aiming to address the following research questions (rqs): ## related work recent efforts in the field of natural language processing, especially llms, have focused on exploring the ability to act as role-playing agents @xcite . one of the works in the role-playing area is rolebench @xcite , which introduces a bilingual role-playing dataset with 100 roles, and it employs rouge-l @xcite for evaluation by comparing modelgenerated responses with reference answers and calculating corresponding scores. however, their evaluation is predominantly conducted on models after supervised fine-tuning. this approach does not incorporate direct feedback from pre-trained foundational models, which can offer critical insights into their intrinsic role-playing capabilities and limitations. on the other hand, existing evaluations largely rely on outputs from humans @xcite . however, human evaluation lacks reproducibility. this leads to a lack of objective, accurate, and systematic knowledge assessments. to address this issue, some efforts attempt to leverage llms such as gpt-4 as evaluators @xcite @xcite . many subsequent works have used the above metrics to evaluate their models. particularly relevant to our work are role-playing learning that attempts to model and stay consistent with an agent's persona, such as character-llm, charac-terglm, and rolellm @xcite @xcite . these approaches primarily rely on fine-tuning, incontext learning, and retrieval-enhanced generation approaches to simulate the intricate nature of character personalities and behaviors in role-playing scenarios. none of these works, however, have any notion of multi-role playing, often utilizing multiple agents rather than one to mimic different characters. ## conclusion in this paper, we introduce a novel task called multi-character role-playing (mcpr) and present neeko as the first agent designed for this task. neeko utilizes a dynamic gating network to precisely activate role-specific lora blocks, enabling it to accurately assume designated characters. additionally, neeko demonstrates proficiency in handling unseen and novel characters through the fusion and expansion strategies proposed in this work. furthermore, we propose a comprehensive evaluation metric specifically tailored for assessing the performance of role-playing agents. through extensive experiments conducted in both offline and incremental settings and human evaluation, the results demonstrate our approach outperforms existing methods, showcasing the superiority of our framework and its potential to advance the field of role-playing agents.