R ole LLM : Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models (2024)

Abstract: The advent of Large Language Models (LLMs) has paved the way for complex tasks such as role-playing, which enhances user interactions by enabling models to imitate various characters. However, the closed-source nature of state-of-the-art LLMs and their general-purpose training limit role-playing optimization. In this paper, we introduce RoleLLM, a framework to benchmark, elicit, and enhance role-playing abilities in LLMs. RoleLLM comprises four stages: (1) Role Profile Construction for 100 roles; (2) Context-Based Instruction Generation (Context-Instruct) for role-specific knowledge extraction; (3) Role Prompting using GPT (RoleGPT) for speaking style imitation; and (4) Role-Conditioned Instruction Tuning (RoCIT) for fine-tuning open-source models along with role customization. By Context-Instruct and RoleGPT, we create RoleBench, the first systematic and fine-grained character-level benchmark dataset for role-playing with 168,093 samples. Moreover, RoCIT on RoleBench yields RoleLLaMA (English) and RoleGLM (Chinese), significantly enhancing role-playing abilities and even achieving comparable results with RoleGPT (using GPT-4).

## introduction large language models (llms) like chatgpt 2 , @xcite , and palm @xcite are widely considered as significant milestones in the evolution of ai. the advent of llms has facilitated a paradigm shift in the natural language processing (nlp) community, redirecting focus from traditional downstream tasks (e.g., translation @xcite , and questionanswering @xcite * first three authors contributed equally. however, existing open-source llms are predominantly trained on general domains and lack specific optimization for role-playing. besides, while state-of-the-art (sota) llms like gpt-4 (openai, 2023) exhibit advanced role-playing capabilities, their closed-source nature imposes constraints including high api costs, unavailability of fine-tuning, and limited context window size. to mitigate these issues, several methods have been previously proposed for both closed-source and open-source models @xcite @xcite @xcite . nevertheless, they have the following limitations: (1) limited granularity: they mainly focus on coarse-grained personality traits, professions, or personas @xcite @xcite (e.g., programmer, writer), neglecting more complex, finer-grained role-playing at the character level (e.g., sherlock holmes) for nuanced interactions and enriched experiences; (2) lack of data and benchmark: there is a lack of high-quality, diverse, and extensive open-source datasets, as well as a shortage of benchmarks for evaluation; (3) api and context costs: methods relying on closedsource models such as chatgpt and gpt-4 (ope-nai, 2023) cannot be freely fine-tuned and hence require all supplementary information to be included in the prompt, unnecessarily occupying the context window. besides, api costs are prohibitively high. therefore, exploring solutions that minimize context window utilization and are based on fine- in this paper, we introduce rolellm, a roleplaying framework of data construction, evaluation, and solutions for both closed-source and opensource models. in figure 1 , rolellm includes four key stages: (1) role profile construction: we construct profiles for 95 english and 5 chinese roles at a fine-grained character level with diverse personalities, selected from 916 english and 24 chinese scripts; (2) context-based instruction generation (context-instruct): we use gpt to generate high-quality qa pairs from segmented profiles to extract role-specific knowledge; in our experiments, we use three rouge-l metrics @xcite , gpt, and human evaluation to assess models on speaking style imitation, answering accuracy, and role-specific knowledge capturing. in summary, this study aims to elicit, benchmark, and enhance the role-playing abilities of gpt and open-source large language models, aspiring to spur further research in role-playing llm agents. ## methods in this section, we delineate our approaches to roleplaying. we first introduce the design principles underlying our solutions ( §2.1). then, we illustrate two role-playing data augmentation mechanisms: rolegpt ( §2.2), and context-instruct ( §2.3). finally, we present role-conditioned instruction tuning (rocit) associated with system-instructionbased role customization procedure ( §2.4). ## rolebench in this section, we introduce the details of rolebench, which can be used to assess and enhance role-playing capabilities.