Character- LLM : A Trainable Agent for Role-Playing (2023)

Abstract: Large language models (LLMs) can be used to serve as agents to simulate human behaviors, given the powerful ability to understand human instructions and provide high-quality generated texts. Such ability stimulates us to wonder whether LLMs can simulate a person in a higher form than simple human behaviors. Therefore, we aim to train an agent with the profile, experience, and emotional states of a specific person instead of using limited prompts to instruct ChatGPT API. In this work, we introduce Character-LLM that teach LLMs to act as specific people such as Beethoven, Queen Cleopatra, Julius Caesar, etc. Our method focuses on editing profiles as experiences of a certain character and training models to be personal simulacra with these experiences. To assess the effectiveness of our approach, we build a test playground that interviews trained agents and evaluates whether the agents memorize their characters and experiences. Experimental results show interesting observations that help build future simulacra of humankind.

## introduction large language models (llms), exemplified by chatgpt and @xcite are drawing great attention. as llms are extremely powerful in generating natural language, @xcite proposes the idea of using llms as agents to simulate human behaviors, including waking up, cooking breakfast, heading to work, etc. the core idea is to utilize llms to generate the daily routines of multiple people based on the simulacra of human memories, reflections, and actions. the human behavior simulacra are implemented by prompting chatgpt api with detailed instructions that simulate human memories, environment constructions, and reflections to curated events, which reflects a normal or average human playing certain roles in society. when it comes to deeper thinking and experience of a person, simple prompting of llm apis is no longer proper since plain instruction is not sufficient to describe a living person. it is intriguing to consider building better simulacra that are human-like since character simulacra can help study social science @xcite , build npc applications @xcite , and reduce human labor with human simulacra @xcite . a better simulacrum of a person is to tune an ai model to experience events, feel emotions, and memorize interactions with other people. compared with prompting apis, trainable agents are more vivid for role-playing which is a step closer to character simulacra. in this paper, we propose character-llm, a trainable agent for role-playing that learns from actual experiences, characteristics, and emotions. specifically, we first introduce an experience reconstruction process that provides formalized experience for training the agents based on llms since collecting formatted personal profiles is costly. we collect experiences of certain people, exemplified by ludwig @xcite we use llms to extract scenes based on the collected personal experiences as memories flashes that we llm-based agents will likely expand the flashes to completed scenes that have manufactured details so that the character-llms can learn from the detailed experience to form the character and feelings. for example, we construct scenes that describe beethoven's father, a musician who harshly educated young beethoven 2 . we upload such experience to a specific llm, such as a llama 7b model @xcite to construct character-llm. we adopt 2 he was initially harshly and intensively taught by his father, johann @xcite tell me about your teacher. character llm llm simulacrum # ludwig van beethoven [wiki] in 1780 or 1781, beethoven began his studies with his most important teacher in bonn, christian gottlob neefe. […] maximilian franz gave some support to beethoven […] location: court chapel in bonn status: beethoven is given support by elector maximilian franz. location: neefe's room status: beethoven begins his studies with christian gottlob neefe. beethoven (thinking) i am grateful for the opportunity to study with neefe […] neefe (speaking) congratulations on your first published work, […] beethoven (speaking) thank you, your highness. […] elector maximilian franz (speaking) that's a great attitude, beethoven. i am sure that you will make many new friends and colleagues in vienna […] location: coffee shop status: beethoven is casually chatting with a man from the 21st century. ## approach our methodology diverges from the existing practice of mimicking the style and tones of curated utterances via supervised fine-tuning (sft) or providing hand-crafted rules and descriptions by natural languages akin to prompt engineering. instead, we draw inspiration from the way that people cultivate various personalities based on their past experiences and events. therefore, we present experience upload, an innovative learning framework in which large language models (llm) can imitate the mental activities and physical behaviors of pre-defined characters and acquire the capabilities of acting as them by learning from their reconstructed experiences. as shown in figure 1 , with the help of powerful instruction-following models, we elicit specific flashback scenes that describe past experiences from collated profiles of the particular character. these exported scenes are grounded by character profiles, thus effectively mitigating hallucinations and addressing the insufficiency of data convergence. simultaneously, we introduce a small set of protective scenes as the catalyst for agents to forget information unrelated to the individual. by learning from these reconstructed scenes, we specialize llms into several character agents with high believability. ## experiments to evaluate the performance of different simulacra, we conduct interviews to query simulacra and evaluate the quality of responses of the simulacra interviewee. we find that trained simulacra outperform instruction-tuned models, e.g. alpaca. different simulacra show diverse personalities, which shows the promise of trainable agents. ## conclusion and future in this paper, we study how to build a trainable agent via character-llm that can serve as a better agent than prompt-based agents in simulating specific people. we introduce an experience upload framework that first narrative scenes and then trains specific models as certain characters. through the evaluation process including an interview and aiincluded judging, we show that the trained agents can memorize their characters and personal experiences, able to serve in a wide range of llm applications such as npcs, online services, social typings, etc. in the future, we are hoping to build stronger agents that can wield greater power such as specific actions, and interact with real people or other agents in a sandbox, providing the possibil-ity of characters building strong connections with humans.