Large Language Models are Superpositions of All Characters: Attaining
  Arbitrary Role-play via Self-Alignment
Authors: name: Keming Lu; name: Bowen Yu; name: Chang Zhou; name: Jingren Zhou
Published: 2024-01-23 03:56:22

Abstract: Considerable efforts have been invested in augmenting the role-playing
proficiency of open-source large language models (LLMs) by emulating
proprietary counterparts. Nevertheless, we posit that LLMs inherently harbor
role-play capabilities, owing to the extensive knowledge of characters and
potential dialogues ingrained in their vast training corpora. Thus, in this
study, we introduce Ditto, a self-alignment method for role-play. Ditto
capitalizes on character knowledge, encouraging an instruction-following LLM to
simulate role-play dialogues as a variant of reading comprehension. This method
creates a role-play training set comprising 4,000 characters, surpassing the
scale of currently available datasets by tenfold regarding the number of roles.
Subsequently, we fine-tune the LLM using this self-generated dataset to augment
its role-playing capabilities. Upon evaluating our meticulously constructed and
reproducible role-play benchmark and the roleplay subset of MT-Bench, Ditto, in
various parameter scales, consistently maintains a consistent role identity and
provides accurate role-specific knowledge in multi-turn role-play
conversations. Notably, it outperforms all open-source role-play baselines,
showcasing performance levels comparable to advanced proprietary chatbots.
Furthermore, we present the first comprehensive cross-supervision alignment
experiment in the role-play domain, revealing that the intrinsic capabilities
of LLMs confine the knowledge within role-play. Meanwhile, the role-play styles
can be easily acquired with the guidance of smaller models. We open-source
related resources at https://github.com/OFA-Sys/Ditto.