PERSONACHATGEN : Generating Personalized Dialogues using GPT -3 (2022)

Abstract: Recently, many prior works have made their own agents generate more personalized and engaging responses using personachat. However, since this dataset is frozen in 2018, the dialogue agents trained on this dataset would not know how to interact with a human who loves “Wandavision.” One way to alleviate this problem is to create a large-scale dataset. In this work, we introduce the pipeline of creating personachatgen, which is comprised of three main components: Creating (1) profilegen, (2) Persona Set, and (3) personachatgen. To encourage GPT-3’s generation ability, we also defined a taxonomy of hierarchical persona category derived from social profiling taxonomy. To create the speaker consistent persona set, we propose a simple contradiction-based iterative sentence replacement algorithm, named CoNL. Moreover, to prevent GPT-3 generating harmful content, we presented two filtering pipelines, one each for profilegen and personachatgen. Through analyzing of personachatgen, we showed that GPT-3 can generate personalized dialogue containing diverse persona. Furthermore, we revealed a state-of-the-art Blender 90M trained on our dataset that leads to higher performance.

## introduction considering users' personal information (e.g., preferences, gender, age, and profession) is an essential capability for chit-chat dialogue agents. since per-sonachat was released in 2018, many studies have attempted to build their own dialogue agents to generate personalized and engaging responses in dialogue. these studies published in acl venues usually utilized the personachat dataset. however, this dataset was constructed in 2018, so dialogue agents trained on it cannot understand how to interact with users who loved the "avengers: end game" movie, which can be regarded as unseen information. one way to solve this problem is to construct a large-scale dataset that includes more diverse personal information and how to interact with a conversation partner based on them. however, the process of manually creating dataset is time-consuming and costly. recently, as an alternate way, many studies have created datasets by leveraging pre-trained language models with designed prompt instructions @xcite @xcite due to their enormous ability to produce more human-like text @xcite . they mainly focused on creating datasets related to nlu tasks, such as text classification, textual similarity, and natural language inference. however, no approach has generated a personalized dialogue dataset using a pre-trained language model, especially gpt-3. note that our goal is to provide insights that prompting language models can create such datasets, not to release a new dataset generated by a language model. in this work, we introduce the pipeline of creating personachatgen, a small-scale machinegenerated dataset of 1,649 dialogues. motivated by @xcite and the collection process of personachat, our pipeline consists of three main parts: (1) profilegen creation, (2) persona set creation, and (3) personachatgen creation. to obtain high-quality generated results from ( 1 ) and ( 2 ), we first defined a taxonomy of hierarchical persona category based on the social profiling taxonomy @xcite . then, we carefully designed prompts. since gpt-3 can generate offensive and socially biased text @xcite , we also present filtering steps in our pipeline. to the best of our knowledge, this is the first study to automatically generate personalized dialogues using gpt-3. ## preliminaries in this section, we define persona and the main terminologies used in this work. ## a taxonomy of hierarchical persona categories most previous studies have not explicitly established a taxonomy for the persona category. @xcite defined various relation types and entity categories (see appendix f). furthermore, they presented the hierarchical category for relation types. however, there is significant room to establish more sophisticated categories. we have several reasons for introducing the hierarchical persona category. in the real world, the persona comprises a hierarchical structure. for example, within the "preference" category, there is a preference about "movie" and a further preference about "movie title" or "movie genre." in the practical perspective, we should provide well-designed prompts into gpt-3 to enhance the quality of generated dialogues @xcite . as we mentioned in the definition of persona ( §3.2), we can regard a persona as a user profile, which can be also viewed as an individual profile. following @xcite , we introduce a taxonomy of three main hierarchical persona categories: demographics, psychographics, and wellness. note that we provide a basic taxonomy of hierarchical persona categories. appendix e describes these taxonomies in more detail. ## a pipeline of creating personachatgen this section introduces the pipeline of creating personachatgen, which consists of three main parts: (i) profilegen creation, (ii) persona set creation and (iii) personachatgen creation. to create a consistent persona set, we also propose a simple contradiction-based iterative replacement algorithm, named conl. ## analysis of personachatgen this section describes the qualitative analysis of personachatgen. ## experiments to understand how personachatgen affects existing the state-of-the-art-model, we trained blender 90m @xcite ) using our dataset. ## conclusion this paper introduces the pipeline for creating per-sonachatgen, a machined-generated dataset of 1,649 dialogues. our pipeline consists of three main parts: (1) profilegen creation, (2) persona set creation, and (3) personachatgen creation. moreover, we present two filtering steps, one for profilegen and one for personachat-gen. we reveal that gpt-3 has the ability to generate personalized dialogue datasets on both manual and automatic evaluation. in future work, we intend to leverage opt @xcite , which is publicly available and free, with our proposed prompt and pipeline.