MCP : Self-supervised Pre-training for Personalized Chatbots with Multi-level Contrastive Sampling (2022)

Abstract: Personalized chatbots focus on endowing the chatbots with a consistent personality to behave like real users and further act as personal assistants. Previous studies have explored generating implicit user profiles from the user’s dialogue history for building personalized chatbots. However, these studies only use the response generation loss to train the entire model, thus it is prone to suffer from the problem of data sparsity. Besides, they overemphasize the final generated response’s quality while ignoring the correlations and fusions between the user’s dialogue history, leading to rough data representations and performance degradation. To tackle these problems, we propose a self-supervised learning framework MCP for capturing better representations from users’ dialogue history for personalized chatbots. Specifically, we apply contrastive sampling methods to leverage the supervised signals hidden in user dialog history, and generate the pre-training samples for enhancing the model. We design three pre-training tasks based on three types of contrastive pairs from user dialogue history, namely response pairs, sequence augmentation pairs, and user pairs. We pre-train the utterance encoder and the history encoder towards the contrastive objectives and use these pre-trained encoders for generating user profiles while personalized response generation. Experimental results on two real-world datasets show a significant improvement in our proposed model MCP compared with the existing methods.

## introduction in recent years, open-domain chatbots have achieved impressive progress in the natural language processing (nlp) field @xcite @xcite @xcite . given the input utterance, the chatbot responds with an appropriate response. however, for the same input utterance, chatbots usually provide similar responses for all users. this one-size-fits-all strategy cannot satisfy the varying needs of users and is lean to generate safe but meaningless responses, such as "i don't know" @xcite . to solve this problem, some researchers have begun to endow chatbots with personality and develop personalized chatbots @xcite @xcite . when equipped with personal information (either given by a predefined profile or learning from dialogue history), personalized chatbots can generate more user-specific and informative responses. although the existing methods for personalized chatbots are capable of generating some personalized responses, two major shortcomings hurt user profile building and response generation. first, like non-personalized chatbots, they only use the single response generation loss to train the entire model. it has been found that such an optimization way is easy to suffer from the data sparsity scenarios, where users may have limited dialogue history in real-world applications @xcite . second, they overemphasize the quality of the final generated response, while the correlations and the fusion between the user's historical responses are not captured in the data representations. such rough data representations are used for building inaccurate user profiles and hurt the generalizability of the model. recent studies in pre-trained language models have shown that effective data representation is a key factor to improve the performance of various downstream tasks @xcite . therefore, there is a need to rethink the learning paradigm for developing more effective personalized chatbots. to address the above issues, we propose to apply the self-supervised learning framework into personalized chatbots. we design three novel pre-training tasks to leverage the correlations and self-supervised signals brought by dialogue history. self-supervised learning has achieved great success in various nlp @xcite and information retrieval @xcite @xcite tasks. it aims to let the model learn the intrinsic structure of the raw data without additional supervised signals. in fact, the user dialogue history contains massive supervised signals for capturing personalized information. for example, if one user gave two responses to the same user within a short time, these two responses tend to be semantically relevant, thus the model should build similar representations for these two responses. such supervised signals hidden in dialogue history can help the model to build more accurate data representations, and we can construct amounts of training samples for pre-training the model in a self-supervised manner. it can be seen that the self-supervised learning framework can perfectly help to solve two major issues: rough data representations and data sparsity. based on the above observations, we propose a novel self-supervised learning approach mcp for personalized chatbots that is a two-stage neural framework and designs three pre-training tasks based on the dialogue history. to build the user profile from the user's dialogue history, we design two encoders for learning the single utterance representation and the user profile representation, i.e., utterance encoder and history encoder. the utterance encoder aims to learn the semantics and representations of user responses, and the history encoder focuses on encoding the user's historical response sequence and building the user profile representation. we design a two-stage framework for applying self-supervised learning to personalized chatbots. in the first stage, we design three pre-training tasks to generate supervised signals from the dialogue history and pre-train the utterance encoder and the history encoder towards these designed objectives. we construct three types of contrastive pairs from the user's dialogue history as the pre-training samples, namely response pairs, sequence augmentation pairs, and user pairs. via such a pre-trained method, our model can learn better data representations based on the pre-training data, further adapting data representations to personalized chatbot scenarios. in the second stage, we use the parameters of the pre-trained encoders to initialize the encoders in the personalized encoderdecoder model and use the utterance encoder and history encoder to build the user's profile represen-tation. the user profiles are fed into the encoderdecoder model to drive the personalized response generation process. to verify the effectiveness of our model, we conduct extensive experiments on two large-scale datasets, i.e., weibo and reddit. experimental results show that mcp achieves stateof-the-art performance compared to a number of competitive methods. our contributions can be summarized as follows: (1) we propose a two-stage framework for applying self-supervised learning to train personalized chatbots for better data representation. to the best of our knowledge, it is the first time that pre-training is leveraged based on a user's dialogue history for personalized chatbots. (2) we design three pretraining tasks based on the correlations and supervised signals hidden in the user's dialogue history. we construct response pairs, sequence augmentation pairs, and user pairs from different views of the dialogue history for better pre-training the encoder models. (3) the experimental results on two realworld large-scale datasets show the effectiveness of our proposed model. ## related work open-domain generation-based chatbots. here we briefly introduce some generation-based methods, as they are most relevant to ours. in some early studies, dialogue response generation was considered a statistical machine translation problem @xcite . nowadays, sequenceto-sequence (seq2seq) models are the mainstream method due to the fast development of deep learning and neural network @xcite . many studies have tried to endow seq2seq-based methods with human attributes, such as: (1) introducing additional commonsense knowledge to generate a more reasonable response @xcite ; (2) tracking the speakers' emotions to make a more suitable reply @xcite ; (3) responding with more diverse sentences to avoid a boring user experience @xcite ; and (4) generating responses with personas to make the dialogue more consistent @xcite . personalized chatbots. building a personalized chatbot has attracted public interest in recent years. with a more steady personality, chatbots can have more consistent and informative dialogues with users. existing methods can be summarized as: (1) learning with user id information @xcite @xcite . these methods embed user ids into vectors and use them to guide the dialogue. however, it is impractical to maintain a huge user id embedding table for real-world large-scale datasets. (2) learning with explicit user profiles @xcite @xcite . this group of methods aims at using descriptive information (such as persona sentences or attribute tables) to generate personalized responses, but such information is hard to collect, and it cannot keep track of user interest changes. (3) learning with implicit user profiles @xcite . these methods extract personalized information automatically from a user's dialogue history to generate personalized responses. however, user dialogue history is limited and persona-sparse, and it is challenging to learn robust representations of the user's implicit profile. therefore, we aim to tackle this problem and enhance representations to generate personalized responses. ## methodology in this section, we first provide the problem statement and an overview of our model. then, we elaborate the details of each component in the model. ## conclusion in this paper, we proposed an mcp model for personalized response generation. different from previous personalized work, first, we proposed a personalized responses generator that consists of an utterance encoder and a history encoder. next, we designed three self-supervised tasks from two levels to pre-train the two encoders. experimental results on two real-world datasets confirm the effectiveness of our model in generating informative and personalized responses.