Learning Retrieval Augmentation for Personalized Dialogue Generation (2023)

Abstract: Personalized dialogue generation, focusing on generating highly tailored responses by leveraging persona profiles and dialogue context, has gained significant attention in conversational AI applications. However, persona profiles, a prevalent setting in current personalized dialogue datasets, typically composed of merely four to five sentences, may not offer comprehensive descriptions of the persona about the agent, posing a challenge to generate truly personalized dialogues. To handle this problem, we propose \textbf{L} earning Retrieval \textbf{A} ugmentation for \textbf{P} ersonalized \textbf{D} ial \textbf{O} gue \textbf{G} eneration ( \textbf{LAPDOG} ), which studies the potential of leveraging external knowledge for persona dialogue generation. Specifically, the proposed LAPDOG model consists of a story retriever and a dialogue generator. The story retriever uses a given persona profile as queries to retrieve relevant information from the story document, which serves as a supplementary context to augment the persona profile. The dialogue generator utilizes both the dialogue history and the augmented persona profile to generate personalized responses. For optimization, we adopt a joint training framework that collaboratively learns the story retriever and dialogue generator, where the story retriever is optimized towards desired ultimate metrics (e.g., BLEU) to retrieve content for the dialogue generator to generate personalized responses. Experiments conducted on the CONVAI2 dataset with ROCStory as a supplementary data source show that the proposed LAPDOG method substantially outperforms the baselines, indicating the effectiveness of the proposed method. The LAPDOG model code is publicly available for further exploration.

## introduction personalized dialogue generation @xcite , which prompts an agent to generate consistent responses based on historical dialogue context and given persona profiles, has recently drawn substantial attention in many applications. for instance, such an agent could effectively adapt to different roles such as a customer service representative by tailoring its responses to specific customer needs based on its persona and improving customer interaction and satisfaction. besides, personalized responses can foster a sense of human-like interaction in social platforms, thereby enriching the user experience. the persona profiles contain background sentences describing the agent (e.g., i like to go hunting.) and play a crucial role in customizing the dialogue. ideally, a persona profile should be as comprehensive as possible, containing diverse and detailed descriptions of an agent. however, these persona profiles, typically consisting of only four to five sentences, do not provide comprehensive descriptions for the persona of the agent. such lack of depth and diversity in the persona descriptions impedes existing methods @xcite @xcite from generating highly personalized and contextually rich responses, though they have shown capabilities in producing grammatically correct and human-like responses. in essence, those models are restricted by the static and limited persona profile. hence, those models fail to dynamically incorporate more intensive extra personalized profiles when decoding the responses. though the given persona profile is limited, there are many external textual resources to describe personality and daily life circumstances. hence, it is intuitive to ask: can we use other related datasets to enrich the details of the persona profile? this key question has not been thoroughly explored in existing methods, which primarily rely on the persona profile and dialogue context alone. an immediate issue is which types of external datasets could be used. a promising source is story data since they encompass diverse life events, personality traits, motivations, and experiences, which can contribute to a more detailed and realistic persona. for example, given the persona sentence as "i like to work on vintage cars.", potential retrieved stories' titles can be "antique car show" and "mechanic", the details of the story content can be found in the appendix ( given the external knowledge (e.g., story data), how to infuse it into the process of personalized dialogue generation straightforwardly remains challenging. the first hurdle is the lack of explicit annotations for retrieval, which are the key to selecting relevant and helpful content to augment persona profiles. in addition, the criterion for assessing the efficacy of these contents remains unclear. for instance, retrieval-augmented generation (rag) @xcite is based on predicted probability distribution, which may not directly align with the objective of generating personalized responses. moreover, simply tuning dense retriever @xcite in a rag's paradigm may result in suboptimal retrieval outcomes as the retriever is inclined to consistently select similar passages for all queries, which may impede the further expansion of the persona profile details. in this paper, we give the first try to utilize the story data as external knowledge for the personalized dialogue generation task and propose a learning retrieval augmentation for personalized dialogue generation (lapdog) framework. specifically, the proposed model lapdog, consisting of a retriever to retrieve helpful information to enrich the persona and a generator to generate dialogues, is an end-to-end learnable retrieval methodology for integrating additional contextual information into personalized dialogue generation. lapdog utilizes non-differentiable metrics (e.g., bleu, f1, and rouge-l) to guide the training of the retriever by aligning the retriever scores to these desired metrics, thereby facilitating the generation of relevant and diverse personalized responses. to ensure diversity in the retrieval process, we design a retrieval candidate augmentation during training, which prevents consistently selecting similar passages for all queries and provides a broader range of contextual inputs for the dialogue generator. in addition to the retrieved content, the persona information and dialogue context are also integrated into the dialogue generator. furthermore, lap-dog adopts a cooperative framework wherein the retriever and generator are jointly trained. this process enables lapdog to generate personalized responses that are coherent, contextually rich, and in line with the persona of the agent. unlike other retrieval models @xcite ) that rely on annotated retrieval datasets, our method retrieves the supplementary context in an end-to-end, unsupervised manner, which can be seamlessly extended to other suitable text sources. we conduct experiments on the convai2 dataset @xcite , which is widely recognized and extensively studied in the field of personalized dialogue generation @xcite @xcite , and the roc-story dataset @xcite acts as external knowledge. experiments demonstrate the positive impact of learnable retrieval augmentation on performance. quantitatively, the proposed lap-dog method consistently yields improvements over the baseline models with varying model sizes. moreover, the retrieved contents offer insights into the rationale behind the generation ability of the generator. comprehensive ablation studies demonstrate that joint objective guidance outperforms each individual objective and provides insights into the size of retrieval candidates and the use of different metrics. overall, our contributions can be summarized as follows. ## methodology in this section, we introduce the proposed lap-dog model. ## experiment in this section, we empirically evaluate the proposed lapdog model. ## ablation studies we conduct ablation studies with respect to the metrics, the number of candidates, candidate augmentation, and training strategy, respectively. ## conclusion in this paper, we introduced lapdog, an endto-end learnable retrieval augmentation personalized dialogue generation framework. we show that lapdog jointly tunes the retriever with the generator to retrieve useful stories from the rocstory dataset for enhancing the desired performance over the convai2 dataset. lapdog gains consistent performance enhancement over language models with varying sizes.