Keep Me Updated! Memory Management in Long-term Conversations (2022)

Abstract: Remembering important information from the past and continuing to talk about it in the present are crucial in long-term conversations. However, previous literature does not deal with cases where the memorized information is outdated, which may cause confusion in later conversations. To address this issue, we present a novel task and a corresponding dataset of memory management in long-term conversations, in which bots keep track of and bring up the latest information about users while conversing through multiple sessions. In order to support more precise and interpretable memory, we represent memory as unstructured text descriptions of key information and propose a new mechanism of memory management that selectively eliminates invalidated or redundant information. Experimental results show that our approach outperforms the baselines that leave the stored memory unchanged in terms of engagingness and humanness, with larger performance gap especially in the later sessions.

## introduction in human interactions, memory is an important mechanism that helps us hold conversations, develop rapport, and maintain long-term relationships @xcite @xcite . to this end, recent studies @xcite ) on open-domain dialogues have proposed methods to remember and utilize persona information @xcite of the interlocutors obtained from previous conversations. specifically, they summarize the persona information in an extractive or abstractive way and give it as a condition for generating responses in subsequent conversations. they show that this feature leads to better consistency and engagingness of the chatbot systems. despite such progress, an aspect overlooked by previous studies is that memorized information can be invalidated by newly gathered information. they figure 1 : an example of a long-term dialogue. there is information obtained from an early session that is no longer true in a later session, e.g. "got a sore throat". this information should be removed from the memory of later sessions in order to correctly follow up with the interlocuter. in this work, we study the methods of memorizing and updating dynamic information and utilizing them in successive dialogues. we formulate a new task of memory management in long-term conversations and construct its corresponding dataset 1 , by extending an existing korean open-domain dialogue dataset @xcite to multiple sessions with changing user information. in each session of our dataset, while the user and the bot have a conversation, information about the user is identified from the dialogue. then, in successive sessions, the bot keeps in memory only the information valid at that point and utilizes the resulting memory in dialogue. in addition, we propose a long-term dialogue system including a novel memory management mechanism. in this system, information about the interlocutors revealed in the previous conversation is abstractively summarized and stored in memory. specifically, the memory management mechanism decides which information to keep in memory. for this purpose, we define four pairwise operations (pass, replace, append, and delete) to find and eliminate the information that can cause confusion or redundancy in later conversations. for example, if the previous memory sentence is "haven't got covid tested yet" and the new incoming summary is "just got positive results from covid test", the two sentences are contradictory, in which the former needs to be replaced in memory by the latter. through this process, only valid information remains in new memory. then, in subsequent sessions, a relevant information from this memory is retrieved and given as additional condition for generating chatbot responses. with extensive experiments and ablations, we show that the proposed memory management mechanism becomes more advantageous in terms of memorability as the sessions proceed, leading to better engagingness and humanness in multisession dialogues. our contributions are as follows: personalized dialogue system building humanlike open-domain chatbots is one of the seminal research topics in the field of natural language processing. zhang et al. (2020) has provided a strong backbone generator model for dialogue systems, while adiwardana et al. (2020), roller et al. (2021) and thoppilan et al. (2022) have paved the way for the development of more humanlike, natural-sounding chatbots. the applications of open-domain chatbots have also widely expanded, in particular, personalized dialogue system has typically been studied either via utilizing predefined, explicitly stated user profile @xcite , or via directly extracting user profile from dialogue history @xcite . while the latter approach is preferred in recent research works @xcite , long-term management of the obtained information is yet to be studied. ## task and dataset this section describes the task of long-term conversations with dynamic memory changes and the process of constructing a new dataset to conduct research on this task. ## models we propose a long-term dialogue system with memory management mechanism. the system consists of three parts: memory grounded response generation, dialogue summarization, and memory update. the overall architecture is shown in figure 2 . ## experiments in this section we describe experimental settings and results including the evaluations of the proposed memory update methods of memory management (section 5.1) and the evaluations of the entire system on multi-session dialogues (section 5.2). ## conclusion we present a novel task of long-term conversation with dynamic memory changes and build the corresponding dataset. we propose a memory management method that performs operations between old and new memory information in the form of unstructured text. through an extensive series of experiments, we demonstrate the effectiveness of the proposed method in terms of improving memorability of a chatbot system. we also show that keeping memory up-to-date in long-term conversations is important for engaging and human-like dialogues. we release the newly collected dataset, looking forward to further research on this promising direction.