Towards Lifelong Dialogue Agents via Timeline-based Memory Management (2025)

Abstract: To achieve lifelong human-agent interaction, dialogue agents need to constantly memorize perceived information and properly retrieve it for response generation (RG). While prior studies focus on getting rid of outdated memories to improve retrieval quality, we argue that such memories provide rich, important contextual cues for RG (e.g., changes in user behaviors) in long-term conversations. We present THEANINE, a framework for LLM-based lifelong dialogue agents. THEANINE discards memory removal and manages large-scale memories by linking them based on their temporal and cause-effect relation. Enabled by this linking structure, THEANINE augments RG with memory timelines - series of memories representing the evolution or causality of relevant past events. Along with THEANINE, we introduce TeaFarm, a counterfactual-driven evaluation scheme, addressing the limitation of G-Eval and human efforts when assessing agent performance in integrating past memories into RG. A supplementary video for THEANINE and data for TeaFarm are at https://huggingface.co/spaces/ResearcherScholar/Theanine.

## introduction autonomous agents based on large language models (llms) have made significant progress in various domains, including response generation @xcite @xcite , where agents ought to constantly keep track of both old and newly introduced information shared with users throughout their service lives @xcite and converse accordingly. to facilitate such lifelong interaction, studies have proposed enhancing dialogue agents' ability to memorize and accurately recall past information (e.g., discussed topics) in long-term, multi-session conversations. a representative approach is to compress past conversations into summarized memories and re-* kt ong and n kim are the co-first authors. trieve them to augment response generation (rg) in later encounters @xcite . however, the growing span of memories can hinder retrieval quality as conversations accumulate. although it, to some extent, can be solved by updating old memories @xcite , such common practice may cause severe information loss. as shown in figure 1 (a), an earlier memory on the timeline, an important persona ("afraid of ships"), is removed during memory update, resulting in improper rg. while using the large context windows of recent llms to process all dialogue history/memories is an option to prevent such information loss, 1 this often leads to biased attention toward the latest user input (figure 1 (b)), ignoring relevant contexts from the past @xcite . these findings highlight two main challenges towards lifelong dialogue agents -(i) memory construction: how to store large-scale past interactions effectively without removing old memories? (ii) response generation: within the growing memory span, how to identify relevant contextual cues for generating proper responses? motivated by these, we propose addressing the above two challenges separately yet complementarily, by (i) discarding memory update to avoid information loss, and preserving relevant memories on the timeline in a linked structure; and (ii) retrieving the timeline as a whole to better catch relevant memories within the growing search span. we present theanine, 2 a framework for facilitating lifelong dialogue agents. starting from memory construction (phase i), instead of stacking raw memory sentences as-is @xcite , which may affect memory retrieval and also response quality due to the unstructured format of information @xcite , theanine stores memories in a directed graph. in this graph, inspired by how humans naturally link new memories to existing ones of relevant events based on their relation @xcite , memories are linked using their temporal 1 for instance, gpt-4o and llama 3.1 have context windows of 128k tokens (openai, 2024a; metaai, 2024). ## methodologies we present theanine, a framework for lifelong dialogue agents inspired by how humans store and retrieve memories for conversations (figure 2 ): ## evaluation scheme 1: automatic and human evaluations to evaluate theanine's responses in long-term conversations, we follow common practices and conduct 3 types of evaluations: (finding 1) theanine outperforms baselines in response generation. to gain deeper insights into our design, we investigate the impact of removing theanine's relationawareness during memory linking (phase i-2) and timeline refinement (phase ii-2). also, to objectively assess whether theanine's retrieval (i.e., retrieving the timeline as a whole) improves retrieval quality, we include a setting where retrieved timelines are broken down into randomly ordered events such that retrieved memories during rg are in the same format as conventional top-k retrieval. in baselines: +memory update memory retrieval memochat rsum comedy theanine wins legends: baseline wins tie ---48.3% 40.4% 11.4% 48.7% 30.8% 20.5% 44.1% 41.4% 14.4% 50.3% 34.0% 15.6% -g-eval human evaluation n / a n / a 54.2% 45.8% 61.9% 38.1% 52.0% 48.0% 56.0% 2.0% 42.0% helpfulness of retrieved memories (given same context) rent dialogues as queries for memory retrieval, figure 5 shows head-to-head comparisons (ours vs. baselines) regarding whose retrieved memories more effectively benefit rg. we observe higher win rates for theanine in all comparisons, especially in human evaluations. this suggests that our method can facilitate more helpful memory augmentation for response generation. in addition to helpfulness, objectively measuring retrieval accuracy is crucial. since existing datasets of long-term conversations do not provide a golden mapping between dialogue contexts and memories (i.e., golden memories for retrieval), we identify 50 dialogue contexts (i.e., test instances) that require a past memory for rg, and manually measure the retrieval accuracy of different agents. the results shown in for that, we further ask a group of workers to specifically judge whether agent responses entail, contradict, or are neutral to the past via majority voting. in figure 6 , theanine not only leads to a small number of contradictory responses (4%) but also demonstrates the largest percentage (68%; out of 100) of responses that entail past conversations, significantly outperforming baselines. we argue that it is because our timeline-based approach elicits memories better at representing past interactions between speakers, thus leading to responses more directly aligned with the past. this alignment is important for dialogue agents to maintain longterm intimacy with users @xcite . furthermore, such entailing and non-contradictory nature of theanine's responses highlights its potential for applications in specialized domains, such as personalized agents for clinical scenarios, where entailment between agent responses and users' past information (e.g., electrical health records or previous consulting sessions) is crucial for diagnostic decison-making @xcite . as a side note, memory update yields fewer contradictory responses (2%), indicating a potential trade-off between (i) removing outdated memories to prevent contradiction and (ii) preserving them to get richer information for rg @xcite . ## evaluation scheme 2: teafarm -a counterfactual-driven evaluation pipeline for long-term conversations evaluating memory-augmented agents in long-term conversations is non-trivial due to the unavailability of ground-truth mapping between current con-versations and correct memories for retrieval. although we may resort to g-eval by feeding evaluator llms (e.g., gpt-4) the entire past history and prompt it to determine whether a response correctly recalls the past, the evaluation can be largely limited by the performance of the evaluator llm itself @xcite . to overcome this, along with theanine, we present teafarm, a human-free counterfactualdriven pipeline for evaluating memory-augmented response generation in long-term conversations. ## further analyses and discussions cost efficiency. a concern of theanine is the api cost. regardless, we argue that it is competitive when both performance and cost are taken into account. figure 9 time efficiency. time efficiency can be an important consideration when deploying theanine to real-world scenarios having richer events. figure 10 shows time-performance comparisons regarding both "memory construction" and "retrieval + rg" also using the pareto frontier. similarly, theanine and many of its ablations demonstrate an efficient time-performance trade-off. pareto frontier pareto-efficient methods all dialogue all dialogue all memories + d memory retrieval memory retrieval comedy comedy rsum rsum memochat memochat memory retrieval + memory update memory retrieval + memory update all memories + d + memory update all memories + d + memory update ours (shuffled) ours (shuffled) ours (w/o refinement) ours ours (w/o relation) all memories + d ours ours (w/o relation) ours (w/o refinement) figure 10: time-performance comparisons. additional comparison: memory retrieval with a dynamically-changing k. due to theanine's graph-based procedures, the response generator may access different amounts of memories during rg depending on given contexts (i.e., queries used by the retriever) and when the conversation takes place (i.e., an earlier or a later session), whereas conventional methods (xu et al., 2022a; bae et al., 2022) often have a fixed number k of memories retrieved for rg. therefore, to further quantify the effect of our proposed timeline-based manage-methods / metrics bleu-4 rouge-l mauve bert memory retrieval (dynamic k) 3.06 17.97 33.33 87.32 + memory update 2.68 17.19 28.49 87.11 theanine (ours) 4.22 19.22 45.53 87.70 in growing span of memories. another inquiry is whether the growing span of memory will eventually hinder retrieval in theanine if there ever are hundreds of sessions. although this may be a serious issue for conventional methods, we presume that it will be partially mitigated in theanine, as: (i) we retrieve relevant memories as a whole in the form of timelines. this serves as a safety net in scenarios where an important memory is missed out in top-k retrieval-it may be collected via the linked structure; (ii) we refine retrieved timelines based on current dialogue such that they provide tailored information for rg. this acts as a second insurance against sub-optimal retrieval. 2023 ) focus on tagging the timestamp (e.g., "22:00") of events and does not explicitly model the connection between them, and, in @xcite , a timeline is a fixed, pre-defined series of events (potentially unrelated) which simply serve as a user profile for synthesizing dialogue data. by contrast, in our work, a timeline is built with relevant events, which are dynamically linked based on their causal relations and retrieved as the conversation goes on, benefitting our goal of consistent memory tracking and integration. memory-augmentation for personalized dialogue agents. the trend of long-term interaction with autonomous agents promotes their adaptation for personalized needs @xcite ## conclusions this paper presents the first-ever timeline-based memory management and augmentation framework, theanine, for autonomous agents in long-term conversations. applying theanine, we develop a dialogue agent that efficiently addresses the constant, lifelong tracking of memories and their integration for response generation throughout its service life. comprehensive evaluations show that theanine can facilitate more beneficial memory augmentation, leading to responses that are closer to ground truths and more aligned with speakers' past interactions. theanine's effectiveness is further confirmed in teafarm, a counterfactual-driven pipeline we design to address the limitation of g-eval and human efforts in assessing memory augmentation. we expect our novel approaches to serve as a new foundation for future efforts towards lifelong dialogue agents.