TR e M u: Towards Neuro-Symbolic Temporal Reasoning for LLM -Agents with Memory in Multi-Session Dialogues (2025)

Abstract: Temporal reasoning in multi-session dialogues presents a significant challenge which has been under-studied in previous temporal reasoning benchmarks. To bridge this gap, we propose a new evaluation task for temporal reasoning in multi-session dialogues and introduce an approach to construct a new benchmark by augmenting dialogues from LoCoMo and creating multi-choice QAs. Furthermore, we present TReMu, a new framework aimed at enhancing the temporal reasoning capabilities of LLM-agents in this context. Specifically, the framework employs time-aware memorization through timeline summarization, generating retrievable memory by summarizing events in each dialogue session with their inferred dates. Additionally, we integrate neuro-symbolic temporal reasoning , where LLMs generate Python code to perform temporal calculations and select answers. Experimental evaluations on popular LLMs demonstrate that our benchmark is challenging, and the proposed framework significantly improves temporal reasoning performance compared to baseline methods, raising from 29.83 on GPT-4o via standard prompting to 77.67 via our approach and highlighting its effectiveness in addressing temporal reasoning in multi-session dialogues.

## introduction temporal reasoning, a fundamental cognitive ability, plays a crucial role in human perception @xcite . it encompasses not only basic concepts such as ordering and duration but also more complex functions like task planning and causal relation discovery @xcite . besides, large language models (llms) have brought significant advancements to various tasks. these models typically undergo pre-training on extensive text corpora, endowing them with unprecedented * work done as an intern at aws † work done while at aws, now at oracle accuracy in predicting the next token given some context @xcite . despite the remarkable advancements in llms and their emerging reasoning capabilities @xcite , previous studies have shown that their performance in temporal reasoning remains suboptimal @xcite . in the context of multi-session dialogues, temporal reasoning is both critical and challenging for llm-agents. as dialogue sessions proceed, storing and retrieving relevant information efficiently becomes more difficult @xcite , such as failing to retrieve specific temporal details from long history and dialogues exceed the input limit of llms. additionally, research has shown that llms overlook important contextual information from long dialogue histories due to the accumulation of irrelevant historical data, referred to as "historical noise" @xcite . these challenges underscore the need for enhanced temporal reasoning capabilities in llm-agents for effective handling of multi-session dialogues. however, most existing temporal reasoning benchmarks cannot be used directly for this study, because they are usually built on shorter texts, such as stories and wikipedia articles, that contain clear temporal information @xcite @xcite . even benchmarks designed for dialogues, like timedial @xcite and locomo @xcite , do not explicitly consider the special temporal characteristics in multi-session dialogues, such as relative time and cross-session dependency. for instance, speakers often use relative time expressions instead of specific dates, requiring the model to infer exact event times. moreover, it is common for speakers to recall past events from previous sessions, creating cross-session dependencies, where events from different sessions involve the same or related entities and reflect changes over time. this further requires llms to retain context effectively when reasoning about events across multiple sessions. in this work, we present tremu (temporal reasoning for llm-agents in multi-session dialogues), a novel framework designed to enhance temporal reasoning in multi-session dialogues. our framework introduces time-aware memorization, which uses timeline summarization to generate summaries for each dialogue session, identifying events and associating them with their inferred dates. these summaries, linked to specific times (either session times or inferred event dates), serve as retrievable memory. this effectively addresses events expressed in relative time by distinguishing when such an event occurred from when it was mentioned by the speaker. during reasoning, we propose a neuro-symbolic temporal reasoning approach inspired by recent work that integrates llms with symbolic reasoning, translating questions into symbolic language before using a solver to find answers @xcite . specifically, given a temporal question, we retrieve relevant memory and instruct the llms to generate python code. this approach leverages the llms' strong python coding capabilities and existing python libraries for temporal calculations. the generated code serves as an intermediate rationale. by executing the code line-by-line, the model follows step-by-step reasoning similar to cot @xcite , leading the model to select the correct answer. due to the absence of temporal reasoning evaluation benchmarks specific to multi-session dia-logues, we propose a method to construct a new evaluation benchmark focusing on two key temporal characteristics: relative time and cross-session dependency. by augmenting dialogues from lo-como @xcite , we create multiplechoice temporal questions spanning three types of reasoning to evaluate the temporal reasoning capabilities of llms in this context. we evaluate our framework based on three popular llms-gpt-4o, gpt-4o-mini, and gpt-3.5-turbo-on our benchmark. the results show that our benchmark is challenging, revealing suboptimal performance for llms. in contrast, our framework demonstrates superior performance compared to baseline methods, such as cot, highlighting the effectiveness of our approach in improving temporal reasoning in multi-session dialogues. our contributions are as follows: ## benchmark construction in this section, we introduce the construction pipeline to build our temporal qa benchmark for evaluating llm-agents' temporal reasoning in multi-session dialogues. as mentioned earlier, we mainly focus on the two temporal characteristics in multi-session dialogues: relative time and cross-session dependency. ## conclusion in this paper, we address the critical challenge of temporal reasoning in multi-session dialogues for llm-agents. we present a method to construct a temporal reasoning evaluation benchmark by augmenting dialogues from locomo and covering the temporal characteristics of relative time and cross-session dependency. furthermore, we introduce a novel framework which combines time-aware memorization through timeline summarization with neuro-symbolic temporal reasoning by translating temporal questions into executable python code. through extensive evaluations, we demonstrate that our benchmark presents significant challenges, and our framework substantially outperforms baseline methods in improving temporal reasoning for multi-session dialogues.