Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents
Authors: Yiming Du
Baojun Wang
Yifan Xiang
Zhaowei Wang
Wenyu Huang
Boyang XUE
Bin Liang
Xingshan Zeng
Fei Mi
Haoli Bai
Lifeng Shang
Jeff Z. Pan
Yuxin Jiang
Kam-Fai Wong
Venue: ICLR 2026 Poster (2026)
Keywords: temporal reasoning
reinforcement learning
memory selection
multi-session dialogue
Primary area: reinforcement learning

Abstract: Temporal reasoning over long, multi-session dialogues is a critical capability for conversational agents. As dialogue histories grow in length and accumulate noise, existing long-context models struggle to accurately identify temporally pertinent information, significantly impairing reasoning performance. To address this, we introduce **Memory-T1**, a framework that learns a time-aware memory selection policy using reinforcement learning (RL). It employs a coarse-to-fine strategy, first pruning the dialogue history into a candidate set with temporal and retriever filters, followed by an RL agent that selects the precise evidence. The RL training is guided by a multi-level reward function optimizing (i) accuracy, (ii) evidence grounding, and (iii) temporal consistency. This temporal consistency reward provides a dense signal by evaluating alignment at both the session-level (range proximity) and the utterance-level (evidence density), enabling the agent to resolve subtle chronological ambiguities. On the Time-Dialog benchmark, Memory-T1 boosts a 7B model to an overall score of 67.0\%, establishing a new state-of-the-art performance for open-source models and outperforming a 14B baseline by 10.2\%. Ablation studies show temporal consistency and evidence grounding rewards jointly contributing to a 15.0\% performance gain.Moreover, Memory-T1 maintains robustness up to 128k tokens, where baseline models collapse, proving effectiveness against noise in extensive dialogue histories.

Published as a conference paper at ICLR 2026
MEMORY-T1: REINFORCEMENT LEARNING FOR TEM- PORAL REASONING IN MULTI-SESSION AGENTS
Yiming Du Baojun Wang?, Yifan Xiang 1.2 Zhaowei Wang', Wenyu Huang", Boyang Xue Bin Liang 1.2 Xingshan Zeng", Fei Mi², Haoli Bai Lifeng Shang Jeff Z. PanZ, Yuxin Jiang"; Kam-Fai Wong!.
The Chinese University of Hong Kong
2 MoE Key Laboratory of High Confidence Software Technologies, China
3 Huawwi Technologies Co.,Lid *HKUST 'The University of Edinburgh (ydu, kfwong) @ www.cuckk........... jiang.yuxin22 hwwwei..caammmmm
ABSTRACT
Temporal reasoning over long, multi-session dialogues is a critical capability for conversational agents. However, existing works and our pilot study have shown that as dialogue histories grow in length and accumulate noise, current long- context models struggle accurately identify temporally pertinent information, significantly impairing reasoning performance. To address this, we introduce MEMORY-TI, a framework that learns a time-aware memory selection policy us- ing reinforcement learning (RL). It employs a coarse-to coarse-to-fine strategy, first pruning the dialogue history into a candidate set using temporal and relevance filters. fol- lowed by an RI. agent that selects the precise evidence sessions. The RI training is guided by a multi-level reward function optimizing (i) answer accuracy, (ii) evidence grounding, and (iii) temporal consistency. In particular, the temporal consistency reward provides dense signal by evaluating alignment with the query time scope at both the session-level (chronological proximity) and the utterance- level (chronological fidelity), enabling the agent to resolve subtle chronological ambiguities. On the Time-Dialog benchmark, Memory-Ti boosts 7B model to an overall score of 67. 17.0% establishing a new state-of-the-ar performance for open-source models and outperforming 14B baseline by 10.2%. Ablation studies show temporal consistency and evidence that grounding rewards jointly contribute a 15.0% performance gain. Moreover, Memory-TI maintains robustness up 128k tokens, where baseline models collapse, proving effectiveness against noise in extensive dialogue histories. The code and datasets are publicly available
INTRODUCTION
Recent advances in memory architectures and large language models (LLMs) have substantially im- proved the capabilities of conversational agents (Yu et al.) 2025 Zhong et al. 2024 Xu al.) (025)
Increasingly, these agents are expected to support long-term multi-session interactions (Du et al., 2025b Ge al. 2025), where central challenge is understanding and reasoning about temporal relationships across dialogue histories (Wu et al.) 2025 Maharana et al. (024). Without this ca- pability, agents may incorrectly order past events, conflate information from different sessions, and ultimately generate inconsistent or inaccurate answers. For example, as shown in Figure correctly resolving a query such as "What time did Emi mention that some 'Suits" characters were together at the Golden Globes?" requires the agent to locate the relevant mention in the dialogue history, un- derstand the key relative temporal expression ("last night"), and grounding it to the correct session date (*10.01 (0.24"2" to infer the accurate date "January 9, 2024". Ultimately, temporal reasoning is essential for factual consistency in long, noisy conversations.
However, existing approaches (Yu et al.) 2025 Xu al. 2025) remain inadequate for temporal
reasoning in conversation. General purpose long-cone long-context models (Team) 2024 Guo et al. 2025b
General-purpose
"Corresponding authors.

---

Published as a conference paper at ICLR 2026
Memory Mili-Session Dilloguia
13.01 18-45:28
session 17 18.01. 01.2024 01.19:44
session 01:10:41
heavy comfortsbia clothing,
heal prolectent you recommend that does contain karatin?
hours
Elise
importenlly your body...
you night
logether
Your curment halrcare routine sounds relly sold with using purple shampoo...
Eise wydd heavy
Emii
the Golden Globes?
blande hair,
and
Emi
ligaller
Lmi merpioned disc ission 1a 2024 do KITOT 180451220
just had my first yoga class and it want amazing!
1OS 2024 15:00000 10.01 20004 00000
Elise
Evaluation: Question & Answering
Question:
For Factf: Emi ment onod hall ips dscussed the scovicus day, and
Question: What tim did Emi menson that some 'Sutt" Eharacters werd Docetherr al Coken Globel?
Fact2
Else hydrated,
dofiiigg, 2625
Salorm and loown her which one happared earlier?
A. Fact happered earler. X Fact 2 heppened earter. X C. They happen
almost the same time.
Answer:
Answer: Jenuary 3, 2024
spen time range
Evident Session: 13
Evident Session:
Figure I: Multi-session QA with time-event annotations, Time range marks when an event or query occurs, either duration an instantaneous point (start and end coincide). key evidence in the utterance.
Event span highlights
Wang et al. 2025) treat dialogue history as flat text and fail locate resolve temporal expres- sions, leading to steep performance degradation on noisy, extensive conversations (Wu et al. [202]] Maharana al. 2024). Time-aware frameworks such as TReMu (Ge et al. 2025) handle explicit expressions but struggle with ambiguous ones like "the week before that", and error accumulation from inferred event summaries undermines robustness. Reinforcement learning (RL) approaches such as Time-RI (Liu et al.) 2025) rely heavily on structured metadata, making them ineffective for unstructured multi-session dialogues. Thus, a robust, scalable solution for temporal reasoning in dialogue remains an open challenge.
To bridge this gap, we introduce Memory-TI, RL-based memory retrieval framework designed for temporal reasoning that combines coarse-to-fine to-eine retrieval strategy with a multi-level reward design. In the Candidate Generation phase, the query temporal scope is predicted using an LLM to prune the dialogue history search space, which acts as hard filter to prune irrelevant sessions. This is followed by a relevance-based retriever to produce a small, high-recall candidate set of sessions. This phase efficiently narrows the vast memory pool to manageable context, setting the stage for a more precise analysis. In the fine-grained selection phase, an RL agent identifies the precise evidence sessions. Training such an agent is challenging because answer-only supervision provides very sparse signals. To overcome this, we design a dense, multi-level reward function. Beyond answer accuracy (R,) and evidence grounding (Rg), introduce novel temporal consistency reward (R₂) that explicitly evaluates (1) session-level chronological proximity and (2) utterance- level temporal density. By rewarding temporally coherent and contextually concentrated evidence, this structured signal provides richer supervision, enabling the agent to resolve ambiguous temporal expressions and to generalize more robustly to noisy, long-context dialogues.
We validate Memory-Ti on the Time-Dialog (Wei et al. 2025) and LoCoMo Maharana et al., 2024] benchmarks. Results show that Memory-Ti achieves state-of-the-ar the-ttt-rtt temporal reasoning per- formance, substantially improving robustness on contexts up to 128k tokens. Notably. Memory-TI enables a 7B model to outperform a 14B baseline, highlighting the effectiveness of temporal-awan retrieval and dense reward optimization. The key contributions are: (1) A coarse-to-fine memory retrieval framework that efficiently narrows dialogue histories into high-quality candidates before fine-grained evidence selection. (2) A novel dense reward design for RL-based retrieval introducing temporal consistency signals at both session and utterance levels, providing insights into training robust temporal-aware retrieval models by overcoming sparse reward limitations. (3) State-of -ft art performance, with Memory-Ti achieving top results and maintaining accuracy under extremely long and noisy conversational contexts.
2

---

Published as a conference paper at ICLR 2026
Phose Fina-grained Selection Reinforcement Learning
Memory
quasitum
Referwnce Modal (Frur)
(dielogus sessinns)
o Sq ec)
A
Pollcy Model
Time Filter
(eg)
Nomalizttio
+
Retriover
Reward Venfeer
Candition
Reward Design: (R,, Rg)
Exidence Grounding
Temporal stency
(Bession) Chronoloiical Proxinily R.
Answeit(: Typa
Example
Metric
Option
(Uterance) Chronoloyical Fidelity Ny
Timestampp
"2024-03-01"
Unii-eoare Accurasy
Time
days"
cEM
R.
Event Order
(UN00495)) (71006027
Hamning Accunsy
40.51
Rg F [-1,1]
R, [-1.]]
Figure 2: An overview of Memory-T1. The framework employs a coarse-to-fine cascade to select time-consistent memories for multi-session temporal reasoning.
2 RELATED WORK
Temporal Reasoning in LLMs. Temporal reasoning has become active area of research for LLMs (Song et al. 2025 Wei al. 2025 Liu al., (025). Benchmarks such as TimeBench (Chu et al. 2024) and TIME (Wei al. 2025 reveal that even strong models struggle with tempo- ral relationships, event ordering, factual consistency, and long-range reasoning. To these gaps, prior work has aligned knowledge with temporal contexts (Zhao et al. 2024). introduced special- ized training such as Timo
Su et al. 2024) and TG-LLM (Xiong et al. 2024), or applied RL, as in DeepSeek-R1 (Guo et al.) 2025b) and Time-R1 (Liu et al. (025). However, these methods often depend on explicit supervision or handcrafted structures, limiting their applicability to multi-session dialogue. Furthermore, temporal reasoning has recently become important problem in the mem- ory of LLMs. TReMu (Ge et al.) 2025) leverages memory via timeline summaries but relies on timestamp accuracy for temporal reasoning.
few memory-related works (Mai al., 2025 Du et al. 2025a) also highlight the importance of temporal reasoning in long-term memory modeling. Building on this perspective, our Memory-TI framework directly learns implicit memory selection and temporal alignment through multi-level time consistency reward, enabling robust reasoning without external tools.
Reinforcement Learning in Agents: Reinforcement learning is core technology driving break- throughs LLM reasoning, from early outcome-based optimization algorithms, such as PPO
(Schulman al. 2017), , to recent variants for agent scenarios, such as GRPO (Zheng et al.) 2025).
DPO (Rafailov et al. (203)). and GSPO (Zheng et al. 2025). RL not only improves training stability but also efficiency, enabling reasoning-centriic models Is like DeepSeek-R1 (Guoo al. 2025a) and Search-R1 (Jin et al. 2025). Beyond isolated reasoning, RL has been applied to agent settings in- volving tool use QQia al. 2025). multi-step planning (Jin et al. (025). ) and long-term interaction (Yu et al. 2025). Recent studies further extend RL to diverse scenarios, including optimized tool integration (Li al. 2025), emergent code execution under large-scale training (Mai et al. 2025).
and generalized frameworks for retrieval and collaboration (Luo et al., 2025). However, temporal reasoning over multi-session dialogues remains an underexplored area, necessitating robust memory retrieval, chronological alignment of events, and reasoning with ambiguous supervision.
3 MEMORY-TI
Temporal reasoning over extended, versational AI. The task requires agents navigate vast and noisy memory banks retrieve tem-
dialogues presents a significant challenge in con-

---

Published as a conference paper at ICLR 2026
porally accurate and contextually relevant information, process where existing models often fail. To address this, we propose Memory-T1, a novel framework for temporal-aware memory retrieval. We proceed follows: Section 3.1 provides formal problem definition, Section 3.2 details the Memory-T1 framework, and Section 3.3 describes the reward design used to train the agent.
3.1 PROBLEM FORMULATION
Temporal reasoning in multi- scenarios is formulated as a QA task (Figure I): given user query q. the goal is to produce an answer grounded in the dialogue history. The dialogue history represented as a memory bank M Si), (T2, S2)..... (Tv: Sy),, where each session S, is associated with timestamp and consists of sequence of utterances paired with referenced events:
S. = ((,,, Ea), (u2,, E2)..... (uLL,;
where diy denotes the j-th utterance in session i, and Egg {el, (e1, e2........ ... is the set of events mentioned in that utterance. Each event e can be optionally annotate with semantic descriptor K and a temporal span (t%. .%% (see Figure l). These annotations are introduced solely for training- time reward computation and are never accessible during inference. Details of the annotation process are provided in the Appendix A
3.2 MEMORY-T1: TEMPORAL- MEMORY RETRIEVAL
MEMORY-TI is temporal-aware memory retrieval framework designed for multi-session dialogue agents. Its architecture follows a coarse-to-fine foneeee filtering principle to efficiently identify relevant and temporally consistent memories from a vast and noisy dialogue history. The process is organized into two main phases: Candidate Generation and Fine-grained Selection.
Phase 1: Candidate Generation: This initial phase aims to rapidly prune the large-scale memory repository down to a manageable set of high-recall candidates. It consists of two sequential filtering stages:
1. Temporal Filtering:
Given user query q. an LLM first predicts its target temporal win- dow (fstart, fend). This predicted scope acts as a hard filter to discard all sessions whose timestamps do not overlap with this range, drastically reducing the search space and get- ting temporally-filese sessions set Mump. which a subset of the given memory bank M (Mump E M).
2. Relevance Filtering:
From the temporally- the remaining sessions by textual relevance the query. This step further narrows the pool to a manageable size that fits within the agent's 's context budget, while preserving sessions that are both temporally and textually pertinent. The top-ranked sessions form the candidate pool C, formally defined as:
sessions, we then use retriever to rank
arg top-k (F..S)) st.. fun Stad
(Retriever(q, St))
Phase 2: Fine-grained Selection via Reinforcement I Learning. While the candidate set is highly relevant, it may still contain temporally imprecise or misleading information. Reinforcement learn- ing enables the agent to refine its evidence selection policy under reward signals that directly penal- ize incorrect or temporally inconsistent citations. In this way, RL encourages the model disam- biguate noisy candidates and learn robust mappings between cited evidence and generated answers. Therefore, after identifying the candidate pool in Phase we employ an RL-finetuned model to perform the final evidence selection and answer generation in an end-to-end manner.
The agent policy wo takes the query q and candidate pool as input and generates a single, composite output string. This output is structured to explicitly cite the session IDs used as ev- idence, followed by the natural language answer. For example, a valid generation would be: beelected__emmrry [session_3, session__n0 answer 19 days.) From this generated string, we can parse both the selected evidence subset C C (e.g., [session_ 3, session_ and the final answer a. This integrated action space allows the model to lear the direct link between the evi- dence it cites and the answer it produces. This agent learns a policy x (S g. C) to select a subset of evidence sessions S from the candidate pool C given the query q-
4

---

Published as a conference paper at ICLR 2026
To train this policy, employ Group Relative Policy Optimization (GRPO) Zheng al. 2025
an effective RL algorithm for LLM ane-tuning that mitigates high reward variance by using a batch- average baseline. Our overall objective is to maximize the following function:
G L min (0) (0)
max JoRro (0) = D.
G
-) D (TO (-1 ()) [a(()((
(3)
Following structure similar to PPO (Schulman et al. 2017). we first define a probability ratio (0) Tu((S (a,))l((e))))
(a, a, ([q,C)) where S and aj represent the evident session id set and answer in j- th generated output. Here, is clipping hyperparameter that restricts the size of policy updates.
The advantage estimate A, corresponding to a sampled generation that yields the pair (S,, is calculated against the batch-average reward:
Al(a, C) (S),,,) = (),,,
R((g. C) (S),
(4)
G
The reward R is given by a multi-level function in Section 3.3
The second term in Eq. 3 is KI divergence penalty regularizing the current policy against a frozen reference Tet to ensure training stability. Algorithmic details are in Appendix B
3.3 REWARD DESIGN
In this section, we describe the design of our verifiable rewards. The core motivation of the multi- level reward is to address the limitation of sparse supervision. As shown in Table models such as MemAgent (Yu
2025), which are trained solely on answer accuracy (R,), fail develop effec- tive temporal reasoning abilities. Thus, it is necessary to jointly optimize evidence grounding (Rg- ensuring the correct sessions are used) and temporal consistency (R2, ensuring temporal alignment with query) to form dense, structured reward signal. Since all rewards assume that the model output can be successfully parsed into the required format (e.g.. {selected memory answer : ....).. assign fixed penalty of -05 if parsing fails. The overall reward is defined as:
W. Ra + u Rg w. Rr, parsing succeeds,
R 0.5,
RE 1-11)
(5)
otherwise,
where Way Wgg Wy are tunable weights with w. + wy + wt =1 Exact values are in Appendix C.1 and sensitivity to different settings is analyzed in Appendix
Accuracy Reward (R,) This reward ensures that the final predicted answer is correct, providing the most direct supervision signal for the agent's output quality. As tasks require different answer formats, R2 is multifaceted metric tailored to four main types, each with specialized evaluation function. For Option Answers (c.g., "A", , "A C"), we use strict Exact Match (EM). For numer- ical answers involving dates durations, employ more flexible metrics: Timestamp answers (e.g.. "2024-09-01") are assessed with Unit-aware Accuracy, while Time Interval answers (e.g., "13 days") use c-Exact Match (c-EM). Finally, for sequential answers like Event Order, we Hamming Accuracy to credit partial correctness. The final reward R, is normalized to the range [-1, with detailed formulations in Appendix
C.3
Evidence Grounding Reward (Ry) This reward encourages the model to retrieve and utilize in- formation from the correct dialogue session(s). Specifically, this reward is calculated by comparing the set of session IDs C cited by the agent against the gold-standard evidence set, M provided in the dataset. The degree match is quantified using the Jaccard Index, which measures similarity by dividing the size of the interscction of the two sets by the size of their union. This score is then scaled to range [-1, where perfect match (Jaccard Index of 1) corresponds to a reward of +1, and complete mismatch (Jaccard Index of results in reward -1.
5

---

Published as a conference paper at ICLR 2026
Temporal Consistency Reward (R,): This reward component enforees fine-grate fine-grinedd temporal alignment between the selected sessions and the query. It is composed of two sub-rewards: chrono- logical proximity (R,) and temporal coverage (Ry).
= aR, + BR,, (a+B=1)
(6)
This reward measures the temporal distance be- tween the selected session timestamp U and the gold temporal range To of user query. Recognizing that hard-cutoff penalty is too rigid for the temporal ambiguities in real-world dialogues (e.g.. time- zone shifts, extended topies), we employ logistic function create soft, differentiable penalty that better handles this imprecision. The reward is formulated as:
I.Chronological Proximity (R,, session-level):
R
d,
Rg E (-d, c- c-d, d).
exp(x)
where the normalized distance is defined as:
gap(U. UU IQ) m
(8)
Here, SUp(U, IQ) is the minimum temporal distance (in days) between spans U and IQ (zero if they overlap). The hyperparameters offer fine-grained control: the tolerance margin m penalty-free grace period (e.g.. 7 days), the scale factor controls the penalty curve sharpness, and the parameters c, d scale the final reward magnitude (to a range (-0.5, 1)). This logistic approach ensures that sessions close to Q are highly rewarded while distant sessions are penalized. For detailed settings, please refer to Appendix C.I
2. Chronological Fidelity (Ry, utterance-level). While R, handles session-level relevance, R₁ evaluates the fine-gre tne-grained quality of events within each utterance. rewards sessions dense with evidence that temporally aligned with the time range of the query, Iq. First, assign discrete score тe to each event based on its temporal overlap with lo:
+1, if the time range of event is fully within lq. rele, IQ) = +0.5, if partially overlaps with Tq,
1,
if no overlap with lq.
The final fidelity reward Ry is then calculated by first averaging these event scores within each relevant utterance (u E Usa)), and then averaging the resulting utterance scores across the session:
Σ role, (e))
if Unrll O,
(y =
Urel
Em
(10)
otherwise.
This reward structure effectively penalizes a common failure mode: selecting a session from the correct time period but grounding the answer in textually similar but temporally incorrect utterance from within it. It incentivizes the agent to select sessions that are not just broadly relevant but also densely packed with chronologically precise evidence. By combining these three reward signals (Ra, Ry: Rt). our multi-level reward structure guides the agent to develop robust, generalizable temporal reasoning policy that does not overfit superficial cues.
EXPERIMENTS
4.1 DATASETS
Time-Dialog We use Time-Dialog the core benchmark, extended from the dialogue portion of the existing Time dataset (Wei et al.) 20055). containing 4,716 QA examples corresponding with the multi-session dialogue history shown in Figure [o train robustly time-aware agent, augment the dataset with fine-grained annotations for supervision, specifically annotating the target time range for each query, utterance-level events with their time spans, and the ground-truth session

---

Published as a conference paper at ICLR 2026
IDs for ideal evidence retrieval. Further details shown in Appendix grained annotations serve exclusively as a ground-truth signal for computing our multi-level rewards during training. To ensure fair and realistic evaluation, this enriched information is withheld from all models during inference. The final dataset of 4,716 examples is partitioned into training (4,065), validation (451), and held-out test (200) sets.
A Crucially, these fine-
LoCoMo To assess the out-of domaan (OOD) generalization of our trained policy, employ the LoCoMo benchmark (Maharana et al. 2024), an established testhed for multi-session conversational memory. LoCoMo is composed of five distinct subtasks, evaluate temporal reasoning. This makes it an ideal held-out test set to validate whether our model has learned a generalizable temporal reasoning skill, rather than overfitting to the patterns of the Time-Dialgg dataset.
of which is specifically designed to
4.2 EXPERIMENTS SETUP
Baselines Our proposed method, MEMORY-TI, is built upon Qwen2.5-3B and Qwen2.5-7B-
Instruct (Team 2024). We compare it against a comprehensive suite of baselines, including stan- dard methods like Full Context, which is evaluated across wide range of open-source models (Qwen2.5-3B/7B/14p Gemma-4B-it (Team) 2025). -LLaMa-3.--8--Insst
(Dubey et at... 2024) Generation (RAG) (Lewis et al. 2020); the agentic ReAct framework using GPT-4 as its backbone (Yao et al,
and the closed-source GPT-4 (Achiam al., 2023 standard Retrieval-Augmente
2023) and Supervised Fine-Tuning (SFT) model fine-tuned from Qwen2.5-3B (Ouyang al.
2022 Furthermore, we benchmark against two state-of-the-arr specialized agents, MemAgent Yu al. 2025) and Time-R1 (Liu al.) 2025), by evaluating their public checkpoints in zero-shot setting. Finally, to isolate the benefits of our contributions, we include an RL (Task Reward Only) ablation baseline, which uses the same architecture as MEMORY-TI but is trained only with task accuracy reward (Ra), omitting our proposed temporal consistency (R₂) and evidence grounding (Ry)) rewards.
Implementation All our experiments build upon Owen2.5-3BBIInsteee and (.wen..5-7B-Instruue as the main models. We adopt BM25 as retriever model due to the efficiency. We adopt the GRPO training strategy within the VERL framework (Sheng et al.) (024). We implement our RL training with batch size of 32, a learning rate of x 10 K=8 rollout responses per prompt, KL coefficient 0.1, and a maximum sequence length of 16k tokens.
4.3 RESULTS
As shown Table MEMORY-TI establishes new state-of-the-art, with our 3B and 7B models achieving top overall scores of 66.9% and 67.0%. This performance represents significant leap over a diverse set of basclines. Compared to specialized SOTA models, our trained agent surpasses the zero-shot performance of both the temporal reasoning model Time-R1 (49.4%) and the memory- based framework MemAgent (49.9%) by over 17 absolute points, highlighting the necessity of tar- geted training for this complex task. Crucially, our approach proves superior simply increasing model scale. Our 3B model not only consistently outperforms larger models from different fami- lies, including Gemma-4B (45.0%). L.lama--8BB (48.4%), and even the much larger Qwen2.5-14B 14BB (60.7%), but also performs nearly identically to our 7B variant. This strongly suggests that the per- formance gains stem primarily from our learned policy rather than the scale of the base model. No- tably, MEMORY-T1 also outperforms standard GPT-4 configurations, surpassing both Full Prompt (64.8%) and ReAct (62.8%). While gap remains the ideal GPT-4 (Oracle) score of 86.2%, this overall dominance confirms that our learned memory policy is both necessary and effective. This advantage driven by our model's particularly strong performance on complex reasoning tasks, such order reasoning (OR) and range reasoning (RR), directly validating the effectiveness of its temporally grounded memory selection policy.
4.4 ABLATION STUDY
Ablation Study on reward components. Our multi-component reward function is crucial for robust performance, as shown in Table Training with only task accuracy (R.) leads to catas-

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison across different models and training strategies on temporal rea- soning subtasks. Category A metrics include Location (Loc.), Duration Comparison (DC.), Com- parison (Comp.), Order Comparison (OC.), and Extraction (Ext.). Category B's metries covers ER.=Event Reasoning, OR.=Order Reasoning, RR.=Range Reasoning. Category C's metrics com- prises CTE:=Cnntxtual Temporal Filtering, Co--mp-=Co-temporalit: TL.=Timeline. Bold and underline denote column-wise best and second-best among non- non-GPT rows. fracle setting using gold test evidence.
Category
Category
Category C
Experiments
Overall
Loc.
DC.
Comp.
OC.
Ext.
ER.
OR.
RR.
CTE
Co-tmp.
TL.
GPT-4 (Oracle Evidence)
88.9
78.3
54.9
95.0
66.7
100.0
88.9
93.8
83,3
100.0
35.4
86,2
GPT-4 (Full Prompt)
61.1
87.0
46.7
85.0
22.2
64.1
55.6
81.3
50.0
77.8
27.1
64.8
GPT-4 (ReAct)
66.7
43.5
39.2
75.0
32.2
67.1
72.2
70.8
68.5
84.3
29.2
62.8
Gemma-4B-it
5.6
60.9
12.7
55.0
33.3
62.2
38.9
50.0
44,4
61.1
15.3
45.0
Time-RI
11.1
47.8
13.9
65.0
55.6
76.9
27.8
44.4
55.6
66.7
25.0
49.4
Qwen2.5-3P (Instruct)
5.6
56.5
7.2
70.0
44.4
66.7
33.3
56.3
55.6
77.8
12.5
49.4
Qwen2.5-30D 5-3B+SFT
22.2
56.5
7.9
65.0
44.4
66.7
33.3
56.3
55.6
77.8
16.7
50.6
Memory- (3B)
50.0
52.2
7.1
75.0
55.6
82.1
66.7
88.9
94.4
12.4
66.9
Llama- (Instruct)
22.2
43.5
5.6
75.0
33.3
79.5
27.8
56.3
72.2
27.8
14.6
48.4
MemAgent--7
55.6
47.8
10.2
55.0
40.7
61.5
38.9
62.5
38.9
72.2
27.1
49.9
Qwen2.5-77 5-7B (Instruct)
61.1
52.2
0.0
75.0
55.6
63.3
38.9
54.2
50.0
72.2
16.7
53.2
Qwen2.5-1ee 1.ee2..--14B (Instruct)
47.8
4.4
55.6
84.6
69.7
94.4
20.8
60.7
MemoryTl (7B)
61.1
52.2
8.6
65.0
56.7
71.8
83.3
87.5
88.9
94.4
27.1
67.0
80.0
Legend: Barse LiceelRecall
70.0
Z
Owen225-3gg (ne lime litert
Qwen2.5-36 5-28
Rocall time fittrr)
MamoryyT1 (381
69.9
80.0
Flsse (N))
Recill (%)
60.0
549
$52
60.0
$1.4
52.5
50.0
95.9
41.6
42.9
40.0
40.0
30.0
10
Figure 3: Performance comparison between Memory-Ti (3B) and Qwen2.5-3B (Instruct) under different top-k values (bar charts represent overall F1 scores; line charts represent evidence session recall rate. Comparison conditions: With/without temporal filtering; Top-k refers to the number of sessions retrieved in the candidate generation phase.)
trophic 22.4% drop the overall score, with performance on complex reasoning (Category B & C) collapsing. Removing the evidence grounding reward (w/oRy) significantly extraction-bass tasks (Category A, 17.4%), causing a 9. 1% overall performance drop and confirm- ing its role in preventing distraction. The temporal consistency reward (R,), composed of sequence (R.)) and fine-grained (Ry) components, is vital for structured reasoning. Most revealingly, ablating only the sequence component RR, creates sharp trade-off: simpler tasks (Category A) unexpect- edly improve by 23.4%, while complex reasoning (Category B) collapses by 56.2%. This highlights crucial synergy: Rg grounds the model in what evidence use, while RG teaches it how to reason with that evidence temporally. To clarify the non-monotonic effects of R. components, Category- duration tasks rely on two complementary mechanisms: global timeline consistency (R,) and content-level temporal relevance (Rz). Removing only one leaves the other compensatory con- straint, improving simpler timestamp- or gap-based reasoning. Full removal of RS eliminates both regulating factors, preventing correct event selection and temporal alignment, which explains the sharp performance drop.
siniitliyy harms localization and
4.5 MODEL ANALYSIS
Out-of-Domail Generalization. Our model demonstrates strong oft-of-domain (OOD) general- ization the LoCoMo benchmark (Table 3 Table 2 in Appendix C)). MEMORY-TI achieves top score of 37.7% %, a significant improvement over the 33.5% from the base Qwen-2.5-3B model. This advantage is particularly consistent in the AG setting 31.9% 36.7%), driven by substantial
8

---

Published as a conference paper at ICLR 2026
Ablation Study candidate generation phase. Figure validates our coarse-to-fine candidate generation strategy. First, increasing retrieval depth top-k to 10 is essential to achieve high evidence recall (90%). Our temporal filter proves highly precise, as the overlapping recall lines show removes distracting context with- out sacrificing this crucial evidence. Second,
Table 2: Ablation study the reward func- tion of Memory-Ti (3B). Relative changes com- pared to the full model are shown in parenthe- ses.
Model
Category
Category
Calegory
Overall
Memory-TI TI (3B)
49,5
79.5
80.3
66.9
solo R2
(-7.9%)
45.6
(-5,5%)
75.1
(-19.9%)
64.3
(-5.1%)
63.5
even with the same unfiltered context, our RL-
remove P. only
(+21,455)
61.1
(-56.55)
34.8
(-17.4)
66.3
(4.9%)
66.3
tuned MEMORY-TT agent outperforms the base model (54.0% vs. 44.2%). The synergy of com- bining broad retrieval for high recall with sharp, suddence-preserinng filtering creates an optimal candidate pool that enables the agent achieve its final 66.9% score.
cemove only
(-1.0%)
50,0
(-2899)
56.5
(-11.66)
63.0
(-319))
6188
wro X2
(-17.4%)
40,9
75.3
(-3.5%)
75.9
(-919)
60,8
R. only
(-11.9%)
43.6
(-27.7%)
57.5
(-26.66)
59.0
(-22.4%)
51,9
Table 3: LoCoMo benchmark: -ioma-n evaluation of Qwen-2.5-33 Srren-2.5-3B-Insttie Instruuttt and Memory- TI (3B) under RAG and Non-RAG settings. Values are shown percentages; best results in cach column are bolded. AOverall shows improvement relative to Qwen-2.5-3BB 2.---B.----B-tt-- (Non- -Ao).
Model Family
Setting
Single-Hop
Multi-Hop
Temporal
Open- Open-Domain
Adversarial
Overall
AOveral (%)
Qwen-2.5-3B (Insiract)
Non-RAG
49.8
28.7
24.5
13.5
16.6
33.5
RAG
46.0
22.0
27.3
11.4
19.5
31.9
-1.6%
Memory-TI (3B)
Non-RAG
51.2
315
15.8
26.0
+4.2%
RAG
48.9
25.8
30.7
14.6
29.8
36.7
+3.25%
Table 4: Robustness of Memory-TI under increasing time label noise.
Category A
Category
Category C
Noise Level
Overall
Loc.
DC.
Comp.
OC
Ext.
ER.
OR.
RR.
CTE
Co-tmp.
TL.
20%
27.8
43.5
5.0
55.0
25.9
76.9
72.2
81.2
94.4
94.4
16.7
60.0
10%
50.0
43.5
10.6
65.0
55.6
74.4
67.4
81.2
88.9
94.4
18.8
63.4
5%
50.0
60.9
60.0
55.6
82.0
77.8
87.5
94.4
88.9
16.7
67.0
gains in the Temporal and Adversarial subtasks. Intriguingly. MEMORY-TI yields better perfor- mance in the Ron-RAG setting compared to the RAG setting, suggesting it has learned superior internal memory management skill. The Adversarial subset is a notable exception, which focuses on answerability detection (saying "I don't 't know" when the information is missing). Without the RAG setting, the Post-filter candidate pool remains lengthy and is prone to "lost in the middle" ef- fects and spurious snippets that encourage hallucination. With RAG, the condensed candidate pool prunes spurious in-dialog segments, preserving compact set lacking supporting evidence. This makes it easier for the RL policy learn "unanswerable" behavior more effectively (26.0 - 29.8). It introduces mild distribution shift and loss of temporally key candidates on standard tasks but benefits adversarial detection by simplifying evidence incompleteness detection.
Robustness in Long-Contedd Scenarios. To assess how models handle increasingly complex di- alogues, we partition the test by context length and evaluate performance each bracket (Fig- 4. As context length increases, the performance of baseline models collapses due to attentional dilution; the Qwen2.5-7B baseline, for instance, drops by over 30 Fl points. contrast, MEMORY- TI maintains a high and stable Fl score across all lengths. This creates a performance gap that widens dramatically with context, growing from +9.8 point advantage to a massive +25.0 point lead for MEMORY-TH (7B) in the 64k-128k bracket. This resilience stems directly from our learned policy, which effectively filters context and shields the model from distraction, confirming its su- periority for long-range reasoning. Further controlled experiments on lost-in-the-middle effects are provided in Appendix C.5
Robustness under increasing time label noise. As shown in Table 4 with 5% noise (realistic er- rate), overall F1 remains 67.0, and key temporal reasoning tasks such as Counterfactual (CTE),

---

Published as a conference paper at ICLR 2026
1150
-4
11.0
4000
Qwest 3-15 (bstrac)
@wes1 5-10 Instrsti
Me (38)
Mannay- (7175)
Figure 4: Comparison of Qwen2.5 and Memory-TY models on the test set, where examples are grouped by the length of each test example (tokens) (Ok 8k, 8k-16k, 16k-32k, 32k 64k, (2kk) to assess performance variation across lengths, along with overall evaluation.
Table Analysis of model performance and Retrieval-Augmented Generation (RAG) Latency (time in seconds)
Model
Num of Q
Total Inf. Time
Avg Latency
Total Inf. R
Retrieval Time
Time-R1
200
248.62
1.24
256.35
0.01
MemAgent
200
312.72
1.56
320.47
0.01
Qwen2.5-3B (Instruct)
200
271.83
1,36
279.74
0.01
Mcmory-T1
200
252.08
1.26
259.81
0.01
Co-temporality (Co-tmp.), and Relative Reasoning (RR.) stay high at 94.4, 88.9, and 87.5, respec- tively. Increasing the noise to 10% and 20% leads gradual but moderate degradation of the overall score to 63.4 and 60.0. Notably, the most temporally demanding tasks remain robust: CTF. and Co-tmp. stay above 88.9 even at 20% noise. The main decline is concentrated in time- span span-related subtasks (such as Localization and Extract). This confirms the Memory-17 1 is resilient to realistic label noise, supporting its practical applicability real-world settings where time labels are imperfect.
Efficiency Analysis Memory-T1 incurs negligible additional inference latency (Table 5). The average latency (1.26 seconds per query) highly comparable to baselines such as Time-RI (1.24 seconds) and Qwen2.5-3B (1.36 seconds). Crucially, the retrieval overhead (0.01 seconds) is in- significant relative to the total LLM generation latency, confirming that the framework achieves its improved performance with minimal computational cost.
Qualitative Analysis. We focused our qualitative analysis the six subtasks (ER., OR., RR., CTF, Co-tmp., and Loc.) where Memory-TI exhibits the largest performance gains (Table [0] in Appendix). A consistent pattern emerges across these subtasks: the base model often relies semantic similarity rather than temporal correctness, which leads to systematic errors such as ne- glecting time constraints, confusing event order, overlooking co-temporal relations, and failing to incorporate counterfactual adjustments. Memory-TI mitigates these issues through explicit time- range filtering and RL-based selection that enforces temporal consistency, yielding more accurate localization, ordering, and co-temporality. These qualitative observations align with and explain the performance improvements observed on the six subtasks.
5 CONCLUSION
In this work, we introduce MEMORY-T1, novel reinforcement learning framework addressing the critical challenge of temporal reasoning over long, multi-session dialogues. The framework employs a coarse-to-fine strategy, guided by multi-level reward function that incorporates answer accuracy, evidence grounding, and temporal consistency signal. This design provides the agent with dense supervision to effectively handle temporal ambiguities and noise. Experiments show that MEMORY-TT achieves state-of of-the-art performance on the Time-D -iialo benchmark, enabling a 3B model to outperform 14B baseline and maintaining strong robustness in dialogue histories up to 128k tokens. This work demonstrates that selecting temporally consistent memory evidence is a critical step toward building reliable and factually consistent long-term conversational agents.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We are committed to ensuring the transparency and reproducibility of our research. To support this commitment, will publicly release annotated dataset and all source code, facilitating future extensions and community research. Comprehensive details of our methodology are provided throughout this paper: the annotation process and prompts are illustrated Appendix A Figures 21 20 and 22 training and evaluation prompts are shown in Figure 23 and Figure 24] respectively.
Furthermore, detailed algorithmic procedures can be found in Appendix B We believe that releas- ing these assets will lower the barrier for replication, enable fair comparisons, and foster further exploration in this line of research.
ETHICS STATEMENT
The main artifact of this work is the annotated Time-DDalog dataset. To facilitate the process, develop a dedicated cvidence-annotation website (Figure and engage three experienced NLP re- searchers annotators. Approximately 200 human hours are devoted verifying GPT-4-assisted annotations, categorizing error types, and refining the protocol through several iterations. All an- notators are properly briefed and held regular discussions to resolve ambiguous cases. Model eval- uations are conducted by three trained research assistants, each compensated at $20/hour, which is above the local average. Prior to release, all data underwent rigorous screening to ensure the exclu- sion of personally identifiable information and offensive content. Both the dataset and code will be publicly released under an MIT license to encourage transparency and community use.
ACKNOWLEDGEMENTS
This work is partially supported by Hong Kong RGC GRF No. 14206324. the National Natural Science Foundation of China 62576120, the Major Key Project of PCL2025A09 and Key Labora- tory of Computing Power Network and Information Security, Ministry of Education under Grant No.2024ZD020.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303.0877 2023.
Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Haotian Wang, Ming Liu, and Bing Qin. TimeBench: A comprehensive evaluation of temporal reasoning abilities large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.). Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 1204-1228. Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: URL
10.18665/v1/2224.achh
202. acl atc.:/2222....1.noee..aaaa
66/
Yiming Du, Wenyu Huang. Danna Zheng. Zhaowei Wang, Sebastien Montella, Mirella Lapata, Kam-Fai Wong, and Jeff Z Pan. Rethinking memory in ai: Taxonomy, operations, topics, and future directions. arXiv preprint arXiv:2505.00677 2025a.
Yiming Du, Bingbing Wang. Yang He. Bin Liang, Baojun Wang, Zhongyang Li, Lin Gui, Jeff Z Pan, Ruifeng Xu, and Kam-Fai Wong. Bridging the long-term gap: A memory-active policy for multi-session task-oriented dialogue. arXiv preprint arXiv:2505.20231, 2025b.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Yubin Ge, Salvatore Romeo, Jason Cai, Raphael Shu, Monica Sunkara, Yassine Benajiba, and Yi Zhang. Tremu: Towards neuro-symbolic temporal reasoning for Ilm-agents with memory in multi-session dialogues. arXiv preprint arXiv:2502.016630 2025.

---

Published as a conference paper at ICLR 2026
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma. Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhi- hong Shao, Zhuoshu Li, Ziyi Gao. Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Honghui Ding, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Ji- ashi Li, Jingchang Chen, Jingyang Yuan, Jinhao Tu, Junjie Qiu, Junlong Li. J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaichao You, Kaige Gao, Kang Guan, Kexin Huang. Kuai Yu, Lean Wang. Lecong Zhang. Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingxu Zhou, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang. R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye. Shiyu Wang. Shuip- ing Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Tao Yun, Tian Pei, Tianyu Sun, T. Wang. Wangding Zeng, Wen Liu, Wenfeng Liang, Wenjun Gao, Wengin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xi- aotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su. Xuheng Lin, X. Q Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaox- iang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan. Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang. Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Viyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang. Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma. Ying Tang, Yukun Zha, Yut- ing Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha. Zhe Fu, Zhean Xu, Zhenda Xie. Zhengyan Zhang, Zhewen Hao, Zhicheng Ma. Zhigang Yan. Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie. Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. DeepSeek-RI incentivizes reasoning in LLMs through reinforcement learning. Nature, 645(8081)6633--33 638, September 2025a. ISSN 1476-4687. doi: 10. 0011088/41186-025-00444
10. 1038/0380000 8 586 09422-22222
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12944 2025b.
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Serean Arik, Dong Wang. Hamed Zamani, and Jiawei Han. Search-rl: Training lims to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.000 .05 16, 2025.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Sttrival-augmeete gener- ation for knowledge-intensivy nlp tasks. Advances in neural information processing systems, 33: 9459 9474,9974, 2020.
Xuefeng I.i, Haoyang Zou, and Pengfei Liu. Torl: Scaling tool-integrated rl. arXiv preprint arXin:2500.23338,, 2025.
Nelson Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157-173, 2024. doi: 10.1162/tacl a 00638. URL
https: http::///acaattho conntoooogy.oo org 2024 tacl-
Zijia Liu, Peixuan Han, Haofei Yu, Haoru Li, and Jiaxuan You. Time-rl: Towards comprehensive temporal reasoning in llms. arXiv preprint arXiv:2505. 13508, 2025.
Xufang Luo, Yuge Zhang, Zhiyuan He, Zilong Wang, Siyun Zhao, Dongsheng Li, Luna K Qiu, and Yuqing Yang. Agent lightning: Train any ai agents with reinforcement learning. arXiv preprint arXiv:2508.0 03680, 2025.
Adyasha Maharana, Dong-Ho Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of LLM agents. In Lun-Wei Ku, Andre
12

---

Published as a conference paper at ICLR 2026
Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13851-13870, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.18653/w1/2224.acchbb URI wt...ttss:// //ap::://c// htt///aclaanchoccc htwpbhooolgy..ooo/222.......
Xinji Mai, Haotian Xu, Weinong Wang, Jian Hu, Yingying Zhang, Wengiang Zhang, et al. Agent rl scaling law: Agent rl with spontaneous code execution for mathematical problem solving. arXiv preprint arXiv:2505.0777: 2025.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Türe Gokhan Tur, and Heng Ji. Toolrl: Reward all tool learning needs. arXiv preprint arXiv:2504.13955 2025.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36::3728-533777 2023.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024.
Jiayu Song, Mahmud Elahi Akhter, Dana Atzil-Slonim, and Maria Liakata. Temporal reasoning for timeline summarisation in social media. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 28085-28101, Vienna, Austria, July doi: 18.13633/v// org/2025.2222222 acl acl..ong. 1362/
2025. Association for Computational Linguistics. ISBN 979-8-89176-251-4 2025 2025.acl-long. 1362. URL
Zhaochen Su, Jun Zhang, Tong Zhu, Xiaoye Qu, Juntao Li, Min Zhang, and Yu Cheng. Timo: Towards better temporal reasoning for language models. arXiv preprint arXiv:2406. 0e0p4406.449999 14192, 2024.
Gemma Team. Gemma 3. 2025. URL httthhtt////ggg htt/:/oo..geeeeemmaeeeeeeee
Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL httpps://qwentt itttuub.i//blog. qww...///////////////
Zhaowei Wang, Wenhao Yu, Xiyu Ren, Jipeng Zhang, Yu Zhao, Rohit Saxena, Liang Cheng, Ginny Wong, Simon See, Pasquale Minervini, et al. Mmlongbench: Benchmarking long-context vision- language models effectively and thoroughly. arXiv preprint arXiv:2505.106/1 2025.
Shaohang Wei, Wei Li, Feifan Song, Wen Luo, Tianyi Zhuang, Haochen Tan, Zhijiang Guo, and Houfeng Wang. Time: A multi-level benchmark for temporal reasoning of llms in real-world scenarios. arXiv preprint arXiv:2505. 12891, 2025.
Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. Long- memeval: Benchmarking chat assistants on long-term interactive memory. In Proceedings of the 2025 International Conference on Learning Representations (ICLR), 2025. URL https: Accepted at ICLR 2025.
/oopenzvvew.
Siheng Xiong, Ali Payani, Ramana Kompella, and Faramarz Fekri. Large language models can learn temporal reasoning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceed- ings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp- 10452- 10470, Bangkok, Thailand, August 2024. Association for Computa- tional Linguistics. doi: 10. 10.18663/vll 1/224/1112222..1oooggggg
URL https: //tc h/t////aaaaannoo.....
org/2024 acl 1lg.. 5537
13

---

Published as a conference paper at ICLR 2026
Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. A-mem: Agentic memory for Ilm agents. arXiv preprint 2rXiv:2502.12110, 2025.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma, Jingjing Liu, Mingxuan Wang, et al. Memagent: Reshaping long-context Ilm with multi-conv rl-based memory agent. arXiv preprint arXiv:2507.02259) 2025.
Bowen Zhao, Zander Brumbaugh, Yizhong Wang, Hannaneh Hajishirzi, and Noah Smith. Set the clock: Temporal alignment of pretrained language models. In Lun-Wa Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings the Association for Computational Linguistics: ACL 2024, pp. 15015-15040, Bangkok, Thailand, August 2024. Association for Computational Linguis- tics. doi: 10. 10.11865//112024 1/2024 findings-acl.899 URL https: Findings-aci.891 8927
Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization. arXiv preprint 1iXi::2507.18071, 2025.
Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 19724-19731, 2024.
14

---

Published as a conference paper at ICLR 2026
A DATASET AND ANNOTATIONS
Our experiments conducted the Time dataset, comprehensive benchmark for temporal rea- soning over long-form dialogues. The dataset features complex dialogue histories and structured into 3 levels of reasoning difficulty and 11 distinct QA subtasks. The distribution of these subtasks in the dataset is detailed in Table
Table 6: Distribution and characteristics of QA subtasks in the Time dataset, grouped by reasoning level.
QA Subtask
Format
category
# Samples
Localization
Time Span
381
Duration Corppee
Single Choice
A
385
Computation
Time Span
A
390
Compare
Single/Multi Choices
380
Extract
Single/Multi Choices
A
197
Explicit_Reasonneee
Single Choice
B
363
Order Reasoning
Single Choice
B
381
Relative Reasoning
Single Choice
B
393
Counterfactual
Single/Multi Choices
C
398
Co temporality
Single/Multi Choices
C
397
Timeline
Event Order
C
390
While the Time dataset provides strong foundation, it lacks the fine-grained annotations neces- sary for our reward mechanisms and detailed analysis. To address this, we augmented the dataset with three additional layers of annotations. Our annotation process employed an iterative frame- work where GPT-4 performed an initial annotation pass, followed by human verification to identify systematic error patterns. These insights were then used to refine the prompts for a final, improved annotation pass, achieving an overall accuracy of over 95%.
1. Question Temporal Range (lg). First, for each question, annotate its target temporal range (lq). Many questions implicitly focus on a specific period within the long dialogue history. We prompted GPT-4 to infer and extract this time range. For questions with discernible temporal focus, we assigned a default range starting from "unknown" to our annotation timestamp (c.g., "2025-07-17T11:46:33 As this timestamp later than any event in the dataset, this default range effectively covers the entire dialogue history. The prompt can be found in Figure 21
2. Evidence Grounding (M'). Second, annotate the ground-truth evidence sessions (M²) and utterances required to answer each question. The original dataset's fact bank could not be reliably mapped the dialogue text. We therefore used our iterative GPT-4 (Figure 22p and human- in-the-loop process (Figure 5) to perform this grounding. This resulted in a session level annotation accuracy of over 95% and an utterance-level accuracy of over 85%. To avoid introducing poten- tial noise from less accurate annotations into our reinforcement learning process, we use the more reliable session-level annotations for calculating the Evidence Grounding Reward (Rg).
3. Utterance-level Event Times. Finally, to enable a deeper temporal analysis, we performed utterance-level event extraction and temporal grounding for the entire dialogue history (Figure 20 This annotation is crucial because the timestamp of a dialogue turn (when something was said) often differs from the timestamp of the event being discussed (when something happened). This distinction is the primary motivation for our chronological proximity (R,) reward. For each utterance, we prompted GPT-4 to extract key events and resolve their temporal scope based on the dialogue context. For instance, given dialogue turn '2025-06-20: an utterance mentioning "the meeting last week" would be grounded to a specific range like 12025-06-09, 2025-06-133 -33) For utterances without explicit temporal markers, past tense implies a range from the distant past up to the dialogue time, while future tense implies a range from the dialogue time to the distant future).
used grammatical tense to infer broad range (e.g.,
15

---

Published as a conference paper at ICLR 2026
Jump to Quaition
Fibtr by session ID
Apply Fiter
Ylw Full Contec
Question 27 of
Which of the followern two duration oongrr" "Duratin Benween Samantte parted cute des gn or T-atart Samenthh vent hiang nea bh rattonal park axt stary "Cowspiag""
group
daraliam ane approcrmanll y the sare langht
Wrever
Evidence
people
poutr playing on spot
songe
Sesalon
Wetthedd "Cowoprncy" last week and It' worth watching its about vegorin and sustainob Fyy Check out
Samarthha:
thanl expecedd I'm and from my fro atuenture time Sewn gine serrityy
chards
Section
Jumee
James shams image about (MAGE photospaphy teo paoaple walling upa path mountain] and lays: thyy Good So joined hiking group likkng with photo get
That's coull love adddnng movie music playliss. travnnt had time for aoventure moves litely because theak, but if have IIll check em when have some sine.
swesome!
Speater zaiees
Samartha::
get it when you can, check out "The Secret Ulfe of Warter Mitty' It's owesome adventure move with awmiome story, think you'll Boo in
Sestion 14
Samantha sherds image about JINKEE a photography of waman taking pilture o mounnrri and says Your liking pkk looks umnic ngt want likinng lisst morhh af nearly nationa park. that cool. Hou iten anyehhrr now for haknng soontyy
Jemes
Tharks for the red I'm addinng "Waiter bur Milty" to waldhist Anything now in the vogan doco fiel?? hin looking for sornething good.
Foatched "ooup rary" untaintiitty Check out
it's worth warthwwatdhing it's about veganien
Session
Thankol painted my thatert looks great avesome youtre standing arimal rights Howid procest go? Get pks?
Spenier tementhe
sounds owssomet giota my Thores swwys drndoumentaiee twrning
Semanthe:
what you
Janes
withh Apprectono
Previous
Next
Figure 5: An overview of website for human annotation.
B ALGORITHM
The core of the framework is reinforcement learning agent trained with Group Relative Policy Optimization (GRPO) as shown in [lgorithm[[] Our overall approach involves a two-phase process: first, efficient candidate generation stage to prune the search space, followed by reinforcement learning (RL) fine-tuning stage to train the policy model.
Candidate Generation. Given a query q and the full dialogue memory M, we first generate a small, highly relevant pool of candidate sessions C. This step, detailed in Algorithm 2 is crucial for making the subsequent selection process tractable and efficient.
Phase 1:
Phase 2: RL Fine-tuning. With the candidate set C, we perform an RL update. For each instance in the batch, we sample G distinct outputs from the current policy т&. Each output contains a selected evidence set S, and a generated answer aj. A multi-level reward R2 is then calculated for each of the G' samples by comparing it against the ground-truth labels (M", a", Iq). This reward, detailed in Algorithm 3 provides comprehensive signal reflecting accuracy, evidence grounding, and temporal consistency.
To reduce the variance of the policy gradient estimate, we compute an advantage A, for each sample. Following GRPO, we use a simple yet effective batch-average baseline, where the advantage the sample's reward minus the average reward across all G samples in the batch (A, = R2 RR)
Finally, the policy model's parameters 0 are updated using the GRPO objective function. This objective maximizes the weighttd log probaabiiiyy of the sampled outputs. Crucially, also includes Kullback- Leibler (KL) divergence term, (TL Tet)), weighted by X. This term regularizes the policy update, preventing the trained policy Tg from deviating too drastically from a frozen reference policy Ref, which is essential for maintaining training stability.
16

---

Published as a conference paper at ICLR 2026
B.1 CANDIDATE GENERATION (ALGORITHM
The candidate generation process is critical filtering cascade designed to efficiently narrow down the vast memory repository M small of promising candidates C. This is achieved through two-stage process:
Temporal Filtering. First, we leverage powerful LLM to perform a zero-shot prediction of the likely temporal window (Isarts lena) relevant to the user query We then perform an initial broad-phase filtering by retaining only those sessions (T;, from M whose timestamps Tı overlap with this predicted window. This step effectively prunes the majority of irrelevant sessions based on a strong temporal heuristic.
2. Relevance Filtering. The temporally-filters subset Miemp is then passed to second filtering stage. Here, we use a fast and effective lexical retrieval method, BM25, to rank all sessions in Mremp based on their textual relevance to the query q. The final candidate pool is formed by selecting the top-ranked sessions from this list. This cascade approach- using a temporal heuristic followed by lexical matching-allesn allows for an efficient and effective reduction of the search space without relying on expensive semantic models at a large scale.
B.2 MULTI-LEVEL REWARD CALCULATION (ALGORITHM 3
To provide a rich and informative learning signal for our policy, we designed a multi-level reward function that captures three critical aspects of the task. The final reward R is weighted sum of these components.
Task-level Accuracy Reward (Ra). This is sparse, binary reward that directly measures task success. It yields a reward of if the generated answer a is correct with respect to the ground-truth answer a", and otherwise. This component ensures the model is strongly incentivized to produce factually correct final answers.
Evidence Grounding Reward (Ry). This component evaluates the quality the retrieved evidence. We calculate the Fi-score between the of session IDs in the predicted evidence set S and the ground-truth evidence set M This dense reward encourages the model to select the precise of sessions required formulate the answer, promoting better interpretability and faithfulness.
3. Temporal Consistency Reward (Rt). This novel reward component assesses the temporal quality the selected evidence with respect to the ground-truth temporal range Ig. is computed as the average of individual rewards over all selected sessions. For each session U E S, the reward a weighted sum of two sub-components:
Chronological Proximity (R,): This measures the temporal distance between the ses- sion's timestamp U and the gold range Iq. It uses logistic function to provide a soft, differentiable penalty, rewarding close proximity and penalizing distant sessions.
Chronological Fidelity (Ry): This provides more fine-gr fine-grained signal. Within given session U, it assesses whether the events in the utterances are relevant to the query are themselves temporally aligned with the gold range lq. returns positive reward for relevant utterances inside lq. a smaller positive reward for those on the boundary, and negative penalty for those outside.
The final reward R is the weighted sum w. Rg + Ry.. + w, Rt, where the weights allow to balance the relative importance of task accuracy, evidence quality, and temporal alignment.
17

---

Published as a conference paper at ICLR 2026
Algorithm MEMORY-TI Training Procedure
Require:
Full dialogue memory repository M
Training dataset D [(; at : (qi))==> where M is the ground-truth evidence set, a" is the ground-truth answer, and lo is the ground-truth temporal range Policy model to be trained The and frozen reference policy Rre Hyperparameters: KL divergence weight A, reward weights Wa, Wg. Wt. group size G
Ensure:
Optimized policy model Ter
1: function TRAINMEMORY-TI(MM. D, No, Tref)
2: Initialize policy parameters
3: for each training iteration do
4: Sample a batch (g. M a,. IQ) from D
D Phase Candidate Generation
5:
C x- GenerateCandidates//
D Call Algorithm|
D Phase 2: RL Fine-tuning
6:
SampledOutputs + for j = to G do
7:
> Sample K outputs from the policy Generate output string from conditioned on (q. C)
8:
9:
Parse the selected evidence set S, and answer from the string
Add aj) to SampledOutputs
11:
end for Rewards +
13:
for j = 1 to G do
>> Calculate reward for each sample
R f CalculaeReward((S,..))) (M", IQ))
Call Algorithm
15:
Add R to Rewards
end for
17:
for j A, R, - Σ R
1 to G do
> Calculate Advantage
18:
GRPO's batch-average baseline
19:
end for Policy Update Update model parameters 0 using the GRPO objective: x0 >
20:
(oJ(0))
Va log TH ((S),a,) (q. C), D (eet)
22:
end for
23:
return the trained policy model Te
24:
end function
C REWARD SUPPLEMENTARY
C.I HYBERPARAMETEE DESIGN
The Temporal Consistency Reward R, = =R,, + 3Ry and its component R, are governed by set of hyperparameters (c, d, m, at 9). The function of each parameter group remains the same: Tolerance and Leniency (m, s) defines the "softness" of the temporal alignment. The tolerance margin (m)
grace period, while the scale factor (s) control the sharpness of the penalty curve outside this margin. Incentive Scaling (c, d): controls the magnitude of the reward and penalty, allowing us calibrate the strength of the positive and negative incentives. Component Weighting (a,, 3): balance the importance of chronological proximity (R2) versus chronological fidelity (Rt)).
C.2 SNSSTIIV ANALYSIS
To assess the sensitivity of the model to the composition the reward function, we evaluate its performance under four different weight configurations for accuracy (w.), evidence grounding (w,),, and temporal consistency (wy), with results Figure 6 We find that optimal performance (67.0%) achieved with the configuration (0.6, 0.2, 0.2). This result suggests that while task accuracy is the primary objective, substantial weights for both evidence grounding and temporal consistency are essential to guide the reasoning process of the agent effectively. Deviating from this balance leads
18

---

Published as a conference paper at ICLR 2026
Algorithm 2 Candidate Generation
Require:
User query !
Full dialogue memory repository M = (TN)).....,,....................
Ensure:
Candidate session pool C.
1:
function CENNERATCAANDDDATSS.....
I. Temporal Filtering
2:
Predict target temporal window (faun, bend) for query g using an LLM
3: Miemp 0
4:
for each session (r;, S;) in M do
5:
if timestamp Ty overlaps with (tstart: fend) then
6:
Mempp + Muep U {TT,, S:)]
7: end if
8:
end for
Relevance Filtering
9:
Rank all sessions in Mump by textual relevance to query g using BM25
10: + Select top-ranked sessions from the sorted list
11: return C
12:
end function
Table Heuristic configuration of hyperparameters for the temporal reward function.
Parameter(s)
Value
Rationale
d
1.5, 0.5
Normalizes the maximum reward (c d) to 1, bound- ing the reward R. to the range (-0.5, 1]. This pro- vides a strong positive signal for perfect match and a moderate penalty for distant ones.
m
(days)
Based on the domain knowledge that a one-week win- dow is a reasonable span for contextual relevance in conversational data.
Set to default value to create a standard and pre- dictable logistic decay curve without excessive sharp- leniency.
0, B
0.5, 0.5
Establishes robust baseline by giving equal impor- tance to the two sub-rewards: chronological proxim- ity (R2) and chronological fidelity (R2)).
Wa Wgg Wt
0.6, 0.2, 0.2
Selected based on extensive experiments to balance accuracy, evidence grounding, and temporal consis- tency.
to clear degradation in performance. An accuracy-skewed weighting of (0.8, 0.1, 0.1) diminishes the influence of our guiding rewards, causing the score to drop to 64.0%. Similarly, a uniform distribution (3. 3. proves suboptimal (62.2%), likely because it fails to sufficiently prioritize the main task goal. These findings underscore that the reward components are synergistic; peak performance hinges careful balance rather than maximizing any single objective in isolation.
C.3 ACCURACY REWARD (Ra)) METRICS
The Accuracy Reward (R.) evaluates the correctness of the final answer, tailored for four main types. Each metric yields Score in the range [0, 1], which then normalized to the final reward R. E [-.
Option Answers (Exact Match, EM) For categorical answers (c.g., "A". "A C"), we use strict Exact Match. The score is defined Scoreem = T(Apred Agola), where T(.)) is the indicator
19

---

Published as a conference paper at ICLR 2026
Algorithm 3 Multi-Level Reward Calculation
Require:
Generated evidence and answer (S, a)
Ground-truth evidence, answer, and temporal range (M", a,", IQ)
Reward weights Way W'gs We; Ox; 1
Ensure:
Total scalar reward
1: function CALCULATER AELLUULATRREARRI a), (M", a", ,Q))
D Task-level Accuracy Reward (Ra)
2: Ra + (1 if is correct
a* else 0)
D> 2. Evidence Grounding Reward (Rg)
3:
Rg +Fl_ [i_score((s)) FI_scoreesession (session ids(S), session ids))
D 3. Temporal Consistency Reward (Ra)
4:
Riat +-
5:
if S is not empty then
6:
for each selected session E S do
Chronological Proximity (Rt)
7:
x - (gaa(U,,1)) m)/3
8:
Rg c/ + cxp(xz))
b. Chronological Fidelity (R₃)
9:
UTe {u U text similarity q) threshold]
10:
Uual > then
Ry -- 1
[haa EnEuna [E,]] EEEE, rele; D re scores events (+1, (+1,+0.5,- +0.5,
based overlap else
12:
13:
RS +-
14:
end if
RU, 1Q) t aR, 33, Re.total & Re,toral + (U,UU end for
16:
18:
R₂ + Re.una//
Average over all selected sessions
else
20:
R. +0
21:
end
Combine for final reward
22:
R- W. Rug + W. Rg + we Re
23:
return R
24:
end function
function. For instance, if the gold answer Agoua is "B". prediction of "B" scores 1, while "C" scores
0.
Timestamp Answers (Unit-aware Accuracy) To handle various date/time formats, this metric compares canonical representations. Both prediction and ground truth are normalized via a function (-.)) before comparison, making the score robust to format differences:
ScoretnitAward (Apred, Agola) I(N = N N(Aolaa)
For example, prediction of "2025-09-24" correctly matches the gold answer "September 24, 2025," as both normalize to the same value, yielding a score 1.
Time Interval Answers (e-Exact Match, e-EM) tolerance e for minor calculation differences. The score is the absolute difference between the predicted value V(Apred) and the gold value V(Agota) is within this tolerance:
For numerical durations, this metric allows
FM Agota) (Appe)) (AAgoaa) <)
For gold answer of "13 days" and with e = 1, predictions from "12" to "14 days" are considered correct.
20

---

Published as a conference paper at ICLR 2026
100
Wi 49.9
78.1
82.4
67.0
Roward Wught Combiiin mamionaioon
W2-
77.6
82.4
Table PPO vs. GRPO: Fl performance on Memory-Tl TI models of different sizes.
Categury
Category
Category
3B, GRPO
49.5
79.5
80.3
66.9
W4
42.9
75.4
78.3
62.2
3B, PPO
41.2
61.7
68,7
54,5
(-16.8%)
(-22.4%)
(-14.4%)
(-18.5%)
Category Category Calegory Overall
3B. GRPO
49.9
78.1
82.4
67.0
Figure 6: Sensitive analysis. Heatmap level-wise performance. W1. W2. W3. W4 correspond to reward weights com- -00.6, 0.2,,
3B, PPO
(+9.2%)
54.5
(-28.45%)
55.9
(-30.6%)
57.2
(-17.0%)
55.6
bination (Wa, Wgg wt) (0.5, 0.25, 0.25), (0.8, 0.1, 0.1), 3:
Been2: 5-38 (Instruct)
GeenZ 5-76 C1 (lectruct))
8k
$1.7
9.7
45
25.
18.0
22.2
31
50,
33.3
8.3
25.
40.
14.0
40
0
32
0
4P
22
10.4
21.2
75
16k
59.4
13.
36.
22
35:
30.0
44
10.4
38.0
16k
47.0
11.2
50
41
12.8
33
37.5
44
55.0
8.3
33.0
50
32K
40.8
12
20
0
27
22.
53
40
14.0
42.0
50.0
1L
44
1
Overall
Gon
(TB)
100
100
12.4
75
16k
44.2
22.2
68
63.3
61.1
14.0
42
32k
41
7A
84
17.9
82K
26.0
0.0
60.
44.4
18.0
$1
87.5
100.
83.3
18.7
47.5
25
128k
64k
47.0
13
11
40
41
81.
88
94
12.5
55.3
25
67F
Orarel1
Gvaral1
Go
Figure 7: The impact of memory context length temporal reasoning: Fl performance comparison of Qwen2.5 models and Memory-T1 across context windows of 8k, 16k, 32k, 64k and 128k tokens (retaining the nearest context to the query).
Sequential Answers (Hamming Accuracy) To award partial credit for ordered lists, we use Hamming Accuracy, which is the fraction of correctly positioned items. For prediction Apred (Pt....,PIt) and gold sequence Agold = (g1....,91. of length T., the score is:
1
ScorcHamming (Apeed, Agola) T,
Σ
Upp =g:)
i=1
For example, if Agold is "(1), (3), (2), (4)" and Apred is "(2), (3), (1), (4)", the score is }. only the second item and fourth item are correct.
Final Reward Normalization The final reward R. is designed to strongly penalize completely incorrect answers while directly rewarding any degree of correctness. an answer is entirely wrong (Score 0), it receives reward of -1. For any partially or fully correct answer (Score 0), the reward is equal to the score itself. This is formulated as:
--1
if Score = 0
R, =
Score
if Score >> 0
C.4 COMPARATIVE PERFORMANCE OF GRPO AND PPO
Compared with GRPO, PPO generally underperformed across most categories (Figure For the 3B models, PPO showed substantial declines relative GRPO, with reductions -16.8% in Category
21

---

Published as a conference paper at ICLR 2026
Owen2.5_38,.0 instruct
iresttutt
MamoryTT, siilaarity MemoryT1
107
5oore
Col A01
0.0
OR
Commp:
overa
Figure 8: Impact of text similarity filtering component in chronological fidelity reward.
A, -22.4% in Category B, -14,4% in Category C, and 18.5% overall. For the 7B models, PPO achieved modest improvement in Category A (+9.2%), but suffered marked decreases in Category B (-28.4% and Category C (-30.6%), leading to a - 17.0% drop overall. These results indicate that GRPO provides more stable gains across categories, whereas PPO less consistent, especially for more challenging tasks (Category B and C).
C.5 TEMPORAL REASONING UNDER CONTROLLED CONTEXT WINDOWS.
To investigate how the "lost-in-the-mee middme" problem (Liu et al. 2024 Wang et al. 2025 affects temporal reasoning, we conduct controlled experiment truncating input context various window lengths (Figure 7). The baseline model peaks at a "sweet spot" (e.g. 32k tokens) and then collapses as the context grows longer, failure caused by attentional dilution. In contrast, our MEMORY-TI framework is completely resilient to this effect. This resilience stems from our coarse-to-fine candidate generation, which filters the noisy history down to a concise and relevant evidence set. By shielding the final agent from irrelevant context, our framework maintains high and stable performance even when the original context exceeds 128k tokens. With this clean, high- quality context, the specific impact of the RL-tuned agent becomes clear. The fine-tuning is highly targeted: enables the agent to achieve near-perfect "mastery" on specific complex reasoning tasks, with MEMORY- T1 exceeding 0.9 F1 scores Order Reasoning (OR), Range Reasoning (RR), and Contextual Temporal Filtering (CTF). Conversely, the near-zero scores Comparison (Comp) and Timeline (TL) highlight the limitations of the current agent paradigm on tasks requiring deeper compositional logic. Finally, we observe synergy between model scale and fine-tuning, with the RL policy acting as a more powerful performance multiplier on the more capable 7B base model.
C.6 IMPACT ON TEXT SIMILARITY IN REWARD
Ablation results, as shown in Figure g clearly demonstrate the pivotal role of the text similarity reward. When this component is present, the model learns to filter out irrelevant dialogue history, thereby anchoring temporal spans more precisely and improving performance on duration-sensitive subtasks. Once the similarity reward is removed, performance on duration computation (DC) and compositional reasoning (Comp.) drops sharply, indicating that the model struggles to maintain accurate temporal spans without explicit guidance to suppress noise. Although slight gains appear in tasks such as Loc. and Ext., these are outweighed by the decline in -eeeedeen metrics. This suggests that text similarity primarily functions as a noise-reduction mechanism, ensuring that the reasoning process remains grounded in relevant context, which is especially critical for complex temporal reasoning tasks.
22

---

Published as a conference paper at ICLR 2026
Table 9: MemAgent, Time-RI model performance comparison: RAG vs. Non-RAG Settings
Model Family
Params
Setting
Score
Setting
F1 Score
Timc-RI R1
3B
RAG
31.4
Non-RAG
29.2
MemAgent
7B
RAG
37.6
Non-RAG
40.2
Memory-T7
3B
RAG
36.7
Non-RAG
37.7
C.7 SUPPLEMENTARY EXPERIMENTS:
GENERALIZATION
Memory-T1 (3B) demonstrates strong OOD generalization (Table 9) achieving 37.7% (Non-RAG), which is significant improvement over Time-RI (29.2% and nearly matches the larger MemAgent (7B) (40.2%). This high performance, particularly in the Non-RAG setting, suggests that Memory- TI's learned policy provides a superior internal memory management and reasoning skill that highly effective and robustly generalizable across domains, outweighing the benefit of RAG ob- served in other baselines.
D LLM USAGE
We utilized large language models support both manuscript polishing and data annotation. In particular, the GPT-4o API is employed to assist with the annotation of the Time-Dialog dataset. Further details of this process are provided Appendix
Localization
Type: Localization
Format: time_span
span
Level: level_
Question When is Debra Ryan working starting her business?
Options:
Answer: 8:35 pm, February 21, 2020
Figure 9: Localization subtask.
Duration Comparison
Type: Duration Duration_Compare
Format: single_choice
Level: l
Question Which of the following two durations longer?
Options: A. Duration 1 is longer.
Duration 2 is longer.
C. The two durations are approximately the same length.
Answer: A
Figure 10: Duration Comparison subtask.
23

---

Published as a conference paper at ICLR 2026
Computation
Type: Computation
Format: time_span
Level: level_
Question How long between Debra Ryan going skydiving and India Brown attending
a street art fest in Brazil?
Options:
Answer:
19 days
Figure Computation subtask.
Order Comparison
Type: Order_
Order_Comppr Compare
Format: dingle_choies
Level: level_
Question For Factl: India Brown became a Queen fan, and Fact2: India Brown found flowers by lake in the park., which one happened earlier?
Options: A. Fact happened earlier.
B. Fact 2 happened earlier.
C. They happen at almost the same time.
Answer: A
Figure 12: Order Comparison subtask.
Extract
Type: Extract
Format: single_choice choice
Level: level_1
Question Which of the following are time expressions mentioned in the context?
Options: A. April 17, 2021
B. 2018
C. March 16, 2020
D. March 14, 2019
Answer: C
Figure 13: Extract subtask.
Explicit Reasoning
Type: Reaioiii_Re
Format:
single_choice
Level: level_2
Question What notable artistic or outdoor activities did India Brown participate between
April 1, 2020, and April 9, 2020?
Options: A. India Brown attended street art fest in Brazil.
B. India Brown took photo of feather and shells beach.
C. India Brown went hiking and sketching at nearby national park.
D. India Brown received positive feedback her artwork.
Answer: B
Figure 14: Explicit reasoning subtask.
24

---

Published as a conference paper at ICLR 2026
Order Reasoning
Type: Lnder_RRasoning
Format: single_choice
Level: level_2 _
Question What was India Brown's third teaching engagement in 2020?
Options: A. Running a painting workshop for kids.
B. Teaching art at an orphanage in Cambodia.
Conducting live demonstration for her college art club.
D. Instructing pottery class at a local studio.
Answer: A
Figure 15: Order reasoning subtask.
Relative Reasoning
Type: Relative Reasoning
Relative_Reasoning
Format: Single_choice
Level: level_2
Question What India Brown's most recent job before 12:00 am, March 09, 2020?
Options: A. New series abstract artworks.
Travel guide based her trip experiences.
C. New painting technique from street art festival.
D. Testing watercolors for her new series.
Answer: A
Figure 16: Relative reasoning subtask.
Counterfactual
Type:
Counterfactual
Format:
single_ choice
Level: level_3
Question What notable artistic or outdoor activities did India Brown participate in between
April 1, 2020, and April 9, 2020, she visited the Louvre in Paris in March 2020?
Options: A. Mini soap sculpture.
Photo of feather and shells.
C. Photograph in Santorini, Greece.
D. Sketched waterfall during a hike.
Answer: B
Figure 17: Counterfactual reasoning subtask.
Co-temporality
Type: Co Ca__eeo_ooooe jo_temporaltty
Format: single_choice
Level: level_3
Question At the same time as Debra Ryan is learning to play the guitar, what collection does India Brown have?
Options: A. Soap sculptures.
Watercolor paintings.
C. CDs.
D. Vinyl records.
Answer: C
Figure 18: Co-temporality subtask.
25

---

Published as a conference paper at ICLR 2026
Table 10: Qualitative analysis of subtasks showing significant improvement (over 10%) in Memory- TI. (e.g.. Qwen2.5-3B Instruct Model: (Loc.): 0.278 + 0.500, (ER.): 0.692 + 0.821, (OR.):
0.333 - 0.667. (RR.): 0.563 0.875, (CTF.): 0.556 0.889, (Co-tmp.): 0.778 0.944)
Answer
Answer Qwen2.5-3B (Wrong)
Memory-
Subtask Question
Options
T1
(Correct)
8:35
9:32 pm, May 20, 2020
Loc.
When is Debra Ryan N/A starting her own busi- ness?
February
21, 2020
ER.
What notable artistic or outdoor activitics did India Brown participate
A. India Brown attended street art fest B
C
in Brazil.
B. India Brown took photo of a feather and shells beach.
in between April 1, 2020, and April 2020?
9,
C. India Brown went hiking and sketch-
ing at a nearby national park.
D. India Brown received positive feed- back her artwork.
OR.
What was India Brown's 's third teaching engagement 2020?
A. Running a teaching workshop for B kids.
A
B. Teaching art an orphanage Cam- bodia.
C. Conducting a live demonstration for her college art club.
D. Instructing pottery class at local studio.
RR.
What Brown's most recent job before 12:00 am,
India
A. India Brown is working new se- ries abstract artworks based on her trip.
B
A
March 09, 2020?
B. India Brown working as travel guide based on her trip experiences. C. India Brown is working a painting technique learned at street art festival.
D. India Brown is testing watercolors for her new series of abstract artworks.
CTF.
What notable artistic or outdoor activities did India Brown participate in between April 1, 2020, and April 9, 2020, if she visited the Louvre in Paris in March 2020?
A. India Brown carved mini sculpture C from soap bar.
B
B. India Brown took photo of feather and shells on a beach.
C. India Brown took a photograph in Santorini, Greece.
D. India Brown sketched waterfall during a hike.
Co- tmp.
At the same time as Debra Ryan learn- ing to play the guitar, what collection does In- dia Brown have?
A. India Brown has collection of B tercolor paintings. B. India Brown has a collection of wa- tercolor paintings.
C
C. India Brown has collection of CDs.
D. India Brown has collection of vinyl records.
26

---

Published as a conference paper at ICLR 2026
Timeline
Type: Timeline
Format: event_order
Level: level_3
Question Below are 8 facts. You need sort these facts in chronological order.
Options: (1) New painting technique. (2) Shared mural image. (3) First art show. (4)
Became Queen fan. (5) Invited to exhibit. (6) Beach photo. (7) Sketched waterfall. Received feedback.
Answer: (415)))11)11221((66)
Figure 19: Timeline subtask.
27

---

Published as a conference paper at ICLR 2026
Prompt for Event Extraction and Time Coverage Annotation
You a precise temporal reasoner that analyzes utterances in multi-turn dialogue. Your goal to analyze each individual utterance, based on content and prior dialogue history, and extract:
One or more events described in the utterance
For each event:
A short summary of the event being described
The estimated time range of that event
The recurring pattern (if applicable) of that event
Input:
Session start time: [session_start_tiing
Dialogue history: (dialogue_hissorry)
Current utterance: (target_utterance)
Current speaker: (speaker)
Reasoning Rules:
Event Time Range Estimation:
Explicit date (e.g., "August 14"): use full-day range -> start: 00:00:00, end:
23:59:59
"yesterday": the day before the utterance time "last week": 7 days ending day before the utterance time
Past tense, no time mentioned: start = "unknown", end utterance time
Future tense: start = utterance time, end = "unknown"
Habitual/ongoing action:
start "unknown", end = "unknown", mark recur-
rence
Recurring Field: choose from none (default), daily, weekly, monthly, yearly, habitual
Output Format (JSON):
"speaker": "Debra Ryan", "utterance": "I took this photo last week.", "event_sunmary" "Debra took a photo", timee: " 077.:-2......000.....000000000.3 ["2000-0-00100::0::000
"recurring": "none"
"speaker": "Debra Ryan", "utterance":"1 met friend who was visiting
from out of town. ",
"event_suree devent_sunmary": "Debra met a visiting friend"
"event tivent_ttmm": ("tr==iiiiiin_000000000000000000 |"2020-02-01100:0000000 "2020-02-07733:5:550
"recurring": "none"
Ensure your output is valid ISON. Only output the JSON, no extra text.
Figure 20: Prompt used for event extraction and temporal coverage annotation.
28

---

Published as a conference paper at ICLR 2026
Prompt for Question-based Event Reasoning
You precise temporal reasoner that analyzes user's question. You given the user's question.
Input:
User's question: (user_question)
Reasoning Rules:
Event Time Range Estimation:
Explicit date (e.g., "August 14"): use full-day range -> start: 00:00:00, end:
23:59:59
"yesterday": the day before the utterance time
"last week": 7 days ending day before the utterance time
Past tense, no time mentioned: start = "unknown", end (current_time_ste
Future tense: start [current_timen seveenttttm_stttee end "unknown"
Habitual/ongoing action: start "unknown", end "unknown",, mark recur- rence
Recurring Field: choose from none (default), daily, weekly, monthly, yearly, habitual
Output Format (JSON):
"question": "What creative or social activities did India Brown participate in between April 16, 2020, at 06:22 and April 19, 2020, at 07:22?", "time_range": ["2020-04-16106:22:00 "recurring": "none"
"2020-04-19100:22:000
Ensure your output is valid JSON. Only output the JSON, no extra text.
Figure 21: Prompt used for time range annotation over user questions.
29

---

Published as a conference paper at ICLR 2026
Prompt for Fact-Evidence Alignment
Determine which utterance contains the most relevant evidence that supports each of the given facts.
Your task:
Input:
Facts: [facts_list,
Sessions: (sessions_datae
Output: Return the most relevant utterance for each fact using the following format:
"Fact_evidence":
_bact_index: 0, "session_id"" "session_id", "utterance_ide id"; "id"
"fact_index" 1, "session_id"; "session_id", "utterance_id"" id"; "id"
b
Example:
"fact "Fact_evidence"" [
"fact_indexx 0,
"session_id"" 2,
"utterance_id"" id": 3
Constraints:
Only include utterances that clearly support the fact (no hallucination inference beyond what's stated).
Select exactly ONE most relevant utterance per fact.
If utterance supports fact, return null for that fact.
fact_index corresponds to the index in the facts list (0-based).
session_id must be one of the provided session IDs.
Figure 22: Prompt used for fact-evidence alignment multi-sessin dialogues.
30

---

Published as a conference paper at ICLR 2026
Prompt for Memory-Tl Training
You are memory-aware reasoning assistant. Your task to answer temporal questions based on multi-tum dialogue history. Carefully analyze the provided context, reason about time and events, and respond strictly in JSON format.
The required ISON structure is:
{
selected_meeorrg" ""ssssio___ _"" "session_Y"" "answer": "X")
Answer Format Rules (by type):
1. Single choice: A, B,
2. Multiple choice: A E (space-sepel separated)
3. Time: "HH:MM:SS am/pm,, Month DD, YYYY"
Example: "02:30:00 pm, March 22, 2024"
4. Sequence: (1(3))2)44)))))0117)
Input format:
previous_memory memory -edddaalouusssssionns /prees//ppreeeeeeeeeeeee
question >
Time (current_lime)
Question: {question]
/question
Output example:
{
" selected memory" I' ["sssiion__" "session_""] "ansser" A"A") }
Figure 23: Prompt used for training Memory-T1,
Prompt for Evaluation
You are presented with temporal question and previous memory, please answer the question with the correct format. The last line of your response should of the form: Answer:
SAnswer, where SAnswer is the answer to the problem.
Output requirements:
1. Single choice: ||.. (uppercase)
Multiple choice: A B C (space-scparated uppercase)
3. Time: HH:MM:SS am/pm, Month DD, YYYY
Example: 10:45:41 pm, January 15, 2024 4. Sequence: (11)))244)))))))
Input format: previous_memory [dialogue_sesssoons] o http://seeeeesooooooppppreoooeee hppeeiiousmmmmoryy
>
question
Time courent_timat
Question {question]
</question
Remember to put your answer on its own line after 'Answer':
Figure 24: Prompt used for evaluation of temporal reasoning tasks.
31


Reviews:
[{"review_id": "aEyuBTaFvq", "reviewer": "Reviewer_XvHc", "summary": "This paper proposes Memory-T1, a two-stage framework for temporal reasoning over long, multi-session dialogues. Stage-1 performs coarse-to-fine candidate generation: an LLM predicts the query’s temporal window to hard-filter sessions, then BM25 ranks by textual relevance to form a high-recall pool. Stage-2 fine-tunes a policy with GRPO to jointly pick evidence sessions and produce the answer, supervised by a multi-level reward that combines task accuracy (Ra), evidence grounding via session-set overlap (Rg), and a temporal-consistency reward (Rt) with session-level proximity (Rs) and utterance-level temporal fidelity (Rf).\n\nOn Time-Dialog, Memory-T1 improves a 3B model to 66.9% overall, and the 7B variant reaches 67.0%, outperforming larger baselines. The ablations attribute gains to the temporal-consistency and grounding rewards; removing them substantially degrades performance. The method shows strong OOD results on LoCoMo (37.7% overall without RAG), and robustness up to 128k tokens.", "questions": "See weakness session", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. Timely and important: Tackles temporal reasoning over long multi-session dialogues—an increasingly central capability for agentic systems and memory architectures.\n\n2. Well-motivated reward design for GRPO: Clear decomposition into Ra/Rg/Rt; Rt thoughtfully mixes proximity and fidelity with soft penalties and explicit hyper-parameters; weights and sensitivity are reported.\n\n3. Coarse-to-fine retrieval is executed cleanly with a precise time filter then lexical ranking; the analysis of top-k and recall is helpful.\n\n4. Clear writing and comprehensive experiments: SOTA on Time-Dialog, detailed ablations of reward components, OOD evaluation (LoCoMo), and long-context stress tests.", "weaknesses": "(W1) Table 2 interpretation of Rt/Rs/Rf effects\n\nThe paper states that removing Rs or Rf yields a trade-off aligned with task difficulty (Category-A “simpler” tasks improve, while B/C “complex” tasks degrade). However the full removal of Rt (both Rs and Rf) does not show a monotone extension of the same trend.\nIf Category-A improvements under −Rs/−Rf are due to “easier” temporal structure, why does removing the entire Rt lower Category-A below the full model instead of amplifying that benefit? Two possibilities worth probing: \n1. Reward-interaction / reweighting hypothesis. With wt=0, the optimizer may over-fit Ra+Rg. E.g. Ra for Category-A can reward surface-level answers\n2. Credit-assignment : GRPO with a batch-average baseline can shift exploration when a dense component disappears. Without Rt, advantages may over-credit generations that incidentally match Ra/Rg\n\n\n(W2) Table 3 (LoCoMo) and how RAG is integrated with Memory-T1\n\nTable 3 shows Memory-T1 is best without RAG overall (37.7% vs. 36.7%), supporting the claim that the policy “manages memory” well. But on the Adversarial subset, RAG helps Memory-T1 quite noticeably (29.8 with RAG vs. 26.0 without). Why? Can the authors explain this part? \nHow, precisely, is RAG layered over Memory-T1’s time filter + BM25 + RL policy (i.e., is RAG material merged before/after candidate generation, and does it change the action space for evidence selection)? Please detail. \n\nOne possibility here: distribution-shift hypothesis -- the RL policy learns a specialized memory manager for in-dialogue, time-anchored evidence. Injecting external RAG text can introduce off-distribution segments with weak or missing temporal anchors, which lowers precision overall.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T6yvbDE2fo", "reviewer": "Reviewer_mMws", "summary": "This paper presents Memory-T1, a reinforcement learning framework designed to improve temporal reasoning in long, multi-session dialogue settings. The method introduces a two-stage process: a coarse retrieval stage that filters relevant sessions by predicting temporal scope and ranking past interactions, followed by a fine retrieval stage where a reinforcement learning agent selects evidence guided by multi-level rewards. The model uses three reward components related to answer accuracy, evidence grounding, and temporal consistency, aiming to enhance both factual correctness and chronological alignment. Experiments on Time-Dialog and LoCoMo show that Memory-T1 improves temporal reasoning accuracy and robustness under long-context conditions.", "questions": "- How sensitive is the model to the relative weights of the three reward components?\n- Were the weights tuned manually or derived from validation metrics?\n- Could the method work in domains without explicit timestamps or structured temporal annotations?\n- How much of the observed improvement comes from the reward design itself versus better retrieval filtering?\n- Would the learned policy transfer to different backbone models or datasets?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "The paper targets an important and underexplored aspect of dialogue modeling, namely temporal reasoning across multiple sessions. The proposed coarse-to-fine retrieval framework is intuitive and improves retrieval efficiency under long histories. Integrating reinforcement learning provides a structured way to optimize multiple supervision signals jointly. The experimental results demonstrate strong improvements on established benchmarks, and the ablation studies highlight the contribution of each reward type. The paper is clearly written, and the motivation is easy to follow.", "weaknesses": "The methodological novelty is moderate. The framework largely combines existing reinforcement learning and retrieval techniques with additional temporal supervision. The temporal consistency reward is well-motivated but not particularly new, and its formulation appears heuristic. The paper does not provide enough justification for the chosen reward weights or a systematic analysis of their sensitivity. The ablation results are limited and do not clearly establish whether improvements stem from the reward design itself or from better retrieval heuristics. The method also assumes access to accurate timestamps and event annotations, which may limit applicability to real-world data where such information is noisy or missing. Finally, the evaluation focuses on QA-style reasoning without exploring whether the learned policy improves temporal coherence in free-form dialogue generation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xHI2t8wTUo", "reviewer": "Reviewer_rTBM", "summary": "This paper presents Memory-T1, a reinforcement learning framework designed to improve temporal reasoning capabilities in multi-session conversational agents. The authors address a critical challenge: existing long-context models struggle to accurately identify temporally relevant information from noisy, extended dialogue histories. The framework employs a two-phase architecture: 1) Candidate Generation: LLM first predicts its target temporal window followed by relevance-based retrieval (BM25) to narrow the search space; 2) Fine-grained Selection: Employs an RL agent trained with GRPO to select precise evidence sessions. A multi-level reward function is designed to jointly optimize the evidence selection and answer prediction. The framework achieves 67.0% overall score on the Time-Dialog benchmark with Qwen-2.5-7B, outperforming a Qwen-2.5-14B baseline by 10.2%.", "questions": "What happens when the LLM-based temporal filter incorrectly predicts the query time scope and excludes ground-truth evidence sessions? Does the framework include any mechanism to allow error recovery from such retrieval failures?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The coarse-to-fine retrieval strategy combined with multi-level RL rewards is well-justified. The temporal consistency reward design enhanced the temporal reasoning and evidence selection of the model to better predict the answer.\n- Ablation studies (Table 2) provide clear evidence for each reward component's contribution, demonstrating the importance of individual design choices.", "weaknesses": "- The primary evaluation is conducted on the in-domain Time-Dialog dataset, where Memory-T1 is trained in-domain while other baselines (Time-R1, MemAgent) are evaluated in a zero-shot setting. This makes direct comparison difficult. It remains unclear how Memory-T1 would perform against these baselines on unseen benchmarks such as LoCoMo, particularly in a comparable experimental setup.\n- The proposed framework requires extensive temporal annotations for training, yet the sensitivity to annotation quality and potential noise has not been explored. This raises concerns about the framework's practical applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1zpCvgEkSR", "reviewer": "Reviewer_AsHW", "summary": "The paper proposes Memory-T1, a two-stage framework that improves the temporal reasoning ability of large language models (LLMs) over long, multi-session dialogues. During the first stage, an LLM predicts the temporal window and filters sessions outside the window. Then, a retriever reranks the resulting sessions, and only the top-ranked sessions are kept. During the second stage, an agent is trained with GRPO to select the most relevant sessions. The reward is a combination of accuracy, evidence grounding, and temporal consistency. The approach is applied to Qwen2.5-3B and Qwen2.5-7B-Instruct, and compared against diverse baselines.", "questions": "-  Memory-T1 does not seem to improve with model size. What could be the reason of this limitation? Is it more about the task than the approach?\n- What is the additional latency introduced by the approach compared to the base model and the other baselines? \n- Have you observed any reward hacking that you had to circumvent? Was it associated to particular reward or resulted from the combination of the rewards?\n- Did you have to perform extensive hyperparameter search to obtain the results in the paper?\n- Is it possible to fully automate dataset annotation?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses an important problem which is temporal reasoning on long, multi-session dialogues.\n- The paper is clear and easy to read. The appendix contains additional details about the dataset and setup. \n- Memory-T1 outperforms existing baselines, generalizes out-of-domain and is robust to long-context scenarios. \n- The ablation study emphasizes the contribution of each reward.", "weaknesses": "- Unlike the base models, the performance when applying Memory-T1 does not seem to increase with model size (from 3B to 7B). \n- Training requires a heavy annotation of the dataset which can be very expensive. \n- There is no qualitative analysis showing the kind of mistakes of the base model that Memory-T1 solves.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)