MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent
Authors: Hongli Yu
Tinghong Chen
Jiangtao Feng
Jiangjie Chen
Weinan Dai
Qiying Yu
Ya-Qin Zhang
Wei-Ying Ma
Jingjing Liu
Mingxuan Wang
Hao Zhou
Venue: ICLR 2026 Oral (2026)
Keywords: LLM
memory
agent
RLVR
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose MemAgent, a novel agent workflow for long-text processing,  demonstrating exceptional extrapolation and performance in large-scale tasks after RL Training.

Abstract: Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents without performance degradation during extrapolation remains the ultimate challenge in long-text processing. To solve this problem, We introduce a novel agent workflow, \method, which processes text in segments and updates memory through an overwrite strategy, addressing the challenge of long-context task through enhanced memory management. We further extend the DAPO algorithm to directly optimize memory ability in an end-to-end fashion, facilitating training via independent-context multi-conversation generation. Experimental results demonstrate that MemAgent has superb long-context capabilities, being able to extrapolate from an 8K context to a 3.5M QA task with a performance loss of less than 10\% and achieving over 95\% on the 512K NIAH test.

Published as a conference paper at ICLR 2026
MEMAGENT: RESHAPING LONG-CONTEXT LLM WITH MULTI-CONV RL-BASED MEMORY AGENT
Hongli Y23
Tinghong Chen'
Jiangtao Feng'
Jiangjie Chen
Weinan Dai
Qiying Y23.2,3
Ya-Qin Zhang
Wei- Ma
Jingjing Liu
Mingxuan Wang 2.3+
Hao Zhou 1.3+
1 Institute for Al Industry Research (AIR), Tsinghua University
2 ByteDance Seed
SIA-Lab of Tsinghua AIR and ByteDance Seed
ABSTRACT
Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents without performance degradation dur- ing extrapolation remains the ultimate challenge in long-text processing. To solve this problem, introduce novel agent workflow, MEMAGENT, which processes text in segments and updates memory through an overwrite strategy, addressing the challenge of long-context task through enhanced memory management. We further extend the DAPO algorithm to directly optimize memory ability in an end-to-end fashion, facilitating training via independent-coneee tion. Experimental results demonstrate that MEMAGENT has superb long-context capabilities, being able extrapolate from an 8K context to 3.5M QA task with performance loss of less than 10% and achieving over 95% the 512K NIAH test.
culti-conversation genera-
50
50
RL-Meaagni--U
Qwennarig L1
Qeren2 -ithatruut-/---11
Ceren2.22 1am....-tattuu-7---3
Natil--weena2 328
Qwan 148
Thincation
Context Length Tokens
Figure 1: Accuracy of RULER-HQA (Hsieh et al., 2024; Yang et al., 2018) Even models that employ long-context continual pretraining and extrapolation techniques fail to maintain consistent performance. In contrast, MEMAGENT with RL only demonstrates marginal performance dropping.
INTRODUCTION
While having demonstrated impressive capabilities (OpenAI, 2024; DeepMind, 2024; XAI, 2024; Anthropic, 2024; OpenAI, 2023), industry-leved Large Language Model (LLM) systems (Anthropic, 2025; Li et al., 2025a; Liu et al., 2024; Yen et al., 2024) still face a critical challenge: how to handle long contexts effectively processing many steps, or managing the long-term memory of an agent system all these complex tasks can generate overflowing text that quickly explodes the typical-size context window of current LLMs.
entire book, executing complex chain of reasoning
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Existing approaches to long-contess tasks are three- prrneep.onged. The first involves length extrapolation methods by shifting the positional embeddings in order to extend the context window of the model (Su et al., 2024; bloc97, 2023; Chen et al., 2023; Peng et al., 2023b; An et al., 2024), plus continued pre-training (Liu et al., 2023; Xiong et al., 2023; Gao et al., 2025). Despite promising potential, these methods often suffer from performance degradation and slow processing speed due (nn)) computational complexity when applied to extremely long text. The second school of methods leverages sparse attention (Beltagy et al., 2020; Zhao et al., 2019; Xiao et al., 2023) and linear attention mechanisms (Child et al., 2019; Katharopoulos et al., 2020) to reduce the complexity of attention for more efficient processing of longer sequences. However, this typically requires training from scratch, with inherent adversities such as linear attention facing difficulties in parallel training or sparse attention depending on human-defined patterns. The last line of inquiry investigates context compression (Jiang et al., 2023; Li et al., 2023; Behrouz et al., 2024; Zhang et al., 2024), which aims to condense information in token-level or external external-memory- plugin modules. Such approaches often struggle with extrapolation, and require the integration of additional modules or context operations, which ineluctably disrupts the standard generation process and hinders compatibility as well as parallelizatinn.
Hence, a successful LLM with strong long-context capabilities requires the trinity of: 1) processing infinite length of text; 2) scaling without significant performance drop; and 3) efficient decoding with linear complexity. To pursue this quest, we return to the basic intuition behind long-context modeling (Miller et al., 1956; Hochreiter & Schmidhuber, 1997: Graves et al., 2014; Weston et al., 2014). When humans process long-contex information, we tend to abstract out the main revealing conceptions capture the essence of the whole text, often by making notes of critical details or using short-handed stenograph to record the key points, while discarding redundant and irrelevant data. We do not attempt to memorize every single fact or each small piece of information; instead, we focus our intellectual energy more important aspects of the task at hand. This selective attention not only simplifies the process but also aids tackling complex problems more efficiently.
Following this anthropocentric intuition, we propose a novel use of Reinforcement I Learning (RI.) to equip LLMs with a dynamically updated fixed-length 'memory', as illustrated in Figure 2. During inference, the LLM processes the input text segment-b--sementtt model proactively and selectively updates the memory, which then contributes to the generation aggregated and synergized in the memory. This
As it reads each segment, the
of the final output after all relevant messages clever mechanism allows the LLM to flexibly handle arbitrary text lengths while maintaining a linear time complexity during processing, since the length of the memory is fixed, which leads fixed context window size for the model. This segment-based approach generates multiple outputs from single long-text input, requiring multiple rounds of memory updates and a final round for the generation of the final response. Training this type of agent workflow, which enables dialogues across multiple independent contexts, is still an unexplored territory current LLM study. Existing systems typically handle workflow trajectories via alternating tool calls or environment feedback by either simply concatenating (Ouyang et al., 2025; Jin et al., 2025) them or using a sliding window (Feng et al., 2025) approach, which lacks flexibility and scalability in practice. Our MEMAGENT approach, instead, proposes that treats each context-independerr
conversation as an optimization objective. Based on the DAPO (Yu et al., 2025) algorithm, we implement the Multi-Conv DAPO optimize an arbitrary agent workflow by verifiable outcome reward.
In experiments, an RL-trained model with modest 8K context window (with 1024-token memory and a 5000-token document chunk) trained on 60K length documents exhibits consistently superb capabilities for Question Answering (QA) tasks documents of up to 3.5 million tokens, without performance drop and with linear computation cost. This demonstratively showcases the efficiency and scalability of our long-context memory approach.
Our major contributions are threefold:
We introduce novel approach that enables LLMS to process arbitrarily long inputs within limited context window under linear time complexity during inference, overcoming a significant bottleneck in long-context processing.
We design an agent workflow to implement this mechanism and propose an end-to-end training approach using the culti-convvrsaaionn DAPO algorithm.
2

---

Published as a conference paper at ICLR 2026
Sulving Long Context Task with Long-Comtext LLM
Conpec
14
16
lod Chunk
Quedion
LLM
Sohing Long-Conters Task with Memory Agent via RL.
Figure MEMAGENT is inspired by the way humans process long documents. divides the document into multiple chunks and allows LLMs to process them iteratively, recording relevant information in memory. Finally, LLMs generate answers based on the information stored in the memory.
GRPO
NOWI
Plus
Fill Starr
Multi-conv DAPO
AG
Figure 3: Comparison between vanilla GRPO and Multi-Con DAPO. During the rollout phase of Multi-conv DAPO, each sample generates multiple conversations. The answer contained in the final conversation used to compute the reward and advantage, which are then employed to optimize all preceding conversations.
We empirically demonstrate that our RL-trained method allows models to extrapolate to vastly long documents with minimal performance degradation, pushing the boundaries of what is currently achievable in long-context LLM systems.
METHODOLOGY
In this section, we describe the details of MEMAGENT approach for solving long-context tasks, including the overall workflow (5 2.1), Multi-conv RL algorithm for training MEMAGENT ($ 2.2) and the formal modeling of our architecture($ 2.3).
2.1 THE MEMAGENT WORKFLOW: RL-SHAPED MEMORY UNBOUNDED CONTEXTS
As illustrated in Figure 2, MEMAGENT views arbitrarily long document not monolithic block but as controlled stream of evidence. At every step, the model sees exactly two things: the next chunk of text and compact, fixed-lengee memory that summarizes everything deemed important so far. Crucially, the memory is just sequence of ordinary tokens inside the context window, so the core generation process of the base LLM remains unchanged.
After reading new chunk, the model overwrites the previous memory with an updated one. This overwrite strategy seems almost too simple, yet it is precisely what enables the system scale: be- cause memory length never grows, the total compute per chunk stays ((1) and end-to-end complexity strictly linear to the number of chunks. We formulate the overwrite decision reinforcement
3

---

Published as a conference paper at ICLR 2026
External Input
External Output
External Input
External Output
Controller
p(c mm
p(m' |e c.m"
Read Head
Write Head
@
mn!
m
Memory
1=
I=
Architecture of MemAgent
Graphical Model of MemAgent
Figure 4: The architecture and graphic model of MEMAGENT. The memory modeled as a latent memory variable, thereby enabling the decomposition of the autoregressive language model into multiple steps of reading from and writing to the memory.
learning problem: the agent is rewarded for retaining information that will later prove useful and for discarding distractors that would waste precious tokens. By optimizing this objective with our newly introduced multi-conversation DAPO algorithm (detailed in $ 2.2), the model learns to compress aggressively while preserving answer-critical facts.
The workflow naturally decomposes inference into two modules. Within the Context-Processin module the model iterates over chunks, updating memory with a prompt template (Table 5, top). Once the stream is exhausted, final Inswer-Generation module is invoked (Table 5, bottom) where the model consults only the problem statement and the memory to produce its boxed answer. Because positional embeddings apply in both modules, unlocking the model's latent length-extrapolation capability without any architectural modifications.
re-scaled patched, the same tokenization and attention layout
MEMAGENT therefore enjoys three benefits from this design: (1) Unlimited length: the document can be millions of tokens because it is processed encourages the memory to retain exactly the information needed, yielding near-lossless extrapolation (Figure 1); (3) Linear cost: a constant window size implies decoding time and memory consumption grow linearly with input length (O(N) (detailed in B.) This renders practical recipe for turning any moderately context-sized L.M into an efficient long-context reasoner with minimal engineering overhead.
stream; (2) No performance cliff: RL
2.2 TRAINING MEMAGENT WITH MULTI-CONV RL
By viewing memory update in context processing for answer-generation tasks as part the policy be optimized by RL, we adopt the RLVR recipe (OpenAI, 2024; Guo et al., 2025; Seed et al., 2025) to train AGENT We adopt DAPO (Yu al., 2025), an efficient and effective algorithm refined from Group Relative Policy Optimization (GRPO) (Shao et al.. 2024), as our training algorithm. Due to the nature our MEMAGENT approach, which generates multiple single query as illustrated in Figure 2, we treat each conversation as an independent optimization target. This approach necessitates an extension of the loss computation from the conventional (group, tcken) structure to (group, conversation, token) dimensionality, as shown in Figure 3.
independent conversations for
Formally, the policy model New samples group of G' individual responses {or,) AG for an input I. Let n, denote the number of generated conversations (oi,1, (0,,,, 0z.n;) for a given sample (gi, a.). 0j.j further decomposes into token-level outputs (0s,j.1, Oi.j.2.... 0r.. The advantage value is derived from the conversation that contains the final answer, then uniformly applied to all conversations originating from the same sample, shown in Eq 1. Following Dr. GRPO (Liu et al.,
4

---

Published as a conference paper at ICLR 2026
Table 1: Main experimental results comparing model performance across various context lengths. All values represent accuracy (%)
Length
Model
7K
14K
28K
56K
112K
224K
448K
896K
1.75M
3.5M
72.66
75.00
72.66
60.94
31.25
17.19
13.28
11.72
N/A
Owen2.5-Instruee 1-----------11
60.16
60.94
50.00
57.03
50.00
37.50
8.59
0.00
N/A
N/A
Owern255-nsttuuctBB-N
61.72
56.25
53.91
55.47
51.56
33.59
12.50
0.00
N/A
N/A
70.31
66.41
65.62
46.88
23.44
13.28
7.81
7.03
N/A
N/A
DS-Distill-
Saalll--een-eelll
64.06
64.84
57.03
40.62
14.84
8.59
3.12
6.25
N/A
N/A
DS-Distill- Diistill--wen--1
30.47
12,50
3.12
0.00
0.00
0.78
0.00
0.00
N/A
N/A
Swen2.5-Instruct-33
69.53
64.84
60.16
51.56
44.53
21.88
14.06
N/A
N/A
Owen2.5-Instruct-le
75.00
67.19
57.03
54.69
44.53
21.88
10.94
2.34
N/A
N/A
Qwen2.5-Instruct-78
52.34
57.03
51.56
44.53
32.81
13.28
6.25
1,56
N/A
N/A
RL-MEMAGENTT-4B
80.47
82.03
82.03
83,59
81.25
77.34
79.69
75.78
78,91
71.09
RI-MEMAGENI--7
81.25
81.25
82.03
80.47
79.69
75.78
76.56
74.22
77.34
71.88
2025), we do not devide the advantage by the standard deviation. Eq describes our loss function.
A R₂ mean( (Rs)
JDAPO(0)
E (. 1i Not (-2,, o15--)
G
lo
ΣΣΣ
Ciss - (KK (rrr)
E 2ド=1 loisl i=1
1=1
Cij,t = min (Te.j. (0) Aigas clip (0), Stom; Ehigh Ansit)
where
T0(0i,j q. O1gist)
Ta (oi.j.t q, 015.ct)
Following the RLVR recipe (Guo et al., 2025; Jin et al., 2025; Yu et al., 2025), we train the model with final outcome reward computed by rule-based verifier:
R(g v) = equiv(y.)
where ij is the predicted answer while refers to the ground truth.
2.3 RETHINKING MEMAGENT FROM AUTOREGRESSIVE MODELING PERSPECTIVES
Tto get a deeper sense of the MEMAGENT design, we propose to re-think language-mode factor- ization in the following fashion. A standard autoregressive LLM factorizes the joint likelihood of sequence X1:N p(Xi.N) [ p(zn Xim-1), implicitly assuming that every past token (or at least its hidden state) must stay in the active context. This is what turns quadratic attention into the long-context bottleneck.
MEMAGENT replaces the unbounded history with a fixed-length memory E V as shown in Figure 4. The input text is streamed through the model in K contiguous chunks .!...., k (each of length < C). After chunk & is read, the model overwrites the panel with a new vector m & that summarizes all evidence seen so far. Because |= = is constant, both compute and memory per step are OCC' M), yielding overall linear complexity O(N).
Introducing the latent sequence 1:K-- decomposes the original likelihood as
p(XINN) Σ
I
P(c
m
p(m"
c m
(4)
read
write
with base case m Inside each chunk, we still run an ordinary transformer decoder. but conditioned on a constant context window (c* m m---).. The read path factorizes token-by-tokcer
p(c* m --1 --==(- in the same autoregressive fashion.
X1--1, m ==1, while the write path generates the next memory

---

Published as a conference paper at ICLR 2026
Table 2: Model performance on LongBench-SUN. All values represent recall rates (%). Bold marks the highest value and Underline marks the second highest value in each column.
GOV REPORT
QMSUM
Model
ROUGE-1
ROUGE-2
ROUGE-L
AVG
ROUGE-1
ROUGB-2
ROUGE-L
AVG
Owee2.5 Instruct- Dwes2.5-nnstruct-
52B
23.67
5.46
12.57
14.90
42.77
11.29
28.17
29.08
Qwen2.5 1oem2.5-Instraat 14B
31.19
10.96
14,96
19.04
47.53
11.46
28.28
29.09
Owen2.--Insrruut77
12.01
QwenLong-Li
Dwes2.--nnstradt 148-
30.38
11.93
13.51
19:34
41.31
13.13
29.07
29.84
Qwen2.5 Instract IM
31.02
11.47
15.30
19:26
46.72
12.33
28.66
29.24
DS Distill Owen 32B
26.13
5.86
12.95
15.99
39.09
8.75
23.96
23.93
DS Distill Qwen-I 14B
28.24
9.72
13.78
17.25
41.25
8,95
25.00
25.07
DS-Distill-Quen QwennBB
21.57
RI-MEMAGENT-78
30.38
15.37
19.34
48.49
14.41
30.91
31.27
In our formulation, the model's reading and writing operations the context constitute Markov Decision PrDC) and the objective RI. is to optimize the final reward obtained by this MDP. Therefore, MemAgent's learning objective is generate a read-write memory trajectory that maximizes the reward, which corresponds to learning an optimal distribution over memory states conditioned on the input context. This further theoretically illustrates the intrinsic unity between our RL formulation and long-tex text modeling.
3 EXPERIMENTS
3.1 EXPERIMENTAL SETUP
Training Details. To maintain compa- rability with previous work, we choose Dwee2.5-7B-IInstrel and @wen2.5514BB- Instruct (Yang et al., 2024) as backbone models. We implement the framework for multi-conversation with independent contexts based on verl (Sheng et al., 2024).
Table Model performance on LongBench-
2Wiki
VisiQQu
Quspee
QwenLLong-L---00
$3.0
69.5
51.0
24.0
50.7
Owen2.5 Ilstrrttt Qwen2.5 Insruct
65.5
67.5
DS. Disal Qwen 32B
$3.5
69.0
47.5
34.0
21.0
49.0
DS-Dieill-Quaa (aall--0m--44 14R
K3,5
67.0
43.0
22.0
20.0
47.3
7B
Qwent 5-Inamaa--12
68.5
65.0
31.0
24.5
22.5
457
Qwen2.5 Instrct 14B
67.0
63.5
39.0
30.0
20.5
42.0
We employ a two-stage curriculum RL strategy:
Doen22.-Aestruttahe
52.5
59.0
24.0
19.0
19.0
34.7
MrsrAonNT-14R MIMAAGINN-766
20.0
53,0
53.0
3500
1600
$1.01
74.0
69.5
43.0
21.5
29.0
48.2
1) stage focuses on enabling the model to acquire fundamental memory capabilities; 2) stage trains the model to transfer these capabilities to more diverse contexts and challenging tasks. Specific hyperparameters for training are detailed in $ A.3.
During training, intentionally limit the model extrapolation capabilities. This 8K window is allocated as follows: 1024 tokens for the query, 5000 tokens for the context chunk, 1024 tokens for the memory, and 1024 tokens for the output, with the remaining tokens reserved for the chat template.
8K context window demonstrate its
Benchmarks. We conduct comprehensive evaluations several long-text benchmarks to assess the model's capabilities across various text types and tasks.
1. RULER-HQA. This benchmark is created using the same synthetic method in the first- stage training data. It consists of tasks with moderate information density and controllable length, where the context distribution is close natural language, serving as quantitative evaluation of extrapolation performance.
2. LongBench-OU QA. This benchmark composed of NarrativeQA (Kočiský et al., 2018), Qasper (Dasigi et al., 2021), HotpotQA (Yang et al., 2018), 2WikiMultihoQAA Ho et al. (2020), and MuSiQue (Trivedi et al., 2022). The tasks are relatively short but have a high information density, which severely tests the model's 's flexible memory management. It also evaluates the model's ability to generalize its memory capabilities to various materials, such novels, news articles, and Wiki items.

---

Published as a conference paper at ICLR 2026
NIAH-Level 1
NIAH-Level 2
100 00
100.00
100.00
100.00
100.00
1.0.00
100.00
100.00
100.00
99.22
95.31
94.53
100
RL-Mamagant-76 100.00
100.00
100.00
100.00
100.00
99.22
100.00
100.00
100.00
99.22
97 66
96.88
93.75
92:97
MemAgent-522 w/o RL -100.00
100.00 100.00
97.66
96 96.09
09
96 88
100.00
09
91
81
25
75.00
78.1
60:16
MemAgent-14B -100.00
96.09
100.00
17:39
MemAgent-78 w/o
-98.44 97.66 96.09
58.44
93:31
95.31
100.00
98
99.22
97.66
89:06
Lwentong
2000 00 300 00 100.00
100.00
88.28
42.97
21:88
100.00 100 00
100
00
88.28
22:12
1953
14:34
Owen2.5-instral i-nnnttttttt 1M 100 00 100 00 100.00 100.00
100.00
100,000
1000.00
59.2 29 100 00
100
00
100.00
190 00
100 00
B6
Qwen2. 54nstruct-78-111 1M
-100.00 100
100.00
100.00
DS.Distill- Qwen 228
S--Distll--en-2222
1,00
100.00
300.00
83.06
42.97
21.88
100..00
98
29.69
18.75
14.06
DS-Distill -wetlll-Qwer-18
100 00 100 00
100,00
98.44
84.38
40.62
21.09
99 22 100
00
100
00
60
18.75
8.59
11.72
DS-DistillOween-70
73:44 27.34
8.59
0.00
0.00
0.00
0.00
51
18.75
0.00
0,00
0.00
0.00
0.00
Owen 2.5-Instruct-326-
100.00
000
100.00
100.00
89:06
42.97
21.88
100.00
100
100
00
100
09:06
45.31
23:44
9wen2.5-hneructt144-
100.00
100.00
100
89.06
23:44
60
Qwee22.-frsiruc-778
85
30,28
21,09
25,00
11,72
9.38
3.12
92.
68.
75 64 06 88. 20
50,70
25,00
17.97
16K
32K
64K
256K
16K
64K
128K
256K
NIAH-Level
NIAH-Average
NLMemagent:148 99.22
96.09
100.00
100.00 97.65
96. 08
100.000
00.00
99.
98.
70
100
00
00
98.96
97:40
90 18
RL-Memagent-778
100.00
96.09
100.00
22
95.31
100.000
96 06
100.00
98 70
99
74
98
96
97
66
96 62
MemAgent-52B w/o RL
100.00
92.19
92.19
88.26
72.66
60.16
$8.59
100.00
09
94
89
06
81
78.39
71
MomAgent-14B
72
MemAgent-70 w/o RL 99.22
90.62 92.97
97.66
89
04
86.77
84.3
99 22
09
95 83
98
94.01
93.23
00 54
Qwentong-L_1122 1-326- 96.09
96.88 97.66
93.75
40.62
21.86
15.62
96. 70
98 96
99
94.01
52:44
28:13
17:45
wwwn2.--nsttuut-1485 1M
19.22
100 00 100.00
100.00
100.00
100.00
92:07
99.
00
100.00
100.00
100 00
00
93:23
Qwen2. 54nnstrutt78-111 -100.00
100
100.00
100.00
100.00
99
73:44
D5.Distil-QWerr Qwen-328
93.22
00 97.66
89.06
31:25
17:19
11.72
99.
98
50.00
26.30
15.89
D-Dittil-Qwen-144
-9.22
100 00 100.00
71.88
23.44
10.94
8.59
90 48
100 00
100 00
76. 83
42.19
20.05
13,80
Disttill-Qwen-75
57.03
10.94
0.00
0.00
0.00
0.00
0.00
62 76
19.01
2.85
0.00
0.00
0,00
0.00
(wenn2.-instruuct222
328
100.
100.00
40.62
100.00
100
42:97
23:44
2wer22.-Instructe
100.00
93.22
100.00
100.00
93.31
40.62
25.00
100.00
100
43:23
84,38
05.34
80,28
96 08
78.81
38, 28
24,22
07 50
64 32
01
70
05
47.14
24,22
15,10
16K
64K
128K
256K 512K
16K
32K
64K
128K
256K
512K
Figure 5: Performance heatmaps on NIAH benchmark across different context lengths.
3. NIAH. Needle in haystack (NIAH) (Kamradt, 2023) is a series of extremely long synthetic tasks with very low information density. To succeed, the model must identify key information and maintain its integrity throughout long process, thereby testing the robustness of memory.
4. LongBench-SUNN. We also adopt two long-com long-context summay tasks, GovReport(Huang et al., 2021) and QMSum(Zhong et al., 2021) from LongBench(Bai et al., 2024) to evaluate the performance in different task category that is different from retrieval QA.
Baselines. We use Deepseek-R1-Distil--ee 2025). Qwen-2.5-Instrrn (Yang et al., 2024)and QwenLong-LI (Wan al., 2025) baselines. Their generation configurations are shown in Table 6, while MEMAGENT uses the same context management as described previously in Training Details. We also compare MEMAGENT with other agent method, detailed in D.2.
(Guo et al., 2025). 2.-n-225-Instruut--1 (Yang et al.,
3.2 MAIN RESULTS
RULER-HQA. The results are reported in Table 10. We conduct comparative analysis of all model performances with context lengths ranging from 7K to 896K. For MEMAGENT, we extend the evaluation to ultra-long contexts of 1.75M and 3.5M to assess its extrapolation capabilities.
MEMAGENT exhibits remarkable length extrapolation capabilities with only marginal performance decay as the input context-length increases. In contrast, baseline models show distinct failure patterns. DS-Distill- Qwen series show rapid performance degradation. QwenLong- L1 maintains reasonable performance within its training length but experiences substantial degradation afterward. The -. M series maintains acceptable performance up to 112K tokens, but the performance deteriorates to zero at 896K tokens, well before reaching their theoretical 1M token capacity. This suggests that despite extended context windows, these models struggle with effective information utilization in ultra-long contexts.
LongBench-QA. The results the LongBench-QA benchmark are presented in Table 3. MEMA- GENT demonstrates superior overall performance, outperforming larger long-context or reasoning models. Reasoning models such as the DS-Distill families and the QwenLong model which are trained on a complex dataset, exhibit strong performance. In contrast, the shows limited improvement over its backbone model. This suggests that LongBench-QU QA emphasize
series
7

---

Published as a conference paper at ICLR 2026
7B Instuct
7B MemsAgent RLL
78 MomuAgent RL
Instruct
MimsAgent
Mers&ggent
80
60
40-
Perforance (%)
Score
2WHki
HQA
MuSiQue
Memagent-144
80
M.-Memagent778
20
MemAgent-14B w/o RL
80
MeimAgent-78 w/o
Qwen2 5-instruct1146
Owee2t-inntructt
20
14K
28k
56K
112K 224K
Context Length Tokens
NOA
Casper
Figure 6: Ablation result of RL training RULER-HQA.
Figure 7: Ablation result of RL training on Longbench-QAl
deeper understanding of text rather than simple retrieval ability. The performance of MEMAGENT demonstrates that the memory capabilities acquired through reinforcement learning are generalizable.
NIAH. We adopt three variants of NIAH from the RULER benchmark Hsieh al. (2024) with increasing difficulty across three levels. As depicted in Figure 5, the majority of baselines struggle to maintain consistent performance even within 128K context window, even Iiven2.5-IntrruttIM also experience a performance drop at 512K. EL-MEMAGENT despite suffering some performance fluctuations, shows only a minimal performance loss of less than 5% 5 512K. This robust performance is particularly noteworthy given that the evaluation at 512K involves more than 100 turms of dialogue.
LongBench-SUUM. We evaluate summary quality by the recall scores of ROUGE (1, 2, L). RL- MEMAGENT achieves SOTA on almost all metrics, demonstrating that the model has learned general memory and context management capabilities, rather than abilities specific the QA task.
3.3
ABLATION STUDY
3.3. RL TRAINING
To investigate the impact of reinforcement learning, we conduct ablation experiments. The results of RULER-HOX and NIAH presented in Figure 6 and Figure respectively. MEMAGENT without reinforcement learning training outperforms the backbone models; however, it still exhibits substantial decline in performance the input length increases. The results of Longbench-QA shown in Figure 7, demonstrate that directly applying MEMAGENT leads to only marginal or even negative improvements. In contrast, EL-MEMAAENT achieves significant improvements in both evaluation scenarios, indicating reinforcement learning training is essential to develop generalizable and robust memory abilities.
3.3.2 MEMORY LENGTH
Selecting an appropriate MEMAGENT setting involves certain trade-offs. larger memory size allows the model to store more useful information, but it also introduces challenges in memory management and increases the likelihood redundancy. Conversely situation may lead to insufficient storage capacity, leaving the model without the necessary references.
To achieve reasonable compression ratio while keeping the total context length within 8.192 tokens, set the default configuration of MEMAGENT to use 1,024-token memory and context chunks of 5,000 tokens, based on preliminary validation results.
To investigate the effect of hyperparameter choices, conduct an ablation study memory length ranging from 256 to 4096. The results presented in Figure and Figure 9, showing that our chosen configuration constitutes reasonable sweet spot, and that MEMAGENT 's performance is robust over different memory size. We further examine the impact of varying the context size in $ D. and observe similar trends.
8

---

Published as a conference paper at ICLR 2026
NIAH-Level 1
NIAH-Level 2
14B-m4096
-100.00 100.00010000000 00 100.00 100. 00 100 100.00 100.00
100.00
100.00
99.22
98.22
99.22
100.00
100
148-m2048
100.00
60 100. 00 100. 00 100 100.
100.00
99.22
100 100.00
100
99.22
99.22
100.00
148-m1024
100.00
00 60 100.00
100 00 100 00 100.
100.00
100.00 100.0001000000000000000000000000 00 100.00
100 00
99.22
95.31
94.53
148-m512
100.00
60 100.00 100.
100
100.00
100.00
100
93.75
14B-m225
1100.00 100 00 100.00 100 00 100
00 100 00 100.00
100.00 100 00 99.22
98.44
86.72
92.19
88.28
00
78-m4096
100..00
00 100.00000000
100.00
100. 100
78-m2048
100.00
00 100.00 100. 100.
00 100.000000000 100.00
100.00 100 00 100.00
97.66
94.53
93.75
96.09
78-m1024
100.00
00 100.00
100.00 100.
99.22 100.00
100.00 100.
97.66
93.75
92.97
78-m512
100.00
100.0000000 100.00
100. 00 100.
100.
100.00
100.00
100
97.66
96.88
95.31
91.41
89.
78-m256
100.00 00 10 00
100, 100
00 100 100. 100.00
99.
100
97 66
93.75
98.75
91,
60
8K
16K
32K
64K
128K
256K
512K
8K
16K
32K
64K
126K
256K
512K
NIAH-Level
NIAH-Aver Aver-ee
148-m4096
100.00
96.09
100.00
100. 00
99.22
100. 00
100.00
100.00
98.70
100.00
99.74
99.48
99.74
100.00
14B-m2248
100..00
96.09
100.00
100.00
99.22
100.00
100.00
99.74
98.70
100.00
100.00
99.48
99.74
100.00
40
148-m1024
99.22
96.09 100.00
100. 00
97.66
96.88
100.00
99 74
98.70
100.00
100.00
98.96
40
98.18
148-m512
100.00
96.09 100.00
98.44
95.31
96.09
98.88
100.00
98.70
100.00
98.96
97:40
96.61
95.83
148.m256
100.00
92:97
98.70
95.05
93.75
78-m4096
100.00
96.09 100.00
99.22
99.22
96.88
97.66
100.00 98.70
100.00
99.74
98.70
96.88
92
-20
78-m2048
100.00
100.00
100.
78-m1024
100.00
96.09 100.00
99.22
95:31
100 00
96.88
100 00 98.70
99 74
38. 96
97 40
97 86
96. 62
78 m512
100.00
96.09
99.22
96.88
96.88
93.75
100.00
98.70
96
96.10
94.53
7B-m256
100.00
96.09
98
99.22
96,
97.66
96.09
99 74 98.70
96, 70
97.66
96, 88
96, 09
83
8K
16K
32K
64K
128K
256K
512K
8K
16K
32K
64K
120K
256K
512K
Figure Ablation result of memory-length on NIAH
2Wiki
HQA
MuSiQue
NQA
Qasper
AVG
74.0
76.0
72.0
78.0
67.0
69.0
60.0
71.0
70.0
44.0
46.0
48.0
40.0
42.0
22.0
20.0
18.0
24.0
24.0
26.0
28.0
32.0
30.0
46.0
45.0
70.0
$12
1024
2040
4036
512
1024
2040
4096
38.0
512
1024
ZIMB
16.0
512
1024
2049
22.0
512
1024
ZBMB
4098
44.0
258
512
1024
2048
4008
Figure 9: Ablation result of memory-length on Longhench
3.3.3 CONTEXT DISTRIBUTION
Although our experiments show that MEMAGENT can effectively extrapolate to a length of 3.5M tokens, we still wish to examine whether AGENT is affected by issues such as information- overwritten and the a natural result of end-to-end optimization. During training, the model learns to preserve and track critical information in order to maximize the final reward.
phenomenon. Our hypothesis is that overcoming such problem
To validate this hypothesis, we carefully design a set of probing experiments based on RULER-HQA
where the context is consist of some key information and many distractors. We divided the key information into two groups and placed them at different positions within the context. We constructed five settings: (0%, 100%), (20% 80%), (40%, 60%), (0%, 20%), and (80% 100%), where 0% indicates the beginning of the context and 100% means the end of the context.
For example, in the (0%, 100%) case, the model sees one piece of key information at the very beginning and the other only at the final memory update step. This represents one of the most challenging scenarios for the information-overwriit problem. While (40%, 60%) may serve setting.
challenging ses-in-thhemine
The results shown in table 4 indicates that MemAgent remains consistently robust across all patterns without exhibiting any catastrophic performance degradation. This strongly supports our hypothesis that the general memory abilities acquired through trial and error are not tied to any particular context pattern.
RELATED WORK
Long Context LLMs. Extrapolation methods for RoPE-based LLMs (Su et al., 2024), such as NTK (bloc97, 2023), PI (Chen et al., 2023), YaRN (Peng al., 2023b) and DCA (An et al., 2024),
9

---

Published as a conference paper at ICLR 2026
Table 4: Probe experiment results. Cix. Dist. denotes the context distribution, where the two numbers correspond to the relative positions of the two tey-infomation groups within the entire context. 0% means the beginning and 100% means the end. random indicates randomly shuffling all context items, consistent with the setup in the main experiment. The other rows show the performance difference relative to random. All values represent accuracy (%).
Model
Length
7K
14K
26K
56k
112K
224K
448K
AVG
14B
medomm
80:47
$2.03
$2,03
83.59
81.25
77.34
79.69
75.78
0% 100%
+3.12
+0.78
+3.13
-3.12
+1.56
+7.82
+6.25
+2.79
20% 80%
+0.78
-3.12
+2.35
+0.79
-3.13
+3.47
-3.13
0.00
40% 00%
+2.35
-2.4
-1.56
+3.13
40.11
100%
+3.13
0.00
91.25
79.69
75.78
20%
-1.56
00.88
+3.13
+3.91
+3.12
+3.17
+3.13
+2.35
0% 100%
+0.78
0.00
+2.35
+1.56
+3.90
+4.69
+2.15
+2.23
20% 90%
00.78
+2.35
0.00
0,00
+0.56
30% 100%
+1.56
0.00
+0.78
-0.78
+0.78
+4.69
0.00
+1.00
modify the components positional embeddings, enabling the model to capture long-range semantic dependencies. On the other hand, Linear attention mechanisms (Child et al., 2019; Katharopoulos et al., 2020), Recurrent Neural Networks (RNNs) and State Space Models (SSMs) (Gu et al., 2021; Gu & Dao, 2023; Peng et al., 2023a; De et al., 2024; Feng al., 2024), sparse attention (Beltagy et al., 2020; Zhao et al., 2019; Xiao et al., 2023; Yuan et al., 2025; Lu et al., 2025) focus on architecture improvements. Chunk strategy have also been explored in long-context modeling (Li et al., 2025b; Liao et al., 2025), while MEMAGENT aims to equip memory ability to any backbone model via post-training with standard RL frameworks without heavily changing on architecture.
Memory Mechanism. The Long Short-Term Memory (LSTM) mechanism (Hochreiter & Schmid- huber, 1997) achieved significant success early NLP tasks, while Neural Turing Machines (Graves et al., 2014) and Memory Networks (Weston et al., 2014) demonstrated how to equip neural networks with memory. Existing memory mechanisms integrated to Transformer models are typically realized by adding external memory modules (Martins et al., 2021: Wu et al., 2020; Behrouz et al., 2024; Bulatov al., 2023) or external datahase (Zhong et al., 2024; Lu et al., 2023; Modarressi al., 2023). Recently, retrieval-augmennee memory agent (Fang et al., 2025; Chhikara et al., 2025; Zhou et al., 2025) workflows have attracted the community' 's attention. The diffrence between MEMAGENT and other agent is that we use reinforcement learning enable LLM itself the ability to memorize.
Reinforcement Learning for LLMs. recent RL studies, the reward signals have gradually shifted from human preferences (Ouyang al., 2022) reward models distilled from them (Bai et al., 2022) to rule-based feedback, which has demonstrated great potential in enhancing model reasoning capabilities (OpenAI, 2024; Guo et al., 2025; Qwen, 2024; DeepMind, 2024: Team et al., 2025) with GAE (Schulman et al., 2018) based PPO (Schulman et al., 2017) 7) or GRPO (Shao et al., 2024) training. Algorithmic enhancements (Hu, 2025; Yu et al., 2025; Liu et al., 2025) have mostly focused on improving sustainability and efficiency of these algorithms. To further release the potential of RL, recent works such as Search-RI (Jin al., 2025), Agent-RI (Ouyang et al., 2025) and RAGEN (Wang et al., 2025) have explored the training of tool-using agents based on multi-turn chat. GiGPO (Feng al., 2025) further investigates the use of multiple independent contexts in agent training.
CONCLUSION
In this paper, we introduce MEMAGENT, novel long-context method that employs an RL-trained memory module, which enables large language models (LLMs) to selectively record relevant in- formation while disregarding extraneous details. Our experiments demonstrate that when trained on 60K-length sequences, MEMAGENT exhibits remarkable extrapolation, extending its effective context to 3.5M tokens with only 8K context. The model achieves state-of-the-art performance across a diverse range of long-context tasks. Our ablation studies reveal the critical role of RL-based training in achieving these results and how memory capacity influences performance across different task types, providing key insights into the proposed memory mechanism. We hope that this work may lay a strong foundation for developing more advanced memory architectures and training strategies, thereby paving the way for significantly enhancing the long-context capabilities of LLMss.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
For reprodueibility, we have provided the inplementation details in ($ A), including the prompt template ($ A.1), pseudocode ($ A.2) and training recipe and algorithm hyperparameter ($ A.3) and evaluation settings (8 A.4). The training and evaluation code, as well as the dataset and model weights, will be available in open-source platforms.

---

Published as a conference paper at ICLR 2026
REFERENCES
Chenxin An, Fei Huang, Jun Zhang, Shansan Gong, Xipeng Qiu, Chang Zhou, and Lingpeng Kong. Training-free long-contex, scaling of large language models. arXiv preprint arXiv:2402. 2024.
Anthropic. Claude 3.5 sonnet, 2024.
URL https://ww.anthropcc.comm
claude-3- sonneet...
Anthropic. Introducing claude 4, 2025. URL https//www..ncchoooc.coooo////// claude-4.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08077 2022.
Yushi Bai, Xin Lv,, Jijjie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. Longbench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 3119-3137, 2024.
Ali Behrouz, Peilin Zhong, and Vahab Mirrokni. Titans: Learning to memorize at test time, arXiv preprint arXiv:2501.00665 2024.
Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05/500 2020.
bloc97. NTK-Aware Scaled RoPE allows LLaMA models to have extended (8k+) con- text size without any fine-tuning and minimal perplexity degradation,, 2023. URL
https: //tp:///wwwwwww reddit wtt.c/cm//////eeeeeee L/ LLaMA LLaMA/ 2ommentss14422j5//tttttt scaled_rope aww.ss 1aa.aalaamaaaaaaaa s tt__tt__hae__et to__ave/.
Aydar Bulatov, Yuri Kuratov, Yermek Kapushev, and Mikhail S Burtsev. Scaling transformer to Im tokens and beyond with rmt, arXiv preprint arXiv:2304. 11062, 2023.
Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:23066. 15595, 2023.
Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arX1v:2504.19413,
2025.
Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse transformers. arXiv preprint arXiv: 1904. 10509, 2019.
Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A Smith, and Matt Gardner. A dataset of informattonn-sckking questions and answers anchored in research papers. arXiv preprint mrXiv:2105.03011, 2021.
Soham De, Samuel L Smith, Anushan Fernando, Aleksandar Botev, George Cristian-Murant Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, et al. Griffin: Mix- ing gated linear recurrences with local attention for efficient language models. arXiv preprint arXiv:2402.19427, 2024.
Google DeepMind. Gemini 2.0 flash thinking, 2024. URL httpt:://eeemmineee google/ technol oonnoolliiee es htw.i///taahhhhhiinnnneee
Runnan Fang, Yuan Liang Xiaobin Wang, Jialong Wu, Shuofei Qiao, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. Memp: Exploring agent procedural memory. arXiv preprint arXiv:2508.066 06433, 2025.
Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. Group-in-grout policy optimization for Ilm agent training. arXiv preprint arXiv:2505. 10997888.. 2025.
12

---

Published as a conference paper at ICLR 2026
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi, Mohamed Osama Ahmed, Yoshua Bengio, and Greg Mori. Attention as an rnn. arXiv preprint arXiv:2405. 2024.
Chaochen Gao, Xing Wu, Zijia Lin. Debing Zhang, and Songlin Hu. Nextlong: Toward effective long-context training without long documents. arXiv preprint arXiv:2501.12766, 2025.
Alex Graves, Greg Wayne, and Ivo Danihelka. arXiv:1410.5401, 2014.
Neural turing machines.
arXiv preprint
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, arXiv preprint arXiv:2312.00752, 2023.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces, arXiv preprint arXiv:21 2021.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12945 2025.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing multi-hop hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint 1.XX::0011.010000 2020.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. Ruler: What's the real context size of your long-context language models? arXiv preprint wxXiv:2404006654, 2024.
Jian Hu. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03266. 2025.
Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, and Lu Wang. Efficient attentions for long document summarization. arXiv preprint arXiv:2104.02112 2021.
Huiqiang Jiang. Qianhui Wu, Chin- Lin, Yuqing Yang, and Lili Qiu. LImlingua: Compressing prompts for accelerated inference of large language models. arXiv preprint arXiv:2310.05736 2023.
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Serean Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training iims to reason and leverage search engines with reinforcement learning, arXiv preprint arXiv:2503.09516, 2025.
Gregory Kamradt. Needle In A Haystack pressure testing LLMs. Github, 2023. URL https: //github. com/ hain//gamrradt LLMTest wtw.rrestteeeeeeeniiiieeeee Wew.eeeiiaayytttttt/ttootattae /main.
Angelos Katharopoulos. Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are mnns: Fast autoregressive transformers with linear attention. In International conference on machine learning, pp. 5156-5165. PMLR, 2020.
Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. Transactions of the Association for Computational Linguistics, 6:317-328. 2018.
Aonian Li, Bangwei Gong, Bo Yang, Boji Shan, Chang Liu, Cheng Zhu, Chunhao Zhang, Congchao Guo, Da Chen, Dong Li, al. Minimax-011 Scaling foundation models with lightning attention. arXiv preprini arXiv:2501.083/11 2025a.
Yucheng Li, Bo Dong, Chenghua Lin, and Frank Guerin. Compressing context to enhance inference efficiency of large language models. arXiv preprint arXiv:2310.06201, 2023.
13

---

Published as a conference paper at ICLR 2026
Zhenyu Li, Yike Zhang, Tengyu Pan, Yutao Sun, Zhichao Duan, Junjie Fang, Rong Han, Zixuan Wang, and Jianyong Wang. Focusilm: Precise understanding of long context by dynamic condensing. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 31087-31101, 2025b.
Zihan Liao, Jun Wang, Hang Yu, Lingxiao Wei, Jianguo Li, and Wei Zhang. E2Ilm: Encoder elongated large language models for long-context understanding and reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 19212-19241, 2025.
Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: strong, economical, and efficient mixture-of- experts language model. arXiv preprint arXiv:2405..0 044444, 2024.
Xiaoran Liu, Hang Yan, Shuo Zhang, Chenxin An, Xipeng Qiu, and Dahua Lin. Scaling laws of rope-based extrapolation. arXiv preprint 2rxiv:23/0005200 2023.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1 r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025.
Enzhe Lu, Zhejun Jiang, Jingyuan Liu, Yulun Du, Tao Jiang, Chao Hong, Shaowei Liu, Weiran He, Enming Yuan, Yuzhi Wang, et al. Moba: Mixture of block attention for long-context llms. arXiv preprint arXiv:2502.1318) 2025.
Junru Lu, Siyu An, Mingbao Lin, Gabriele Pergola, Yulan He, Di Yin, Xing Sun, and Yunsheng Wu. Memochat: Tuning llms memos for consistent long-range open-domain conversation. arXiv preprint arXiv:2308.0822. 08239, 2023.
Pedro Henrique Martins, Zita Marinho, and André FT Martins, co-former: Infinite memory trans- former. arXiv preprint 0rXiv2219.00300. 2021.
George A Miller et al. The magical number seven, plus minus two. Psychological review, 63(2): 81-97, 1956.
Ali Modarressi, Ayyoob Imani, Mohsen Fayyaz, and Hinrich Schütze. Ret-llm: Towards general read-write memory for large language models. arXiv preprint ar 14322, 2023.
OpenAl. GPT4 technical report. arXiv preprint arXi::2303.0874,,20 2023.
OpenAl. Learning to reason with 1lms, 2024. URL https: / http//ppeaaa.ccoae indx/ Learning-to- cooo-reasoo--ae with- 11ms/.
Jie Ouyang, Ruiran Yan, Yucong Luo, Mingyue Cheng, Qi Liu, Zirui Liu, Shuo Yu, and Daoyu Wang. Training powerful Ilm agents with end-to-end reinforcement learning, 2025. URL https:
htg.thub.comm/mmmm O Orusswest A/AgenttRgeeeeee RR.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feed- back. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Ad- vances in Neural Information Processing Systems, volume 35, pp. 27730-27744. Curran Asso- ciates, Inc., 2022. URL htt.t::///// http:::::////ooooeoeeeeee wttf/ileeebee http:e/eeee..eeeeeeeeeeee2223333
iprrippspp-o wwwps//p/pppppeeeeeeeeeeeeeeeeee www.ffieeppppeee pdd.
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak. Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, et al. Rwkv: Reinventing mas for the transformer era. arXiv preprint arXiv:2305. 2300:33355.13488 2023a.
Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yam: Efficient context window extension of large language models. arXiv preprint arXiv:2309.0007/ 2023b.
14

---

Published as a conference paper at ICLR 2026
Qwen. Qwq-32b: Embracing the power of reinforcement learning, 2024. URL https://////ee github. ht/blog/////ooooo qwq-3wb/ 32b/.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 2017.
John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation, 2018. URL httthttsp//aaiiiii abs/1506 02438.
ByteDance Seed, Jiaze Chen, Tiantian Fan, Xin Liu, Lingjun Liu, Zhiqi Lin, Mingxuan Wang. Chengyi Wang, Xiangpeng Wei, Wenyuan Xu, al. Seed1. 5-thinking: Advancing superb reasoning models with reinforcement learning. arXiv preprint arXiv:2504.133: 13914, 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, YK Li, Y Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03388 2024.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv:2409.1923) 2024.
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with Ilms. arXiv preprint arXiv:2501.12599 2025.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539-555 3-44, 2022.
Fanqi Wan, Weizhou Shen, Shengyi Liao, Yingcheng Shi, Chenliang Li, Ziyi Yang, Ji Zhang, Fei Huang, Jingren Zhou, and Ming Yan. Qwenlong-ll: Towards long-context large reasoning models with reinforcement learning. arXiv preprint arXiv:2505. 10.5500577666 17667, 2025.
Zihan Wang. Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang. Yejin Choi, and Manling Li. Ragen: Understanding self-evolution in Ilm agents via multi-turn reinforcement learning, 2025. URL 20073.
Jason Weston, Sumit Chopra, and Antoine Bordes. 1rXiv:1410.3916, 2014.
Memory networks.
arXiv preprint
Qingyang Wu, Zhenzhong Lan, Kun Qian, Jing Gu, Alborz Geramifard, and Zhou Yu. Memformer: transformer for sequence modeling. arXiv preprint 17X1v.2010.06891,
A
2020.
XAL. Grok beta the age of reasoning agents, 2024. URL http::////a
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. arXiv preprint 2rXhv:2309.17453, 2023.
Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang. Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, et al. Effective long-context scaling of foundation models. arXiv preprint wwXiv:2309.6033 2023.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li. Dayiheng Liu, Fei Huang. Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint 15/5 2024.
15

---

Published as a conference paper at ICLR 2026
An Yang. Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang. Jingren Zhou, Junyang Lin, Kai Dang, Kexin Yang. Le Yu, Mei Li, Minmin Sun, Qin Zhu, Rui Men, Tao He, Weijia Xu, Wenbiao Yin, Wenyuan Yu, Xiafei Qiu, Xingzhang Ren, Xinlong Yang, Yong Li, Zhiying Xu, and Zipeng Zhang. Qwen2.5-1ng m technical report. arXiv preprint arXiv:2501. 15383, 2025.
Zhilin Yang. Peng Qi. Saizheng Zhang, Yoshua Bengio. William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpoiqa: A dataset for diverse, explainable multi-hop question answering. arxiv preprint arXiv: ht0::/80..0000 809,09600, 2018.
Howard Yen, Tianyu Gao, Minmin Hou, Ke Ding, Daniel Fleischer, Peter Izsak, Moshe Wasserblat, and Danqi Chen. Helmet: How to evaluate long-context language models effectively and thoroughly. arXiv preprint arXiv:2410.02640 2024.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source IIm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, YX Wei, Lean Wang, Zhiping Xiao, et al. Native sparse attention: Hardware-aligned and natively trainable sparse attention. arXiv preprint arXiv:2502. 1089, 2025.
Jiaxin Zhang, Yiqi Wang, Xihong Yang, Siwei Wang, Yu Feng, Yu Shi, Ruichao Ren, En Zhu, and Xinwang Liu. Test-time training on graphs with large language models (Ilms). In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 2089-2098. 2024.
Guangxiang Zhao, Junyang Lin, Zhiyuan Zhang, Xuancheng Ren, Qi Su, and Xu Sun. Explicit sparse transformer: Concentrated attention through explicit selection. arXiv preprint arXiv: 1912. 2019.
Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, et al. Qmsum: A new benehmark for query- auery-based multi- domain meeting summarization. Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5905-5921, 2021.
Proceedings of the 2021 Conference of the North American
Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 19724-19731, 2024.
Zijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang. Meml: Learning to synergize memory and reasoning for efficient long-horizon agents. arXiv preprint arXiv:2506.1588 15841, 2025.
16

---

Published as a conference paper at ICLR 2026
A IMPLEMENTATION DETAILS
A.1 PROMPT TEMPLATE
You are presented with problem, section of an article that may contain the answer, and previous memory. Please read the section carefully and update the memory with new information that helps answer the problem, while retaining all relevant details from the previous memory. <problem> (prompt) </problem>
<memory> {memory) < //memory>
<secticn> (chunk) </section>
Updated memory:
You presented with problem and previous memory. Please the problem based the previous memory and put the answer in \boxed ().
<problem> em {prompt} </problem> <memory> (memory) </memory>
Your answer:
Table 5: Template of MEMAGENT for context processing (top part) and final answer generation (bottom). Curly-brace placeholders (} will be replaced with actual content.
A.2 ALGORITHM
Algorithm 1 Multi-conv DAPO
Require: Policy model ma. reference model Tref (froozen) rule-based verifier V, dataset D, group
size G
1: while not converged do
2: Sample a prompt ~
3: for g to G do
Group of conversations for the same
4: Initialize hg.0 f [g]
5: for /
Cg do
Context-indepeeddnt conversation
6:
Sample Og,t To(- hggt-1)
7:
hg. < high-1 0g.
8:
end for
9:
Vg + Og.cy
Final response used for scoring Rule-based reward
10:
fg + V(g.vb) Wg)
11:
dg < KT(TTo- (ig.ey) Tree(: lgg.e)))
rg < ig - -dg
13:
end for
14:
{A,]e-
1G
+ GroupNormm(ra 1G
15:
for 9 to G do
Pg - To(Un! hgges)
16:
Tem (Vo hggeo)
J, min Agg clip(pg, Elows thigh) )
18:
end for
Jelip 1E
0+ 0 + V/Jolli
21: end while
A.3 TRAINING
We use the DAPO algorithm for training, applying a KL factor x 10 3 and disabling the entropy loss. The AdamW optimizer is employed with constant learning rate of x 10 and linear warm- up scheduler, with the wram-up step set to 20. We use a rollout batchsize of 256, with a group size of 16. Note that due the multi-conversation feature of AGEAT,,T, the actual mi aiitbatchs
17

---

Published as a conference paper at ICLR 2026
is not equal rollout batchsize/16. We utilize off pollcy training by fixing the ratio of the sample batch size to the backpropagation batch size is set to 16.
We shift to stage II when stage are fully converged, which takes about 400 steps. Here is the training data recipe of each stage.
Stage We use 32,768 synthetic QA data instances, each approximately 32K tokens length. These are based the HotpolQA (Yang et al., 2018) dataset and follow the RULER (Hsich et al., 2024) methodology, which involves embedding golden paragraphs (containing correct answers) within extensive distractor content sampled from the same dataset.
Stage II We use 2,560 training instances with maximum length of 60K tokens. This set consists of difficult, high-quality long-text QA data from 2025), mixed with data from the first stage.
(Wan et al.,
Each training sample used in stage I is of 200 articles in HotpoiQA, with approximate total token length 28K. We thoroughly clean the dataset by filtering out questions where Qwen2.5-7B-BB 7B-Base or Breen..-7B-Instruue achieves 100 % Best-Of-2 score without given any context. These questions likely represent common knowledge already internalized within the models' memories. 80,000 samples from the HotpotQA training split are processed through this pipeline and approximately 50% of the data are filtered out. We chose the frist 32,768 samples of processed data as our training set.
We then apply similar approach synthesize 128 samples from the HotpotQA validation set. For extrapolation performance testing, we synthesize test sets with different context lengths using the same pipeline. The number of wiki items ranges from 50 up 6400, corresponding context lengths of approximately 7K to 3.5M tokens.
A.4 EVALUATION
We extract answers from the model outputs using regular expressions, and we prompt the model to respond in the specified format. The chosen format is 'the answer is ANSWER.'
We employ the sub__em score for all benchmarks. This means that answer is considered correct if it contains all the elements the ground truth. When an answer consists of multiple parts and the expected response should include all of them, the score corresponds the proportion of correct parts provided.
Before evaluating the answers, normalize both the ground truth and the extracted responses. For example, we remove definite articles, ignore case distinctions, and apply similar standard normalization steps following previous work (Wan al., 2025; Hsieh et al., 2024; Yen et al., 2024).
Table 6 shows the generation configurations of baseline models.
Table 6: Generation configurations of baseline models.
Model
Context Length
Input/Output Tokens 120,000 / 10,000
QwenLong-LI (Wan et al., 2025)
128K
Qwen2..-fintrruct-IM Series (Yang et al., 2025)
IM
990,000 / 10,000
DeepSeck-RI ---iccsi-----ssil------- Series (Guo et al., 2025)
128K
120,000 / 10,000
Qwen2.5-Instruet Series( YYng al., 2024)
128K
120,000 / 10,000
NIAH n wwi.ahhsnnngeeefl 2, 3} in RULER (Hsieh et al., 2024) benchmark are used in our test.
The yaml configuration of RULER are presented in 7. In level 1, the "haystack" consists repetitive sentences, and the "needle" is seven-digit number associated with a magic word. For level 2, the "haystack" is composed longer essays. Level 3 goes a step further than Level where the "needle" is 36-character UUID string. Question and context are concated as the input of LLMs. We omit the answer prefix provided in original RULER benchmark since it is not compatible with AGEAGENT workflow.
18

---

Published as a conference paper at ICLR 2026
niah single 1:
task: niah
args:
type.haystack:
repeat
type needle k: words
num needle numbers num num.needle.v:
type needle.v v:
num.needee.:
miah.single
task: niah
args:
type hapstacks essay
type_needle.k: words
type needle v:
numbers
num needle k:
num.needle.v:
num.needle.q:
ninh.single.3:
task: niah
args:
type hapstakkst:::: essay
sypeeneedle.k: words
type needle.v. uuids
num needle k:
num needle v:
num num.needle.q:
Table 7: Synthetic Configuration used for NIAH task.
B COMPUTATION COMPLEXITY
adopt the floating-por poonn operations (FLOP) estimator for the Qwen2Model from verl Sheng et al. (2024) to compute the FLOP cost of both the baseline model and our proposed method. The results are shown in Figure 10. The baseline model exhibits an O(n²) complexity. while MEMAGENT achieves an (n) complexity.
1e19
Baseline
2.0
Momory ony Agant
Compute
0.5
256K
512K
Context Length
Figure 10: Floating point operations across context lengths from 8K to 4M
For the baseline model, the number of tokens required to process is q + + o, where represents the length for the problem, is the context length and represents the output length.
19

---

Published as a conference paper at ICLR 2026
For MEMAGENT, total FLOP cost is the sum of the FLOPs from all stages. The detailed stages involved are as follows:
Initializing: In the first stage, the model processes an input consisting + 200 + o, where 200 represents constant added to prompt the model to follow the MEMAGENT workflow.
Memory Updating: The number of repetitions determined by k: / 1. where is the variable component of the input. Each repetition requires an input of length q 200+ N
Final Answering:
The final stage processes input of length + 100 + o, which includes the accumulated output from the previous steps.
We set = 1024, o = 1024, 5000 and c is ranging from 8K to 4M calculate the final result.
C CASE STUDY
To further illustrate the proposed memory mechanism in detail, we conduct case study on generation trajectory MEMAGENT-14B. The input question is: The director of the romantic comedy 'Big Stone Gap" is based in what New York city? This 2-hop question with the following relevant Wikipedia entries:
1) Big Stone Gap is 2014 American drama romantic comedy film written and directed by Adriana Trigiani.
2) Adriana Trigiani is an Italian American best-selling author of sixteen books, television writer, film director, and entrepreneur based in Greenwich Village, New York City.
In the first round, the model is presented with the entry Ghost, which refers production team also based in New York. The model chooses to retain this potentially useful information for future use. In the second round, relevant context is provided; nevertheless, the model maintains its agent state, demonstrating robustness against distraction. In the third round, both relevant entries are presented. The model correctly identifies critical information and updates its memory accordingly, leading the correct answer: Greenwich Village, New York City. At this point, the reasoning process is complete. In the remaining rounds, the model's memory remains unchanged and is used produce the final response.
CHUNK1
Document 20: Ghost (production team)
Ghost is Swedish record producing and songwriting team, composed of Ulf Lindström and Johan Ekhé, based in New York City.
MEMORY
The provided section does not contain any information about the director of the romantic comedy "Big Stone Gap" or their base in New York City. However, it does mention production team called "Ghost" which is based in New York City. Since the problem is specifically asking about the director of "Big Stone Gap," and there is no direct information about the director the given section, we cannot infer anything directly related the problem from the provided text. Updated memory:
Ghost is Swedish record producing and songwriting team, composed of Ulf Lindström and Johan Ekhé, based in New York City. They are perhaps best known for writing and producing Swedish singer Robyn's three first studio albums.
The problem remains unanswered based the provided section. Additional information would be needed to determine the director of "Big Stone Gap" and their base in New York City.
20

---

Published as a conference paper at ICLR 2026
CHUNK2
...(No relevant document)
MEMORY2
The provided section does not contain any information about the director of the roman- tic comedy "Big Stone Gap" or their base in New York City. 2 Therefore, the updated memory remains unchanged with respect to the problem at hand. The memory still notes that the Ghost record producing and songwriting team is based in New York City, but does not provide any information about the director of "Big Stone Gap." Updated memory:
Ghost is a Swedish record producing and songwriting team, composed of Ulf Lindström and Johan Ekhé, based in New York City. They are perhaps best known for writing and producing Swedish singer Robyn's 's three first studio albumss.
The problem remains unanswered based on the provided section. Additional infor- mation would be needed to determine the director of "Big Stone Gap" and their base in New York City.
CHUNK3
Document 81: Big Stone Gap (film)
Big Stone Gap 2014 American drama romantic comedy film written and directed by Adriana Trigiani and produced by Donna Gigliotti for Altar Identity Studios, subsidiary of Media Society.
... Document 96: Adriana Trigiani Adriana Trigiani is Italian American best-selling author of sixteen books, television writer, film director, and entrepreneur based in Greenwich Village, New York City.
MEMORY3 Updated memory:
Ghost is Swedish record producing and songwriting team, composed of Ulf Lindström and Johan Ekhé, based in New York City. They are perhaps best known for writing and producing Swedish singer Robyn's three first studio albums.
Adriana Trigiani is Italian American best-selling author of sixteen books, tele- vision writer, film director, and entrepreneur based in Greenwich Village, New York City. Trigiani has published novel year since 2000.
Big Stone Gap is 2014 American drama romantic comedy film written and directed by Adriana Trigiani and produced by Donna Gigliotti for Altar Identity Studios, subsidiary of Media Society. Based on Trigiani's 's 2000 best-selling novel of the same name, the story is set in the actual Virginia town of Big Stone Gap circa 1970s. The film had its world premiere the Virginia Film Festival on November 2014.
The director of the romantic comedy "Big Stone Gap" is Adriana Trigiani, who is based Greenwich Village, New York City.
This trajectory highlights several core capabilities learned by the memory agent: estimating potentially relevant content from query keywords and storing preemptively, immediately updating memory upon encountering context that matches the query, and remaining unaffected by irrelevant information. Notably, these memory behaviors are not the result of architectural attention mechanisms, but emerge as text generation abilities reinforced through RI.
21

---

Published as a conference paper at ICLR 2026
D SUPPLEMENTARY EXPERIMENTS
D.1 ABLATION STUDY ON CONTEXT SIZE AND MEMORY SIZE
In this ablation experiments, we keep the total length of memory size + context chunk size constant, and linearly adjusted the memory size. The goal is to control the total context length per conversation turn. The results are shown Table and Table
Table 8: Ablation result of memory-size and context-size on NIAH, All number indicates the averaging score of level1level3
Method
8K
16K
32K
64K
128K
256K
512K
14B-m4096-c1928
100.00
99.22
98.70
97.66
98.96
99.22
97.92
14B-m3072-c2252
99.74
100.00
100.00
100.00
98.70
99.48
99.48
14B-m2048-c3976
99.74
99.48
99.48
100.00
100.00
97.13
97.66
14B-m1024-c5000
99.74
98.70
100,00
100.00
98.96
97.40
98.18
7B-m4096-c1928
100.00
99.22
98.44
96.35
98.96
98.18
97.14
7B 7B-m3072-c2952
99.48
100.00
99.74
99.48
96.88
98.18
96.09
7B-m2048-c3976
99.74
99.48
99.48
99.48
98.96
94.53
94.53
7B-m1024-c5000
100.00
98.70
99.74
98.96
97.40
97.66
96.62
Table 9: Ablation result of memory-size and context-size on Longbench-QA
Method
2Wiki
HQA
MuSiQue
NQA
Qasper
AVG
I4B-m4096-c1928
74.5
72.5
48.5
21.5
25.5
48.5
14B-m3072-c2952
76.5
70.5
52.5
24.5
26.5
50.1
14B-m2048-c3976
74.5
71.5
49.5
23.0
27.0
49.1
14B-m1024-c5000
79.0
73.0
52.0
25.0
26.0
51.0
7B-m4096-c1928
70.0
66.0
45.5
19.0
26.0
45.3
7B 7B-m3072-c2952
72.0
64.0
44.0
20.0
25.5
45.1
7B-m2048-c3976
75.0
69.0
43.5
23.0
26.0
47.3
7B-m1024-c5000
74.0
69.5
47.0
21.5
29.0
48.2
D.2 AGENT BASELINES
We compare MemAgent against advanced memory-agent method, MemO(Chhikara et al., 2025). The Mem0 paper also reports that RAG methods using only top-1 or top-2 retrieval form strong and stable baselines for memory- agent tasks. Therefore, we conduct extensive comparisons against RAG agents under multiple configurations.
For MemO, and embedding model respectively and we follow the official GitHub repository for memory updating and retrieval. Specifically, during memory creation, split and processed the entire context in 5,000-token chunks; during retrieval, we selected the top 30 memories.
SOTA OpenAl models, GPT-5.1 and text-embedding-3-laad
langugae model
For RAG, also text-embedding-3-bee chunk size and top-K value.
embedding model and configure it with various
The results show that MEMAGENT outperforms these methods, demonstrating that end-to-end RL-trained memory provides greater flexibility and coherence compared with retrieval-based strate- gies.
22

---

Published as a conference paper at ICLR 2026
Table 10: Result versus RAG Agent RULER-HQA with different top-K settings. We segment the context based natural semantic units, i.e., each wiki item was treated as chunk.
Length
Model
7K
14K
28K
56K
112K
224K
448K
896K
1.75M
3.5M
RAG Qwen2.5-14B
K=2
57.03
54.69
51.56
53.12
50.00
52.34
49.22
48.44
48.44
K=4
66.41
67.19
68.75
67.19
66.41
64.06
66.41
64.84
60.94
59.38
K=6
72.66
75.78
75.78
74.22
69.53
71.88
73.44
67.19
65.62
66.41
K=8
78.12
78.91
77.34
81.25
76.56
78.12
77.34
74.22
70.31
64.84
RI-MEMAGENT-IAL
80.47
82.03
82.03
83.59
81.25
77.34
79.69
75.78
78.91
71.09
RAG + Qwen2.5-7B
K=2
53.91
54.69
53.12
51.56
54.69
51.56
52.34
49.22
48.44
46.09
K=4
67.19
66.41
66.41
67.19
64.84
64.06
62.50
61.72
60.94
59.38
K=6
74.22
73.44
72.66
73.44
70.31
73.44
70.31
67.19
65.62
65.62
K=8
75.00
75.00
75.78
74.22
74.22
77.34
72.66
68.75
64.06
64.84
RL-MEMAGENT-7E
81.25
81.25
82.03
79.69
75.78
76.56
74.22
77.34
71.88
Table 11: Result versus RAG Agent Longbench- QA with different top- and Context size settings. We segment the context using fixed-length chunks. For retrieval, we performed top-k matching using cosine similarity scores.
Method
2Wiki
HQA
MuSiQue
NQA
Qasper
AVG
wen2.55144 + RAG
C= 1024 K=2
51.50
56.50
26.50
15.00
23.50
28.83
C=1024 K=4
70.00
64.50
34.50
17.50
27.00
35.58
C= K=6
71.50
64.00
41.00
19.00
27.50
37.17
C= 1024 K=8
72.50
64.50
39.00
17.50
26.00
36.58
C=2048 K=2
58.50
61.50
33.50
13.50
25.50
32.08
C=2048 K=4
76.00
64.00
36.00
18.50
25.00
36.58
C=2048 K=6
73.00
67.50
41.50
21.00
26.50
38.25
C=2048 K=8
77.50
68.50
42.00
21.00
27.50
39.42
RL-MemAgeett114B
79.0
73.0
52.00
25.00
26.00
51.00
Wwen2.5-7B RAG
C=1024 K=2
41.00
48.50
22.00
14.50
25.50
25.25
C=1024 K=4
49.00
56.50
28.00
17.00
28.50
29.83
C=1024 K=6
54.50
57.50
29.50
17.00
25.00
30.58
C= K=8
50.50
59.00
29.50
18.00
25.00
30.33
C=2048 K=2
49.50
51.50
19.50
12.50
27.00
26.67
C=2048 K=4
50.50
53.00
26.00
17.00
25.50
28.67
C=2048 K=6
50.50
56.50
27.50
22.00
25.50
30.33
C=2048 K=8
50.50
58.00
25.50
21.00
27.00
30.33
RI-MMemAgent-7B
74.00
69.50
47.00
21.50
29.00
48.20
E
LLM USAGE
In this section, report the usage of LLMs in this work. Some sentences in this manuscript drafted or refined by LLMs, but all text is finalized by human authors. the experimental process, L.LMs assist with code completion, but they do not produce novel ideas complete experiments.
23

---

Published as a conference paper at ICLR 2026
F FALIURE PATTERN STUDY
F.1 INFORMATION OVERWRITTEN
In this example, the model accumulates large amount of irrelevant memories in the early stage (Turn 60). When the crucial context appears, the model does capture the relevant information (Turn 289), but attempts append it to the end (Turn 290), where is truncated due to insufficient memory. Later, the model proactively performs summarization (Turn 317), which gives it more space to store the second key piece of context (Tum 433). However, since the information from the first context has already been lost, the model incorrectly relies on irrelevant information for reasoning and fails to answer the question correctly.
Question: Where is the company that Sachin Warrier worked for as a software engineer headquartered?
Relevant Information:
(shown in Turn 289) Tata Consultancy Services Limited (TCS): An Indian multinational information technology (IT) service, consulting business solutions company. subsidiary of the Tata Group, operates in 46 countries, and is headquartered in Mumbai, Maharashtra.
(shown in Turn 433) Sachin Warrier: A playback singer and composer in the Malayalam cinema industry from Kerala. He became notable with the song "Muthuchippi Poloru" from the film Thattathin Marayathu, made his debut with Malarvaadi Arts Club, and working as a software engineer at Tata Consultancy Services in Kochi (later resigned to focus on music). His latest work is composer for the movie Aanandam.
Answer: Mumbai
Memory Diff between Turn and Turn 60
-The provided section does not contain any information about Sachin Warrier or his work as a software engineer. Therefore, the updated memory remains unchanged with respect Sachin Warrier, However, it does provide information about Texas Monthly, which is headquartered in Downtown Austin, Texas. Since problem specifically asking about company where Sachin Warrier worked a software engineer, and no information about Sachin Warrier is provided, the updated memory remains as follows: +The provided section does not contain any information about Sachin Warrier or his work as software engineer. However, it does provide information about Procter & Gamble, which is American consumer goods corporation headquartered in downtown Cincinnati, Ohio, United States of America. Since problem specifically asking about a company where Sachin Warrier worked a software engineer, and no information about Sachin Warrier is provided, the updated memory remains as follows:
Pizza Pizza Ltd. a franchised Canadian pizza fast sood restaurant, with headquarters in Etobicoke, Toronto. Its restaurants are mainly the province of Ontario. Other locations operate in Quebec, Nova Scotia, and in western Canada. Franchises in wester Canada mostly through Alberta-based subsidiary Pizza 73, and in non-traditional locations such as university campuses movie theatres throughout Canada. has over 500 locations. including over 150 non-traditional locations.
TI Automotive develops, manufactures, and supplies automotive fluid storage, carrying, and delivery systems. The company has 23,000 employees 120 locations in 29 countries, and supplies all the world's major automobile manufacturers. TI Automotive serves the automotive aftermarket through Bundy, Walbro, and Marwal brands. The company's 's headquarters are located in Oxford, UK, with Corporate Offices based in Aubum Hills, Michigan. In 2015 TI Automotive acquired by Bain Capital. (...)
24

---

Published as a conference paper at ICLR 2026
Memory Diff between Turn 282 and Turn 289
-The provided section does not contain any information about Sachin Warrier the company he worked for as software engineer. Therefore, the updated memory remains as follows: +The provided section contains information about Tata Consultancy Services Limited (TCS), which is an Indian multinational information technology (TT) service, consulting and business solutions company. It states that TCS is headquartered in Mumbai, Maharashtra. + Uppateed memory:
Cosmos (formerly Cosmos Tours) is UK independent operator providing range of package holidays the UK market. tour operator connected the international Globus Travel Group, founded 1928, which remains family owned with headquarters in Lugano, Switzerland. Under parent company Cosmos Tours Ltd, the tour operator offers holidays through three brands: Cosmos, Archers Holidays and Avalon Waterways.
L&F Jones Holdings Ltd is company based in Westfield, Somerset, England, that includes a convenience store chain, wholesale food business, and a Best Western PLUS hotel. The store chain comprises twelve shops in Somerset, Wiltshire, Dorset, South Gloucestershire, and Bristol.
(...)
Round Table Pizza is a large chain pizza parlors western United States. The company's 's headquarters are located in Concord, California.
Round Table Pizza is large chain of pizza parlors in the western United States. The company's 's
Anton Media Group, formerly Anton Community
Memory Diff between Turn 289 and Turn 290
-The provided section contains information about Tata Consultancy Services Limited (TCS), which is an Indian multinational information technology (IT) service, consulting and business solutions company. It states that TCS is headquartered in Mumbai. Maharashtra.
+The provided section does not contain any information about Sachin Warrier or Tata Consultancy Services Limited (TCS). Therefore, the updated memory remains as follows: -Updated memory:
Cosmos (formerly Cosmos Tours) is UK independent tour operator providing a range of package holidays to the UK market. The tour operator is connected to the international Globus Travel Group, founded in 1928, which remains family owned with headquarters in Lugano, Switzerland. Under parent company Cosmos Tours Ltd, the operator offers holidays through three brands: Cosmos, Archers Holidays and Avalon Waterways.
L&F Jones Holdings I Ltd is company based in Westfield, Somerset, England, that includes convenience chain, wholesale food business, and a Best Western PLUS hotel. The store chain comprises twelve shops Somerset. Wiltshire, Dorset, South Gloucestershire, and Bristol.
Tata Consultancy Services Limited (TCS), an Indian multinational information technology (
25

---

Published as a conference paper at ICLR 2026
Memory Diff between Turn 316 and Turn 317
The provided section does not contain any information relevant Sachin Warrier Tata Consultancy Services Limited (TCS). Therefore, the updated memory remains as follows:
(formerly Cosmos Tours) UK independent tour operator providing a range package holidays to the UK market. The tour operator connected to the international Globus Travel Group, founded 1928, which remains family owned with headquarters in Lugano, Switzerland. Under parent company Cosmos Tours Ltd, the tour operator offers holidays through three brands: Cosmos, Archers Holidays and Avalon Waterways.
L&F Jones Holdings is company based in Westfield, Somerset, England, that includes a convenience store chain, wholesale food business, and a Best Western PLUS hotel. The store chain comprises twelve shops in Somerset, Wiltshire, Dorset, South Gloucestershire, and Bristol.
(...)
The company's 's corporate headquarters Faridabad, Haryana, India.
Diageo plc (or is British multinational alcoholic beverages company, with its headquarters in London, England. It
(...)
Since the problem is specifically about the headquarters of the company Sachin Warrier worked for as software engineer, and the previous memory already states that Tata Consultancy Services Limited (TCS) is headquartered Faridabad, Haryana, India, the updated memory remains follows: The company's 's corporate headquarters in Farid
Memory Diff between Turn 430 and Turn 433
-The provided section does not contain any information about Sachin Warrier his employment as software engineer. Therefore, the updated memory remains follows: +The provided section contains information about Sachin Warrier, which is useful answering problem. Specifically, states:
+ Sachin Warrier is a playback singer and composer in the Malayalam cinema industry from Kerala.
He was working software engineer in Tata Consultancy Services in Kochi.
+ Later resigned from the job to concentrate more on music.
+However, the section does not provide the headquarters information for Tata Consultancy Services. Given the previous memory, we know that the corporate headquarters of Tata Consultancy Services is in Faridabad, Haryana, India. Therefore, the updated memory follows:
The company's 's corporate headquarters is in Faridabad, Haryana, India.
SpaceX an American aerospace manufacturer and space transport services company head- quartered in Hawthorne, California. (...)
Sachin Warrier's company headquarters information is still unknown from the given docu- ments.
Sachin Warrier was working as a software engineer in Tata Consultancy Services in Kochi. Later he resigned from the job to concentrate more on music.
AVG Technologies security software company headquartered in Amsterdam, Netherlands.
(...)
-The updated memory reflects that the headquarters of the companies mentioned in Faridabad, Haryana. India; Hawthorne. California: Amsterdam, Netherlands: Eagan, Minnesota; Pittsburgh, Pennsylvania; and San Antonio, Texas. However, there is no information about the company Sachin Warrier worked for as software engineer. Therefore, the memory remains unchanged in terms of Sachin Warrier's company headquarters information.
+The updated memory confirms that Sachin Warrier worked software engineer for Tata Consultancy Services in Kochi, and the headquarters Tata Consultancy Services Faridabad, Haryana, India.
26

---

Published as a conference paper at ICLR 2026
F.2 MISSING CRITICAL INFORMATION
example, when model encounters first key piece d'information (Turn 143), fails recognize as such because this is a multi-hop question and the prerequisite evidence has not yet been observed. Consequently, even though the model becomes aware of the problem upon seeing the second piece of information (Turn 215). it still fails produce the correct final answer.
Question: who is the younger brother of The episode guest stars of The Hard Easy ?
Relevant Information:
(shown in Turn 143) Brian Doyle-Murray Brian Doyle-Murray (horn Brian Murray, October 31, 1945) an American actor, voice artist, comedian and screenwriter. He is older brother actor/comedian Bill Murray, and the two have acted together in several films...
(shown in Turn 215) The Hard Easy (Adventure Time) "The Hard Easy" is the twenty-thin episode... The episode guest stars Brian Doyle-Murray Prince Huge and Jonathan Katz the Mud- scamp elder.
Answer: Bill Murray
Memory Diff between Turn 142 and Turn 143
-The provided section does not contain any information related to "The Hard Easy" or its guest stars. Therefore, the updated memory remains follows:
+The provided section does not contain any information related to "The Hard Easy" or guest stars. However, it does provide information about Brian Doyle- Poyle-Murray, who is the older brother of actor/comedian Bill Murray. Since the problem is asking for the younger brother of the episode guest stars of "The Hard Easy." and the memory does contain any information about the guest stars of "The Hard Easy," the information about Brian Doyle-Murray and Bill Murray is directly relevant the problem at hand.
+ Thereeoore, the updated memory remains as follows:
(..)
Martin Lee Truex Jr. (born June 29, 1980) is an American championship professional stock car racing driver. His younger brother, Ryan, champion in the K&N Pro Series East division and currently races in Camping World Truck Series.
Memory Diff between Turn 185 and Turn 215
-The provided section does contain any information related "The Hard Easy" or guest stars. Therefore, the updated memory remains unchanged with respect to the problem at hand. The only relevant piece of information is that the younger brother of Martin Lee Truex Jr. is Ryan Truex. +The provided section contains information about "The Hard Easy." episode of "Adventure Time." which includes the guest stars Brian Doyle-Murray Prince Huge and Jonathan Katz the Mudscamp elder. However, does not provide any information about the younger brother episode guest stars.
-Thus, the updated memory is as follows: +The updated memory is as follows:
Martin Lee Truex (born 29, 1980) is an American championship professional stock car racing driver. His younger brother, Ryan, a champion in the K&N Pro Series Fast division and currently races in the Camping World Truck Series. (...)
The younger brother of Martin Lee Truex is Ryan Truex.
+The provided section does not contain any information about the younger brother of the guest stars of "The Hard Easy." Therefore, the updated memory remains unchanged with respect the problem hand. The only relevant piece of information that the younger brother of Martin Lee Truex Jr. Ryan Truex.
27

---

Published as a conference paper at ICLR 2026
F.3
PRIMACY BIAS
This example involves primacy bias. The word country can refer either nation the countryside. The model initially assumes refers nation (Tum and subsequently reinforces this helief when it encounters additional information about countries (Tum 617), result. even though eventually also attends to truly critical piece of information (Tum 728), still fails to answer the question correctly.
Question: Brown State Fishing Lake is in country that has a population of how many inhabitants
Relevant Information:
(shown in Turn 728) Brown County, Kansas Brown County (county code BR) is a county located in the northeast portion of the U.S. state of Kansas, As of the 2010 census, the county population was 9,984. Its county and Indian Reservation of Kansas, the majority of the Sac and Fox Reservation and majority the Iowa Reservation of Kansas and Nebraska.
populous city is Hiawatha. Brown County the location of the Kickapoo
Answer: 9,984
Memory Diff between Turn 0 and Turn 1
-The provided section does contain any information about the population of the country where Brown State Fishing Lake is located. However, does confirm that Brown State Fishing Lake is in Brown County, Kansas in United States.
+The provided section does not contain any information about the population of the United States or country. Therefore. the updated memory remains as follows:
Brown State Fishing Lake is protected area in Brown County, Kansas in the United States.
The lake 62 acres (0.25 km²) area and up feet (4 m) deep. (...)
-Since the problem is asking for the population of the country where Brown State Fishing Lake is located, and only country mentioned in text the United States, we can infer that the question seeking the population of the United States. However, text does not provide the exact population number.
-Given the information available, the updated memory should reflect the following:
+Since the problem is seeking the population the country where Brown State Fishing Lake is located, the only country mentioned the United States, updated memory reflects that population of the United States is the required information. However, the text does not provide the exact population number. Therefore, the updated memory retains the above information without changes related to the population.
Memory Diff between Turn 616 and Turn 617
-The provided section contains information about Kunming, a city in Yunnan Province, Southwest China, which has population of 6,626,000 as of 2014. However, this information is not directly related to the problem at hand, which is about the population of the country where Brown State Fishing Lake is located.
+... Since the problem is asking the population of the country where Brown State Fishing Lake located, and we know that Brown State Fishing Lake is in Brown County, Kansas, the United States, we can infer that the population of the United States is the answer to the problem.
-The updated memory remains consistent with the previous information, but it now includes the new information about Kunming: + Updated memory:
Brown State Fishing Lake is protected in Brown County, Kansas in the United States.
The lake is 62 acres (0.25 km3) in area and up to 13 feet (4 m) deep. (...)
Ruzhou, city in Henan province, China, has than 100,000 inhabitants. (...)
As of 2018, the United States had population of around 227,167,434,
+The problem asking for the population the country where Brown State Fishing Lake is located, which is the United States. Therefore, the population of the United States is around 327,167 ,434 as of 2018.
28

---

Published as a conference paper at ICLR 2026
Memory Diff between Turn 654 and Turn 728
The provided section does contain any information relevant the problem at hand. Therefore, the updated memory remains as follows:
Brown State Fishing Lake is protected area Brown County, Kansas the United States.
(...)
Brown County, Kansas, as of the 2010 census, had a population of 9,984. (...)
As of 2018, the United States had population of around 327,167,434.
The problem is asking for the population of the country where Brown State Fishing Lake is located, which is the United States. Therefore, the population of the United States is around 327,167,434 of 2018.
29


Reviews:
[{"review_id": "QIXKBpljdy", "reviewer": "Reviewer_FzMS", "summary": "This paper presents MEMAGENT to enable LLMs to process arbitrarily long contexts efficiently by learning to maintain and overwrite a fixed-length memory.\nRather than increasing the model’s context window or modifying its architecture, MEMAGENT processes long documents in segments (chunks) and uses a learned memory update policy to selectively retain and discard information relevant to the final task.\nEmpirical results show that MEMAGENT significantly outperforms strong long-context baselines such as Qwen2.5-Instruct-1M, QwenLong-L1, and DeepSeek-Distill.", "questions": "What is the comparison with other long-context modeling methods and memory-based methods?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. Memory update is formulated as a reinforcement learning setting.\n2. The experiments across RULER-HQA, LongBench-QA, and NIAH show the effectiveness of the proposed method.\n3. Detailed ablation study is conducted.", "weaknesses": "1. The comparison is weak. There are some other long-context modeling methods such as FocusLLM (FocusLLM: Precise Understanding of Long Context by Dynamic Condensing) and E2LLM (E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning) which also use chunks. Moreover, there are also some memory-based methods such as Mem0. The proposed method should be compared with them.\n2. Despite inference efficiency, the RL training process is computationally heavy. The time complexity of the proposed method is not discussed.\n3. While the model overwrites memory continuously, the paper does not empirically analyze failure cases where crucial early information is overwritten, nor propose mitigation strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qtNZl0p8AX", "reviewer": "Reviewer_BjEW", "summary": "This paper introduces MEMAGENT, a novel agent-based framework designed to enable LLMs to process arbitrarily long contexts with linear time complexity. The core idea is to process a long document in discrete segments. The model maintains a fixed-size \"memory\" buffer within its context window, which it iteratively updates after processing each segment. This overwrite strategy avoids the quadratic complexity of standard attention mechanisms.\n\nThe key contribution is the training methodology. The authors formulate the memory update mechanism as a policy to be learned through RL. They propose Multi-Conv DAPO, an extension of DAPO, to optimize the entire multi-step agent workflow based on a final outcome reward. In experiments, a model with an 8K context window trained using the MEMAGENT workflow demonstrates remarkable extrapolation capabilities, successfully handling QA tasks on documents up to 3.5 million tokens with minimal performance degradation, significantly outperforming existing long-context models.", "questions": "Section 2.3 presents a theoretical latent-variable view of the process. How does this formal interpretation connect to the practical implementation of the RL agent? Is the agent learning to approximate a posterior distribution over memory states, or is it learning a more deterministic policy to generate a single memory trajectory?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The agent-based memory workflow is an elegant and practical solution to the long-context problem, sidestepping the quadratic complexity of attention.\n2. The experimental results are outstanding. The ability to extrapolate from an 8K training context to a 3.5M token QA task with less than 10% performance drop is great.\n3. By design, the method scales linearly with the length of the input document in terms of both time and memory, making it highly efficient for real-world deployment on extremely long texts.\n4. The paper successfully demonstrates that RL can be used to train an LLM to perform a complex, multi-step reasoning task like dynamic memory management, with the proposed Multi-Conv DAPO algorithm being a key enabler.", "weaknesses": "1. The fixed-size memory is the source of the method's efficiency, but it's also a potential bottleneck. For tasks that require synthesizing many disparate pieces of information from across a long document, the model might discard critical information prematurely. The paper could discuss this trade-off more explicitly and analyze failure cases where this occurs. Moreover, I think a better choice is to use variable-sized memory based on the amount of information contained in the context. How to implement this in the current framework should be explained and experimented with, if possible.\n2. The evaluation is heavily focused on extractive QA. It is unclear how well this memory management strategy would generalize to other long-context tasks like summarization, multi-document comparison, or creative writing, which may require different memory retention patterns.\n3. The RL-based training setup is sophisticated, involving a custom algorithm (Multi-Conv DAPO) and multiple stages. This poses a high barrier to reproduction for researchers without access to similar frameworks and expertise. More detailed pseudocode or implementation guidance would be beneficial.\n4. The choice to apply the final advantage value uniformly to all preceding memory-update steps is a simple credit assignment strategy. This could be suboptimal, as some memory updates are likely more critical than others. This represents a potential area for future improvement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1VvejNlyjD", "reviewer": "Reviewer_BZGs", "summary": "The paper introduces MemAgent, a novel agent-based workflow designed to allow LLMs to process arbitrarily long documents with linear time-complexity and minimal performance loss. Instead of processing an entire document at once, MemAgent divides the text into smaller chunks. It then iteratively reads each chunk alongside a fixed-length \"memory\" buffer. After each chunk, the model uses a simple \"overwrite\" strategy to update this memory with what it deems important.The core innovation is in its training.\n\nThe model is trained using a new Reinforcement Learning (RL) algorithm called Multi-conv DAPO. This method treats the entire sequence of memory updates for a single document as one long trajectory. A reward is given only for the final answer's correctness, and this reward is used to optimize all preceding memory-writing decisions. Experiments show that this approach is highly effective. A model trained with an 8K context window (using a 5K chunk and 1K memory) can extrapolate to answer questions on documents as long as 3.5 million tokens with less than 10% performance degradation—a task where standard long-context models fail completely.", "questions": "1. Do the authors have any intuition how this may be adapted to other tasks, like long-form summarization, where there is no verifiable outcome reward?\n2. I am wondering if the authors have explored the lost-in-the-middle for MeMAgent, i.e. do the models still perform worse when the crucial information is in the middle or can this strategy effectively fix this problem?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. **Strong Performance:** Extrapolating from an 8K training context to 3.5M tokens shows only minimal performance loss,  effectively solves the crucial problem in long-context modeling. The strong results on RULER-HQA, LongBench-QA, and NIAH demonstrate state-of-the-art performance and generalization.\n2. **Novel and Effective RL Framework:** The Multi-conv DAPO algorithm is a clever solution to a difficult credit-assignment problem. By propagating the final answer's reward back to all intermediate memory-update steps, the model is directly optimized to learn what to remember to succeed at the final task. This shows an easy way to improve current model's ability.\n3. **Well written paper**: The paper is well written and easy to follow.", "weaknesses": "1. Memory management seems to be a new lost-in-the-middle: In the ablation shown in Figure 9, increasing memory size does not seem to improve performance but rather decrease, and so this still demonstrate the model inherent problem with handling long-context (in this case the context window of the memory).\n2. The memory length is different for different tasks, and this may seem an additional hyper-parameter which may be costly to tune.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mHQE8vx47t", "reviewer": "Reviewer_JGDV", "summary": "The paper presents MEMAGENT, an agent that processes long documents chunk-by-chunk, using an RL-trained policy to update a fixed-size text buffer that serves as its memory. This sidesteps quadratic attention costs, achieving linear-time inference. The method demonstrates SOTA performance on several long-document QA benchmarks.", "questions": "* Can you provide a failure mode analysis, especially in the 3.5M token QA task. Does the distribution of errors related to the distance of required context? \n* To better show the generalization of MemAgent, Could you provide results on a high-quality long-document summarization benchmark? \n* The performance comparison is against an independent LLM out-of-box or fine-tuned. Could you discuss the performance comparison with other agent solution, too?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "* This paper proposed a novel agent-based framework with clever memory design that gets excellent performance on long-document QA \n* The approach reduce the complexity to linear-time inference complexity.", "weaknesses": "* The paper claims solving the problem of long-context LLM but mostly tested on synthetic or  semi-synthetic NIAH, RULER-HQA and extractive QA benchmarks. Form the memory architecture and prompt setup, this solution should well-suit the benchmarks given the major capability tested in these benchmarks are locating discrete facts. However, the same memory design may not work well with other tasks like summarization, etc.\n* One major concerns regarding to the memory design is the error propagation. If there is a single step injecting error in the memory, it will propagate through the memory and finally affect the final results. More analysis on the failure patterns would make this draft more sound. \n* The comparison is not sufficiently fair. The comparison mainly shows the performance diff between iterative-memory and single-pass attention. The experiment is hard to conclude that MEMAGENT is a \"better\" long-context model, but rather that an iterative approach is superior for these specific retrieval-heavy tasks. A more sound comparison would involve other iterative or memory-based techniques.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)