ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problems
Authors: Egor Cherepanov
Alexey Kovalev
Aleksandr Panov
Venue: ICLR 2026 Poster (2026)
Keywords: RL
POMDP
Memory
Transformer
Robotics
Primary area: reinforcement learning
TL;DR: ELMUR is a transformer model with layer-local external memory and LRU-based memory updates for long-horizon reasoning in POMDPs

Abstract: Real-world robotic agents must act under partial observability and long horizons, where key cues may appear long before they affect decision making. However, most modern approaches rely solely on instantaneous information, without incorporating insights from the past. Standard recurrent or transformer models struggle with retaining and leveraging long-term dependencies: context windows truncate history, while naive memory extensions fail under scale and sparsity. 
We propose ELMUR (External Layer Memory with Update/Rewrite), a transformer architecture with structured external memory. Each layer maintains memory embeddings, interacts with them via bidirectional cross-attention, and updates them through an Least Recently Used (LRU) memory module using replacement or convex blending. 
ELMUR extends effective horizons up to 100,000 times beyond the attention window and achieves a 100% success rate on a synthetic T-Maze task with corridors up to one million steps. In POPGym, it outperforms baselines on more than half of the tasks. 
On MIKASA-Robo sparse-reward manipulation tasks with visual observations, it nearly doubles the performance of strong baselines, achieving the best success rate on 21 out of 23 tasks and improving the aggregate success rate across all tasks by about 70% over the previous best baseline.
These results demonstrate that structured, layer-local external memory offers a simple and scalable approach to decision making under partial observability.

Published as a conference paper at ICLR 2026
ELMUR: EXTERNAL LAYER MEMORY WITH UP- DATE/REWRITE FOR LONG-HORIZON RL PROBLEMS
Egor Cherepanov Alexey K. Kovalev Aleksandr I. Panoy AAXX, MIRIIII
cherepanov@exxx.cer tech
ABSTRACT
Real-world robotic agents must under partial observability and long horizons, where key cues may appear long before they affect decision making. However, most modern approaches rely solely on instantaneous information, without incor- porating insights from the past. Standard recurrent or transformer models struggle with retaining and leveraging long-tew seeg-termmm dependencies: context windows truncate history, while naive memory extensions fail under scale and sparsity. We propose ELMUR (External Layer Memory with Update/Rewrite), transformer architec- ture with structured external memory. Each layer maintains memory embeddings, interacts with them via bidirectional cross-attention, and updates them through an Least Recently Used (LRU) memory module using replacement convex blend- ing. ELMUR extends effective horizons up to 100,000 times beyond the attention window and achieves 100% success rate on synthetic T-Maze task with corri- dors up to one million steps. In POPGym, it outperforms baselines on more than half of the tasks. On MIKASA-Roby sparse-reward manipulation tasks with vi- sual observations, it nearly doubles the performance of strong baselines, achieving the best success rate on 21 out of 23 tasks and improving the aggregate success rate across all tasks by about 70% over the previous best baseline. These results demonstrate that structured, layer-local external memory offers simple and scal- able approach to decision making under partial observability. Code and project page: https: / /lluur-paprrr wigggiit.iia. io/.
INTRODUCTION
Imagine a robot cooking pasta: it stirs once, adds salt, and later adds salt again, repeating until the dish is inedible. The issue simple: the robot cannot remember if salt was already added, since it dissolves invisibly, nor how much is still in the container. This is case of partial observability the world rarely reveals all necessary information. Humans recall past actions effortlessly. but robots lack this ability. Though effective in controlled settings (Kim et al., 2024; Black et al., 2024), robots often fail under partial observability (Fang et al., 2025; Cherepanov et al., 2026a). Standard recurrent (Ouyang al., 2025) and transformer (Gao et al., 2025) models rely heavily on short observation windows, making them brittle under hong-horizon dependencies and sparse signals. This motivates hybrid memory-augmente transformers that explicitly store and retrieve past information (Fang et al., 2025; Shi et al., 2025).
Within the Reinforcement Learning (RL) paradigm (Sutton al., 1998), long-horizon challenges are compounded by sample inefficiency and sparse rewards: real-world exploration is costly and unsafe, while simulation suffers from sim-to-real gap (Zhang et al., 2025a). Offline RL mitigates this with pre-collected datasets (Levine et al., 2020), but usually assumes dense feedback; reshaping sparse rewards demands domain knowledge and risks bias (Wu et al., 2021: Mu et al., 2024: Wang et al., 2025a). In robotics, delayed feedback makes long-term memory indispensable. A comple- mentary paradigm Imitation Learning
(IL.) (Zare et al., 2024), whose simplest form, Behavior Cloning (BC), reduces control to supervised learning on demonstration pairs. Building on this idea, recent Vision-Language- Lagguannn-gtttionn Action (VLA) models (Brohan et al., 2022: Team et al., 2024; Kim et al., 2024) scale with large datasets, yet their fixed transformer windows (Fang et al., 2025; Shi et al., 2025) leave three challenges: (i) extending context without quadratic cost, (ii) mitigating truncation- induced forgetting, and (ii) retaining task-relevant information across long horizons. This motivates our central question: how can we equip IL policies with efficient long-term memory to solve long-horizon, partially observable tasks?

---

Published as a conference paper at ICLR 2026
Token Track
x layes
Sequenne observation
Auss fto
Ausi Merora
First Segnent
Self-
mend tok
block
PokenFFN
Action
Add DOCKE
Hemozy
Loke block
Mcmory TYTEN
LRU
Nemory
Last Begnent
Memory Track
Inšo
comnrformes
dacodec
Figure 1: ELMUR overview. Each transformer layer is augmented with an external memory track that runs in parallel with the token track. Tokens attend to memory through a ok block, while memory embeddings are updated from tokens through tok2mem block. LRU block selectively rewrites memory via replacement convex blending, ensuring bounded yet persistent storage. This design enables token-memory interaction and long-horizon recall beyond the attention window.
To address these challenges, we introduce ELMUR (External Layer Memory with Update/Rewrite) a transformer architecture in which every layer is augmented with a structured external layer mem- ory (Figure 1). ELMUR combines three ingredients: (i) layer-local memory embeddings that persist across segments, (ii) bidirectional moken-memory read/write interaction via cross-attention (mem2tok, too2mem), and (iii) Least Recently Used (LRU) update block that refreshes memory through replacement or convex blending, balancing stability and adaptability. This design enables efficient segment-level recurrence and extends retention of task-relevant information up x00,000x beyond the native attention window, making Long-horizon decision making feasible in robotics.
We evaluate ELMUR the synthetic T-Maze (Ni al., 2023), the robotic MIKASA- Robo (Cherepanov et al., 2026a) suite of sparse-reward manipulation tasks with visual observations, and the diverse POPGym benchmark (Morad et al., 2023a), all designed to test memory under par- tial observability. ELMUR achieves a 100% success rate on T-Maze corridors up to one million steps, nearly doubles baseline performance on MIKASA-Robe ranking first on 21 of 23 tasks and increasing the overall success rate across the suite by roughly 70% relative to the strongest prior method, and obtains the top score on 24 of 48 POPGym tasks. These results demonstrate that EL- MUR enables stable retention of task-relevant information, efficient long-term storage, and robust generalization under partial observability.
Our contributions are twofold:
We propose ELMUR, transformer with layer-local external memory, bidirectional token- memory cross-attention, and an LRU-based update rule rewriting memory via replacement convex blending (Section 3). This design extends memory horizons far beyond the attention window.
We empirically demonstrate that ELMUR achieves robust generalization under partial observability across synthetic, robotic, and puzzle/control tasks (Section 5).
We provide theoretical analysis of LRU-based memory dynamics, establishing formal bounds on forgetting, retention horizons, and stability of memory embeddings (Section 4).
2 BACKGROUND
Many real-world robotic and control tasks involve partial observability, where the agent cannot directly access the true system state (Lauri et al., 2022). This setting is modeled as partially observable Markov decision process (POMDP), defined as the tuple (S,A, Z, R, pu, pu, 7), with latent state space S, action space A, and observation space O. The transition dynamics x A + A(S), where T(s' | s, a) is the probability reaching after taking a in s. The observation function Z S x A A ()) specifies 7(o 8', a), the probability of observing after
T
2

---

Published as a conference paper at ICLR 2026
Algorithm 1 ELMUR layer update for segment i at layer t. Inputs are token hidden states /t R Bxxxd, memory (m, p) with m Bx xd anchors p E 2BxM, and absolute times t. Outputs updated hidden states h' and memory (m',p').
I/ Input embedding (before first layer) to encode observation 1: h < ObsFmcooder(o)
// Token track sequence processing and enrichment with information from memory
2: h + AddNorm(( h + SelfAttention(ft: causal mask))
3: Biet + RelativeBias(t,, p) // bias for adding temporal dependence
4: h < AddNorm(ha (h + CrosssAlention(((-.)) K=m, V=m; noncausal mask, Bread)
5: h <- AddNorm(h (h TokenFFN(())
6: h/ +
// Output decoding (after final layer)
7: a + ActionHead((t) (h') // map to action distribution and compute loss
// Memory track
8: Biel + (e)ss,iveBias(p,t)) // reversed bias for write
9: u + AddNorm((m m + CrossAttention((nm)) K-h', V=hl; noncausal mask, Bwrite)
10: ū < Add
MemoryFFN(w))
11: (m', p') 4- LRU(m, Ps ú, t)
12: return h², (m',p)
reaching s' under action a. The reward function is R A - RR the initial state distribution is Po (S), and y E (0, 1) is the discount factor.
In the special case of full observability, the observation equals the state (o: st), reducing the POMDP to a Markov decision process (MDP). The optimal policy then depends only on the current state, n²" ra; st). In the general POMDP case, however, the agent cannot access s directly and must rely on the full history he = (ou. ag 01, a1..... oz). yielding π (a, he). A practical alternative to approximate history with learned memory state m, {m oe, at-1), To M - (AA), To (a, m²), where fo is a, for instance, recurrent (Hausknecht Stone, 201 5) or memory- augmented (Parisotto et al., 2020) update rule.
METHOD
LRU DetaSled
Initial memory after segment
m5
Many real-worid decision-making tasks involve long horizons and partial ob- servability, where key information may appear thousands of steps before it is needed. Standard transformers are lim- ited by a fixed attention window: naive extensions of context length increase cost quadratically, while truncation causes for- getting. Efficient long-term reasoning thus requires a mechanism store and retrieve task-relevant information across long trajectories. To this end, we pro- pose ELMUR (External Layer Memory with Update/Rewrite). GPT-style (Rad- ford et al., 2019) transformer decoder aug- mented with structured external memory. Unlike architectures that simply cache hid- den states (Dai al., 2019), ELMUR equips each layer with its own memory track and explicit read-write operations, enabling persistent storage and selective updating via the LRU memory manage-
m
New memory written to an empty
m
mi
+
Segract
ro
-New memory writtan in filled
Reguant
mk
m2
m
Segranz
MrL
+
M62
Figure 2: LRU-based memory management in EL- MUR. Each layer maintains M memory slots, initial- ized with random vectors (green). As segments arrive, tokens write updates into empty slots (purple) by full replacement. Once all slots filled, the least recently used slot is refreshed via a convex update with parameter À that blends new content with the previous memory (grey). Anchors below each row indicate the timestep of the most recent update. This scheme en- sures bounded capacity while preserving long-horizon information.
ment.
3

---

Published as a conference paper at ICLR 2026
ELMUR Overview. As shown in Figure 1, each ELMUR layer has two coupled tracks. The token track processes observations into actions, while the memory track persists across segments. Both interact through cross-attention: memory shapes token representations, and tokens update mem- ory. Interaction occurs via mem2tokk (read) and tok (write) blocks, modulated by relative biases from token timesteps and memory anchors. Trajectories are split into segments, processed sequentially for efficiency and recurrent memory updates. At each segment's 's end, hidden states up- date memory, carried forward. LRU memory management fills empty slots first, then refreshes the least used slot by convexly blending old and new information. This bidirectional design provides temporally grounded memory for Long-horizon decisions. Algorithm summarizes the method.
Segment- Recurrence. Feeding infinitely long sequences into a transformer is infeasible, since self-attention scales quadratically. Splitting into shorter segments reduces cost but complicates information flow. Segment-level recurrence addresses this by treating the transformer as an RNN over segments, passing memory from one segment to the next (Dai et al., 2019; Bulatov et al., 2022). In ELMUR, this memory is realized as layer-local external memory instead of cached activations. Each layer maintains memory that read within the current segment and updated before moving to the next. Formally, with context length L, trajectory of length T is partitioned into S = [T/L] segments S₁: h(a) Token Track (S,, sg(m (-1), where h(() Bx denotes the hidden states of tokens in segment i, computed from the segment S; and the detached memory sg( sg(m m i-1) carried from the previous segment.
Token Track. Within each segment S,, observations are encoded into token embeddings Rlxd, where d is the model dimension. The token track models local dependencies and augments them with information from memory m Mddxd Standard transformers rely on fixed-window self- attention, whereas ELMUR also retrieves information from its external memory via cross-attention, allowing predictions depend not only on recent tokens but on distant past events stored in memory. Self-attention, equipped with relative positional encodings (Dai et al., 2019) and causal mask, models local dependencies within the segment:
(1) where AddNorm(.) denotes residual connection followed by normalization. Long- term context is handled by external memory. Tokens' hidden states ha then query memory via the mem2tok:
hg AddNorm(x + SelfAttention(x)).
hmem2ok AddNorm(ha Cross Attentoooo haa K, V (/=m)
Here memory embeddings act as keys and values, with a non-causal mask and a relative bias re- flecting token-memory temporal distance. Finally, representations are refined with a feed-forward network (FFN). In contrast to popular Decision Transformer (DT) Chen et al. (2021) that employ a standard MLP-based FFN, we adopt a DeepSeek-Mor sign of DeepSeck-Vy V3 (Liu et al., 2024a). Mixture-of of-Experts (MoE) improve parameter efficiency and specialization by routing tokens to sparse set of experts, scaling capacity without proportional compute. This design enables expressive updates while keeping inference efficient:
FFN (Dai et al., 2024). following the de-
h AddNorm(bmentt
+ (PFN(mmem2uu)
The resulting hidden states are then passed to the action head, applied only after the final layer. Training is supervised, minimizing the error between predicted and demonstrated actions, using mean squared loss for continuous spaces and cross-entrope for discrete ones. The loss backpropa- gates through the entire network to update model parameters.
Memory Track. Reading from memory is not enough for long-horizon reasoning; the model must also write new information. Without an explicit write path, past events would be forgotten or cached inefficiently. The memory track addresses this by allowing tokens to update persistent memory, retaining salient information while overwriting less useful content.
Each layer maintains its own memory embeddings m E .R Mxd. After processing segment, token states update memory through the 2me2 block:
Mak2mem AddNorm(m ( + CrossAttention( = m, K,V = h))
(4)
As in mem2tok, a non-causal mask is applied, but the relative bias is reversed to favor temporally aligned memory embeddings. Updates are then refined by a FFN with residual connection:
maew AddNorm( (oNNoormmmmneeen (aok2mmm + FFN(
(5)
4

---

Published as a conference paper at ICLR 2026
analogous the token track, the FFN uses DeepSeek-Mon MoE block instead of standard MLP.
Finally, maow is merged with existing slots via the LRU rule (Figure 2, Figure 2), filling empty slots first and otherwise refreshing the least recently used by convex blending. This keeps memory bounded yet consistently updated with relevant information.
Relative Bias. When memory extends across multiple segments, absolute indices become am- biguous: the same token position may correspond to different points in the trajectory. To resolve this, the model requires a signal that encodes relative distances between tokens and memory entries. ELMUR provides this signal through a learned relative bias added to cross-attention logits:
QK
Attn(Q Q2 K)
+ B
(6)
vdh.
The bias B is derived from pairwise offsets A = (t--p) between token position / and memory anchor p (the last update time a slot). Offietts are clamped to [- 1max-11 where Dmax is the maximum relative distance supported by the bias table. These clamped values are shifted into [0. 2Dmax - 22 and used to index learnable embedding table E (2PPuux 1100mm11xx 1)x#f where H is the number of attention heads. Each offset corresponds to a per-head embedding E[4] E RH and stacking these indices produces
E(t p] E RBXHXLXM mem2tok (read)
Bie
Elp 1] E BB tok2mem (write).
In the read path (mem2tok), the bias prioritizes retrieval from temporally close memory embeddings while keeping dis- tant ones accessible. In the write path (tok2mem), offsets reversed, guid- ing updates toward memory embeddings aligned with the writing tokens. Both di- rections draw from the same embedding table E but can learn distinct patterns. By relying on relative rather than absolute timestep, ELMUR ensures consistent and coherent memory interactions across long horizons.
Algorithm 2 LRU update for layer memory. Inputs: current memory (m.p) (may be uninitialized), candi- date updates ú, newest segment time t, blend X [0, 1}, init scale o. Output: updated memory (m', .').
Initialization (cold start) 1: if m. p uninitialized then
2: m 4 NN,00 d²1) // initial slots
3:
- // sentinel anchors
4: end
// Choose write index
5: empty t (p < 0)
6: if any empty then
7:
j -- first(empty) 1/ use first empty slot
a a+1 full replacement
Memory Management with LRU. Ex-
9: else
ternal memory must remain bounded:
storing every token is infeasible, while
10:
+ arg migginn Pj least recently used a a // convex blend
naive truncation risks catastrophic forget- ting. A principled policy is needed to de- cide which slots to refresh or preserve as new content arrives. ELMUR employs Least Recently Used (LRU) block (Fig- ure 2, Figure 2) that manages M slots per layer, each holding vector and an anchor (its last update time). By always updating the least recently used slot, the block en- sures bounded capacity while retaining context.
11:
12: end if
// Integrate
13: blend +- a ujJ + (1 a)) my-
14: m' m; m/. + blend
15: p' + p; py +
16: return (m', p¹)
At training start, initialization samples embeddings from o0)) and marks them empty. While empty slots remain, full replacement inserts new vectors directly. Once all slots are filled, the block switches to convex update. blending the oldest slot with new content:
m A A
i+1
(1-A)
(8)
where E [0, 1] is a tunable hyperparameter that controls the balance between overwriting and retention. By adjusting X, one choose whether memory favors fast plasticity (larger X) long- term stability (smaller A). This policy uses memory capacity fully before overwriting and applies gradual blending thereafter, enabling bounded yet persistent long-horizon memory.
5

---

Published as a conference paper at ICLR 2026
By combining token-level processing with explicit memory system, ELMUR offers three advantages: (i) relative- bias cross- -oos-atttntionn provides temporally grounded read-write access, (ii) the LRU-based manager ensures bounded capacity while remaining adaptive, and (iii) segment-level recurrence enables scalable learning over long horizons.
THEORETICAL ANALYSIS
Understanding the retention properties of ELMUR's memory is crucial for characterizing its ability to handle Long-horizon dependencies. In this section, analyze how information is preserved or forgotten under the LRU update mechanism. We derive bounds on memory retention and effective horizons, and connect these results the empirical behaviors observed in long-horizeg tasks.
At the core of ELMUR's 's memory module the convex update rule with blending factor À E [0, 1]. Let fix memory embedding j at segment index i. If this memory embedding is selected for update with new content m i+1 memory embeddings n * remain unchanged: m, i+1 m If the memory embedding was empty, the update reduces to full replacement m,t i+1 i+1
new the rule (Figure 2) is m ++ X + (1 X) mf, while all other
new
Proposition 1 (Exponential Forgetting). evolves as
After overwrites of memory embedding j, the content
m 1
A)*
m
A(1
A)* m
where m+ denotes the write at update i+ (see Appendix A. for a full derivation). Consequently, the coefficient of the initial content mf after k overwrites is (1 - A)K, and the contribution of the write performed r updates earlier is A(1 - -)=-1.
Corollary (Half-life).
The number of overwrites ko.s after which the contribution of m halves is In2 as + Thus, smaller À extends retention, while larger
ko.s [u[1-X] X1
In(1/2)
x
accelerates overwriting.
Effective horizon in environment steps. Since only one memory embedding is updated per seg- ment of length L, a memory is overwritten once every M segments in expectation. The effective retention horizon (c) thus quantifies how many environment steps stored contribution remains
In(e) influential before its weight decays below a negligible threshold e, i..., H(e) = -M L in(I-X) In particular, the half-life in environment steps is Ho.5 M
M. "2 as X 0.
Unlike models where all memory updated at every step (like RNNs), ELMUR's LRU policy ensures (i) memory embeddings not selected for overwrite retain their content exactly until replace- ment, and (ii) once selected, their contributions decay exponentially with rate This produces a retention horizon that scales linearly with both the number of memory embeddings M and the seg- ment length J., providing a conservative lower bound. In practice, effective horizons are often much longer (Figure 3).
Proposition 2 (Memory Boundedness). A natural question is whether repeated convex updates could cause memory values grow without limit. We show that, under standard bounded-input assumptions, the norm of every memory embedding remains uniformly bounded throughout training and inference. Suppose that every new write norm-bounded |m || C for some constant C' 0, and the initial memory satisfies m, ol < C. Then for all segments i and slots j. it holds that
|| < C. Since each update is convex combination of the previous and bounded new values, the memory embedding always remains inside the closed ball of radius C. This guarantees stability of activations even across arbitrarily long trajectories. See Appendix A.2 for the detailed proof.

---

Published as a conference paper at ICLR 2026
5 EXPERIMENTS
We evaluate ELMUR on synthetic (Ni et al., 2023) tasks, 48 POPGym puzzle/control tasks (Morad al., 2023a), and robotic manipulation (Cherepanov et al., 2026a), all designed test memory under partial observability. Our study is guided by the following research questions (RQs):
1. RQI: Does ELMUR retain information across horizons far beyond its attention window?
2. RQ2: How well does ELMUR generalize to shorter and longer sequences?
3. RQ3: Is ELMUR effective on manipulation tasks with visual observations?
4. RQ4: How consistent is ELMUR across puzzles, control, and robotics tasks?
5. RQS: What is the impact of components of ELMUR on its memorization?
5.1 BENCIMARKS AND BASELINES
We evaluate ELMUR on three benchmarks designed to isolate memory (Appendix, Figure 7). The T-Maze requires recalling an early cue after traversing long corridor with sparse rewards. The MIKASA-Robo suite provides robotic tabletop tasks with RGB observations and continuous ac- tions, including color-recall (RememberColor) and delayed reversal (TakeItBack), Finally, POPGym offers a diverse collection of partially observable puzzles and control environments for evaluating general memory use. Detailed descriptions can be found in Appendix A.4.
We compare against baselines spanning sequence models and offline RL for long-horizon tasks. We include transformers - Decision Transformer (DT) (Chen et al., 2021) and Recurrent Action Transformer with Memory (RATE) (Cherepanov et al., 2026c) - representative architectures for nemory-augmentee policy learning. We also evaluate DMamba (Ota, 2024), a state-space model with efficient recurrence, Cloning (BC) via MI.P as the simplest supervised baseline, Conservative Q-Learning (CQL) (Kumar al., 2020) strong offline RL method, and Diffusion Policy (DP) (Chi et al., 2023) state- of-the-art generative policy. Together, these span transformer, state-space, and offiine/geeerative approaches, providing a competitive reference set for evaluation. We do compare with online RL baselines, since they assume interactive data collection with exploration, yielding incomparable training budgets. Likewise, we omit real-robot experiments avoid confounds such as latency, resets, and safety constraints, focusing instead on controlled, reproducible studies.
recent alternative attention. For IL/offfine RL,
Behavior
Experimental Setup. For RQI, we test T-Maze cue retention by training with short contexts (L-10, 8-3) and evaluating on corridors up to 10 steps. For RQ2, train on 7 T-Maze lengths distributions (9-900) and validate on 11 shorter/longer ones (9-9600) to assess interpo- lation and extrapolation. For RQ3, we use MIKASA- pert demonstrations and evaluating zero-shot. For RQ4, we compare T-Maze, POPGym-48, and MIKASA- Robo to test robustness across synthetie puzzles, control, and robotics. For RQS, we varying M, X, and (L, and remove relative bias, LRU, and
tasks, training by imitation from ex-
ablate CememberCooore comppon per-layer memory to measure component contributions.
Evaluation Protocol. Unless stated otherwise, each model is trained with three (four for T-Maze) independent runs (different initialization). For each run we evaluate on 100 episodes with distinct environment seeds and compute the run mean. We then report the grand mean standard error of the mean (SEM) across the three run means. For per-task leaderboards (e.g., POPGym-48) we apply this protocol per task and aggregate as specified in the benchmark.
Training Details and Hardware. All models are trained from scratch under the same data budgets and preprocessing. We use segment-level recurrence with detached memory between segments; losses are applied on each processed segment. Optimizers, schedulers, and hyperparameters follow the task-specific configuration table in Appendix, Table 7. All experiments were run single NVIDIA A 1000 (80 GB) per job. Training/evaluation code paths, seeds, and environment versions fixed across methods for reproducibility.
7

---

Published as a conference paper at ICLR 2026
1.0
Success Rate
0.4
0.2
ELMUR (Ours)
RMT TXXL
DT DMamba
BC-LSTM
Persistent agent
RATE
BC-MLP
Random agent
0.0 10
10
10 T-Maze Inference Corridor Length
10
10a
106
Figure 3: Success rate the T-Maze task function of inference corridor length. ELMUR achieves a 100% success rate up to corridor lengths of one million steps. this figure, the context length is L 10 with S 3 segments; thus ELMUR carries information across horizons 100,000 times longer than its context window.
5.2 RESULTS
Training Sogeessee Length
We evaluate ELMUR on T-Maze, MIKASA- Robo, and POPGym, addressing RQ1-RQ5 retention, generalization, manipulation, cross- domain robustness, and ablations.
303 Do
130
20
04
Retention beyond attention. To test memory retention, we train on T-Maze corri- dors of length T' while restricting the context size to L < T. forcing the model to solve tasks where the cue must be preserved beyond the na- tive attention span. At validation, we evaluate on much longer
RQ1:
Validation Sequence Length
Generalization of ELMUR across T-Maze lengths. Each cell shows success rate (mean = standard error) for training valida- tion lengths. ELMUR transfers perfectly: models trained shorter sequences retain 100% success up to 9600 steps. Training lengths were split into three equal segments.
Figure 4:
corridors- up to one million steps - without increasing L, thereby probing memory reten- tion far beyond the training horizon. ELMUR achieves 100% success even under this extreme extrapolation (Figure 3), implying retention hori- zons nearly 100,000 larger than the attention window (L=10 with only S=3 segments used during training).
RQ2: Generalization across sequence lengths. We train ELMUR on T-Maze with short contexts (3 to 300 steps) and then evaluate across 11 validation lengths ranging from 9 to 9600 steps. The model transfers seamlessly in both directions: it solves tasks shorter than those seen during training without overfitting
fixed scale, and it also extrapolates to sequences orders of magnitude longer. As shown in Figure 4, ELMUR maintains 100% success across all train/test pairs, demonstrating robust generalization beyond the training horizon.
RQ3: Manipulation with visual observations. Results in Table indicate that EL- MUR achieves higher success rates than other baselines on the MIKASA-Robo tasks. In
Table Success rates (mean + standard error) on MIKASA-Robo tasks, averaged over runs with 100 evaluation seeds. ELMUR outperforms baselines, showing stronger memory in manipulation. See results for all 32 MIKASA-RoOO Aooooo tasks in Appendix, Table 8
Task
RATE
DT
BC-MLP
CQL-MLP
DP
ELMUR (ours)
3
0.65+0.04
0.0110.01
0.27-00.3
0.29:10.01
0.32+0.01
0.89+10.07
RememberColort vO
0.13+0.03
0.0740.05
0.12+0.01
0.15+00.22
0.10+0.02
0.19:10.03
RemenberColorr-vv
0.09::.02
0.0110001
0.12-10.2
0.1510.01
0.1710.01
0.23-10.02
BakkeItBack-pp
0.42-:0.4
0.0811.04
0.33-10.10
0.0410.01
0.05+0.02
0.78-10.03
8

---

Published as a conference paper at ICLR 2026
TakeItBack-v0, it obtains 0.78+0.03 compared to 0.42+00224 for the next-best model, and in RememberColor [3, -v -v performance remains stable the number of distractors increases. Overall, ELMUR shows more reliable performance under visual interference in manipulation tasks with pixel inputs.
Table 2: Aggregated returns on 48 POPGym tasks.
RATE DT
Rand. BC-MLP BC-LSTM ELMUR
its
All (48)
9.5
5,8
-12.2
-6.8
9.0
10.4
Puzzle (33)
0.45
-3.5
-14.6
-11.9
-0.2
1.2
Reactive (15)
9.1
9.3
2.3
5.1
9.1
9.2
RQ4: Robustness across domains. Across synthetic (T-Maze), control/puzzle (48 POPGym), and robotic (MIKASA-Robo) benchmarks, ELMUR consistently outperforms baselines, generaliz- ing across diverse modalities, actions, and rewards. On POPGym, achieves the best overall score (10.4), with the largest gains on memory puzzles (1.2 vs. 0.45 for RATE; DT and BC-LSTM score below zero), showing the importance of explicit memory for long-term dependencies. On reac- tive tasks, ELMUR stays competitive without sacrificing puzzle performance, ranking first on 24 of 48 tasks (full results in Table 5). Figure 5 shows consistent per-task gains over DT, especially on memory-intensive puzzles. Improved retention comes with little overhead: on T-Maze, ELMUR has MM parameters (vs. 1.7M for RATE, 1.8M for DT) yet runs faster per step 06.8=0.55 ms) than RATE (7.2#0.3 ms) and DT (10.7=0.1 ms). Efficiency stems from (i) short attention window with long- term context handled by bounded memory, so complexity depends on memory size not sequence length, and (ii) MoE feed-forward layers, which raise capacity without proportional compute. Thus, explicit memory is both effective and efficient for long-horizon RL.
RQ5: Ablation Study. We ablate ELMUR's memory design (Figure 6, Table 3). Unless noted, models use per-layer memory, relative-bias token token-memory cross- attention, and LRU-based updates; shared memory denotes em- beddings shared across layers. In Figure 6 (b-d) the LRU factor fixed to À 0 to isolate other effects. Results average three runs of 20 episodes. Performance scales with memory size M: when M 2 N (the number of segments needed), success is near- perfect; when M < N, accuracy drops sharply, especially near M ss N (Figure 6, c-d). Intermediate blending (A A 0.4-0.6) is unstable (Figure 6, a), while larger initialization mitigates collapse (Figure 6, b). Finer recurrence (shorter segments, larger N) stresses capacity unless M scales accordingly. Component ablations confirm that capacity and LRU dominate. Removing LRU leaves stale entries, and removing both LRU and relative bias prevents effective retrieval. Relative bias gives modest gains, while shared memory degrades performance, underscoring the value of layer-local design. Finally, replacing MoR-FFN with MLP-FFN preserves accuracy while improving computational efficiency.
Table 3: Ablation study results.
on
Setting
Score
Baseling ELMUR
1.00 0.00
Shared memory
0.45 0.03
No rel. bias
0.95 0.05
No LRU
0.43 0.22
No rel. hias; LRU
0.22 0.11
MoE: MLP
1.00 0.00
To confirm that memory mechanisms do not harm performance on fully observable MDPs, eval- uated all models on the simple control task CartPole v-1 (Towers et al., 2024). ELMUR, RATE, RMT, TrXL, BC-MLP, BC-LSTM, and CQL all achieved the maximum return of 500 + showing that adding memory does not break performance in standard MDP settings. See additional results on more competitive D4RL benchmark (Todorov et al., 2012) with MDP tasks in the Appendix, Sec- tion A.5, Table 4.
100% 10%-
-16
Figure 5: ELMUR compared to DT on all 48 POPGym tasks. Each model was trained with three independent runs, validated over 100 episodes each. Bars show the mean performance with 95% confidence intervals computed over these three means. ELMUR achieves consistent improvements over DT, with the largest gains memory-intensine puzzles.
9

---

Published as a conference paper at ICLR 2026
Sacaaa Bak
Salues Rate
43.000
0.23 3.50
1.00 80000
8:3
6.33
6:73
100
In
11 MY 15-1 1TS 15 ii
Nambor Kanuayy Exbuddage
Figure 6: Ablations of ELMUR's memory hyperparamettess RememberCColor3-v0 (a) LRU blending factor (A), (b) memory embeddings initialization (σ), (c) number of memory embed- dings (M) and (d) segment configuration (L S). Curves compare settings where the number of memory embeddings is smaller than the required segments to solve the task (M < N) versus larger (M N). Results show that sufficient memory capacity (M z N) yields stable success, while under-provisioned memory (M < N) is highly sensitive to X, a, and segmentation.
6 RELATED WORK
Manipulation. Transformer approaches to robotic manipulation can be broadly categorized by their underlying design principles. Perception-centric visuomotor transformers focus on multi-view or 3D perception to improve near fully observable control (Shridhar et al., 2023; Goyal et al., 2024). Sequence/skill modeling distills demonstrations into reusable action chunks but remains bottle- necked by limited context (Huang et al., 2023; Kobayashi et al., 2025). Planning/valuee srinminn/valee-ugmmeneeedd transformers integrate transformers with planning or value learning for closed-loop control under finite context (Zhang et al., 2025b; Hu et al., 2025). Alternative backbones adopt state-space models or diffusion for efficiency, but without persistent memory (Liu et al., 2024b; Chi et al., 2023). Scal- ing to VLA broadens task coverage with language but still suffers from fixed horizons, with some remedies via summarization, feature banks, or hierarchy (Zitkovich et al., 2023; Team et al., 2024; Kim et al., 2024; Fang et al., 2025; Shi et al., 2025). ELMUR differs by training standard IL transformer while removing the context bottleneck through structured, layer-local external memory.
Memory. Efforts to extend sequence models to long horizons take several forms. Implicit recur- rence and state-space models compress history in hidden dynamics, offering efficiency but little control over forgetting (Beck et al., 2024; Gu & Dao, 2023). External memory with learned ac- cess provides addressable storage but complicates optimization (Graves al., 2016; Santoro al., 2016). Transformer context extension retains history via caches or auxiliary slots but keeps mem- ory peripheral (Dai et al., 2019). In RL, memory often implemented through episodic buffers for salient events (Lampinen et al., 2021) or sequence-model adaptations that retrofit transformers for recurrence (Parisotto et al., 2020; Cherepanov et al., 2026b). Architectures vary in integration: RATE (Cherepanov et al., 2026c) concatenates memory with tokens, Memformer (Wu et al., 2020) uses global slots, and Mlock-Recurrent Transformers (Hutchins et al., 2022) recycle hidden states. EL.MUR instead gives each layer an external memory with dedicated mem2tok/tok 2mem cross- attention and LRU updates, yielding bounded memory for Long-horizon tasks (Appendix A.6).
7 CONCLUSION
We introduced ELMUR, a transformer architecture with layer-local external memory, bidirectional token-memory cross-attention, and an LRU-based update rule. Unlike prior methods, ELMUR integrates explicit memory into every layer, achieving retention horizons up to 100,000x beyond the native attention window. Our analysis establishes formal guarantees on half-life and bounded- ness under convex blending, and experiments on T-Maze, 48 POPGym tasks, and BIIKASA-Robe demonstrate consistent improvements over strong basclines, underscoring reliable credit assignment under partial observability. We envision ELMUR simple and extensible framework for long- horizon decision-making with scalable memory in sequential control.
REPRODUCIBILITY STATEMENT
We have taken several measures to ensure the reproducibility of our results. Model details: A com- plete description of the ELMUR architecture, including pseudocode for the layer update and the I LRU--ased memory module, is provided in Section Algorithm 1, and Figure 2. Theoretical
10

---

Published as a conference paper at ICLR 2026
sults: All assumptions and formal proofs of our propositions exponential forgetting, half-life, and boundedness are presented Section 4 and detailed in Appendix A.1-A.2. Experimental setup: Benchmarks, training procedures, and evaluation protocols are described in Section 5, with addi- tional specifications (byperparameters, dataset preprocessing. random seeds, and hardware setup) reported in Appendix 7 and A.4. Baselines: All baselines implemented from open-source li- braries or faithfully re-implemented with hyperparameters matched to their original publications, as described in Section 5 and Appendix A.6. Code and data: An anonymous repository with the implementation of ELMUR, training scripts, and configuration files is provided in the supplemen- tary material. Together, these resources enable full replication of both our theoretical analysis and empirical findings.
REFERENCES
Hervé Abdi and Lynne J Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433-455) 2010.
Amir Hosein Khas Ahmadi. Memory-based graph networks. University of Toronto (Canada), 2020.
Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xlstm: Extended long short-term memory. Advances in Neural Information Processing Systems, 37:107547- 107603, 2024.
Ali Behrouz, Peilin Zhong, and Vahab Mirrokni. Titans: Learning to memorize at test time. arXiv preprint S0LV::2501006666 2024.
Johan Bjorek, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, al. Gr0Ot nl: An open foundation model for generalist humanoid robots. arXiv preprint 2rXiv:2503.14734, 2025.
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. pi_O: A ssiionlanguage-action flow model for general robot control. arXiv preprint arXiv:2410.2416 2024.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.068177 2022.
Aydar Bulatov, Yury Kuratov, and Mikhail Burtsev. Recurrent memory transformer. Advances in Neural Information Processing Systems, 35: 1079-11091, 2022.
Yevgen Chebotar, Quan Vuong, Karol Hausman, Fei Xia, Yao Lu, Alex Irpan, Aviral Kumar, Tianhe Yu, Alexander Herzog, Karl Pertsch, al. Q-transformer: Scalable offline reinforcement learning via autoregressive q-functions. In Conference Robot Learning, pp. 3909-3928. PMLR, 2023.
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learmen via sequence modeling. Advances in neural information processing systems, 34:1 5084-1 2021.
Egor Cherepanov, Nikita Kachaev, Alexey Kovalev, and Aleksandr Panov. Memory, benchmark & robots: A benchmark for solving complex tasks with reinforcement learning. In The Fourteenth International Conference on Learning Representations, 2026a. URL htt.t://////// U /ptp///ppp......ee/o//
n www.eooonnnninneeeeeeee
Egor Cherepanov, Nikita Kachaev, Artem Zholus, Alexey Kovalev, and Aleksandr Panov. Un- raveling the complexity of memory in RL agents: an approach for classification and evalua- tion. In The Fourteenth International Conference on Learning Representations, 2026b. URL
https: /appnrevieeeveeeee/oeeeeeeeeeeee net wtw.arrreeeeeeeeooouuenee.eeee
Egor Cherepanov, Aleksei Staroverov, Alexey Kovalev, and Aleksandr Panov. Recurrent action transformer with memory. In The Fourteenth International Conference on Learning Representa- fions, 2026c. URL https:/ httptts////pperrieeee net
www.:neet/oomeeeee forum? i d=kBBNN40M3e..

---

Published as a conference paper at ICLR 2026
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The Inter- national Journal of Robotics Research, pp. 02783649241273666 2023.
Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Yu Wu, et al. Deepseekmoe: Towards ultimate expert specialization in mixture- of-experts language models. arXiv preprini arXiv:2401.0mm 0.666666666 2024.
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V Le. and Ruslan Salakhutdi- nov. Transformer-xl: Attentive language models beyond fixed-length context. arXiv preprint trXiv:1901.02860. 2019.
Murtaza Dalal, Ajay Mandlekar, Caelan Garrett, Ankur Handa, Ruslan Salakhutdinov, and Di- eter Fox. Imitating task and motion planning with visuomotor transformers. arXiv preprint arXiv:2305. 2023.
Siyu Ding, Junyuan Shang, Shuohuan Wang. Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. Ernie- doc: A retrospective long-document modeling transformer. arXiv preprint arXiv:2012. 15688, 2020.
Kevin Esslinger, Robert Platt, and Christopher Amato. Deep transformer q q-networks for partially observable reinforcement learning, arXiv preprint arXiv:2206.01077 2022.
Haoquan Fang, Markus Grotz, Wilbert Pumacay, Yi Ru Wang, Dieter Fox, Ranjay Krishna, and Jiafei Duan. Sam2act: Integrating visual foundation model with a memory architecture for robotic manipulation. arXiv preprini 2..:::200.18864 2025.
Niklas Funk, Julen Urain, Joao Carvalho, Vignesh Prasad, Georgia Chalvatzaki, and Jan Peters. Actionflow: Equivariant, accurate, and efficient policies with spatially symmetric flow matching. arXiv preprint arXiv:2409.04577 04576, 2024.
Kai Gao, Fan Wang. Erica Aduh, Dylan Randle, and Jane Shi. Must: Multi-head skill transformer for long- honghorrzonn dexterous manipulation with skill progress. arXiv preprint a3Xiv2250..02753, 2025.
Ankit Goyal, Jie Xu, Yijie Guo, Valts Blukis, Yu-Wei Chao, and Dieter Fox. Rvt: Robotic view transformer for 3d object manipulation. In Conference Robot Learning, pp. 694-710. PMLR, 2023.
Ankit Goyal, Valts Blukis, Jie Xu, Yijie Guo, Yu-Wei Chao, and Dieter Fox. Rvt-2: Learning precise manipulation from few demonstrations. arXiv preprint arXiv:2406.08555 2024.
Alex Graves, Greg Wayne, and Ivo Danihelka. arXiv: 1410.5401, 2014.
Neural turing machines. arXiv preprint
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska- Barwinska, Sergio Gomez Colmenarejo. Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Puigdomènech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis. Hybrid computing using neural network with dynamic external memory. Nature, 538:471-444 1-76, 2016. URL https: : / / /api.... semantt n cschol ar.
ID: : 205251479.
Jake Grigsby, Justin Sasek, Samyak Parajuli, Daniel Adebi, Amy Zhang, and Yuke Zhu. Amago-2: Breaking the multi-task barrier in meta-reinffrcemmet learning with transformers. Advances in Neurul Information Processing Systems, 37:87473-87508 2024.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00052) 2023.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00366, 2021.
ByungOk Han, Jaehong Kim, and Jinhyeok Jang. A dual process vla: Efficient robotic manipulation leveraging vim. arXiv preprint 1555, 2024.
12

---

Published as a conference paper at ICLR 2026
Matthew J Hausknecht and Peter Stone. Deep recurrent q-learning for partially observable mdps. In AAAI fall symposia, volume 45. pp. 141, 2015.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9:1735- 1780, 1997. URL http:////..pppe.eeennnccccecoooa otg/Corpusii.......ooo 1915014..
Jiaheng Hu, Rose Hendrix, Ali Farhadi, Aniruddha Kembhavi, Roberto Martín-Martin, Peter Stone, Kuo-Hao Zeng, and Kiana Ehsani. Flare: Achieving masterful and adaptive robot policies with large-scale reinforcement learning fine-tuning. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 3617-3624. IEEE, 2025.
Xiaoyu Huang, Dhruv Batra, Akshara Rai, and Andrew Szot. Skill transformer: A monolithie policy for mobile manipulation. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pp. 10852-10862, 2023.
Chia-Yo Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U Tan, Navonil Majumder, Soujanya Poria, al. Nora: A small open-sourced generalist vision language action model for embodied tasks. arXiv preprint arXiv:2504.1968 2025.
DeLesley Hutchins, Imanol Schlag, Yuhuai Wu, Ethan Dyer, and Behnam Neyshabur. Block- recurrent transformers. Advances in neural information processing systems, 35:33248-33261, 2022.
Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, al. pi (0.5:: a vision-language- action model with open- open-world generalization. arXiv preprint arXiv:2504. 16054, 2025.
Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big sequence modeling problem. Advances in neural information processing systems, 34:1273-1286 2021.
Yunfan Jiang, Agrim Gupta, Zichen Zhang, Guanzhi Wang, Yongqiang Dou, Yanjun Chen, Li Fei- Fei, Anima Anandkumar, Yuke Zhu, and Linxi Fan. Vima: General robot manipulation with multimodal prompts. arXiv preprint arXiv:2210.03094 2(3):6, 2022.
Angelos Katharopoulos, Apoory Vyas, Nikolaos Pappas, and François Fleuret. Transformers are mns: Fast autoregressive transformers with linear attention. In International conference on ma- chine learning, pp. 5156-5165. PMLR, 2020.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.092466 2024.
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Opti- mizing speed and success. arXiv preprint arXiv:2502.19645, 2025.
Masato Kobayashi, Thanpimon Buamanee, Yuki Uranishi, and Haruo Takemura. Ilbit: Imitation learning for robot using position and torque information based on bilateral control with trans- former. IEEJ Journal of Industry Applications, 14(2):161-66, 2025.
Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. Advances in neural information processing systems, 33:1179 18:1179-1100 11991199,,111 2020.
Andrew Lampinen, Stephanie Chan, Andrea Banino, and Felix Hill. Towards mental time travel: hierarchical memory for reinforcement learning agents. Advances in Neural Information Pro- cessing Systems, 34:28182-28 22082222211918 195, 2021.
Mikko Lauri, David Hsu, and Joni Pajarinen. Partially observable markov decision processes in robotics: A survey. IEEE Transactions on Robotics, 39(1):21-40 2022.
Hung Le, Kien Do, Dung Nguyen, Sunil Gupta, and Svetha Venkatesh. Stable hadamard memory: Revitalizing memory- augmentd agents for reinforcement learning. arXiv preprint arXiv:2410. 10132, 2024.
13

---

Published as a conference paper at ICLR 2026
Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tuto- rial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643 2020.
Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational vision-language-actoon model for synergizing cognition and action in robotic manipulation. arXiv preprint ww1.:::::1111965000 2024.
Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, et al. Vision-language foundation models effective robot imitators. arXiv preprint arXiv:2311.01376 2023.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024a.
Hao Liu, Matei Zaharia, and Pieter Abbeel. Ring attention with blockwise transformers for near- infinite context. arXiv preprint arX1v:2310.01889 2023.
Jiaming Liu, Mengzhen Liu, Zhenyu Wang. Pengju An, Xiaoqi Li, Kaichen Zhou, Senqiao Yang. Renrui Zhang, Yandong Guo, and Shanghang Zhang. Robomamba: Efficient vision-language action model for robotic reasoning and manipulation. Advances in Neural Information Processing Systems, 37:40085-4D0 401 10, 2024b.
Jiaming Liu, Hao Chen, Pengju An, Zhuoyang Liu, Renrui Zhang, Chenyang Gu, Xiaoqi Li, Ziyu Guo, Sixiang Chen, Mengzhen Liu, Chengkai Hou, Mengdi Zhao, KC alex Zhou, Pheng- Ann Heng, and Shanghang Zhang. Hybridvla: Collaborative diffusion and autoregression in a unified viston-language-action model, 2025. URL https: arwivv org abs 2253.10611.
Steven Morad, Ryan Kortvelesy, Mattco Bettini, Stephan Liwicki, and Amanda Prorok. POPGym: Benchmarking partially observable reinforcement learning. In The Eleventh International Con- ference on Learning Representations, 2023a. URL https: http:///peeeeieieeeeeeeee net forum?
iid=chDrutOTsOK.
Steven Morad, Ryan Kortvelesy, Stephan Liwicki, and Amanda Prorok. Reinforcement learning with fast and forgetful memory. Advances in Neural Information Processing Systems, 36:72008- 72029, 2023b.
Tongzhou Mu, Minghua Liu, and Hao Su. Drs: Learning reusable dense rewards for multi- stage tasks. ArXiv, 1ixx2204.16679,, 2024. URL pictps://apts semanticcchllrr org/ CorpusID: 2693621333333
Tianwei Ni, Michel Ma, Benjamin Eysenbach, and Pierre-Luc Bacon. When do transformers shine in rl? decoupling memory from credit assignment. Advances in Neural Information Processing Systems, 36:50429-50452. 2023.
Toshihiro Ota. Decision mamba: Reinforeement learning via sequence modeling with selective state spaces. arXiv preprint arXiv:2403. 19925, 2024.
Zikai Ouyang, Kaijun Wang, Junwei Liu, Haibo Lu, and Wei Zhang, Seil: Stage-conditioned imita- tion learning for multi-stage manipulation. IEEE Control Systems Letters, 2025.
Emilio Parisotto and Ruslan Salakhutdinov. Neural map: Structured memory for deep reinforcement learning. arXiv preprint 170.0:170..08866, 2017.
Emilio Parisotto, Francis Song. Jack Rae, Razvan Pascanu, Caglar Gulcehre, Siddhant Jayakumar, Max Jaderberg. Raphael Lopez Kaufman, Aidan Clark, Seb Noury. et al. Stabilizing transformers for reinforcement learning. In International conference machine learning, pp. 7487-7498. PMLR, 2020.
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, et al. Rwkv: Reinventing mns for the transformer arXiv preprint arXiv:2305. 13048, 2023.
14

---

Published as a conference paper at ICLR 2026
Subhojeet Pramanik, Esraa Elelimy, Marlos C Machado, and Adam White. Agalite: Approximate gated linear transformers for online reinforcement learning. arXiv preprint arXiv:2310.1571, 2023.
Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, Jia Yuan Gu, Bin Zhao, Dong Wang, al. Spatialvla: Exploring spatial representations for visual- language-action model. arXiv preprint arXiv:2201. 15830, 2025.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Jack W Rac, Anna Potapenko, Siddhant M Jayakumar, and Timothy P Lillicrap. Compressive transformers for long-range sequence modelling. arXiv preprint arXiv: 2019.
Moritz Reuss, Hongyi Zhou, Marcel Rühle, Ömer Erdinę Yagmurlu, Fabian Otto, and Rudolf Li- outikov. FLOWER: Democratizing generalist robot policies with efficient vision-language-aatiio acgtiigggtttionn flow policies. In 7th Robot Learning Workshop: Towards Robots with Human-Levas Abilities, 2025. URL https:://////. http://pperreeeeeeeee net/ forum? fttpmmmmddiofssss..... foBoWSLSq.
Ivan Rodkin, Yuri Kuratov, Aydar Bulatov, and Mikhail Burtsev. Associative recurtent memory transformer. arXiv preprint arXiv:2407004841 2024.
Adam Santoro, Sergey Bartunov, Matthew M. Botvinick, Daan Wierstra, and Timothy P. Lill- icrap. Meta-learning with memory-augmentedd neural networks. In International Conference Machine Learning, 2016. URL https: wwwanticschonteoeoeeeoeee 6466088.
TergCorppsID:
Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. Memoryvla: Perceptual-cognitive memory vision-language- action models for robotic manipulation. arXiv preprint arXiv:2508. /9236, 2025.
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning, pp. 785-799. PMLR, 2023.
Artyom Sorokin, Nazar Buzun, Leonid Pugachev, and Mikhail Burtsev. Explain my surprise: Learn- ing efficient long-term memory by predicting uncertain outcomes. 07 2022. doi: 10.48550/arXiv. 2207.13649.
Richard S Sutton, Andrew G Barto, et al. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998.
Denis Tarasov, Alexander Nikulin, Ilya Zisman, Albina Klepach, Nikita Lyubaykin, Andrei Pol- ubarov, Alexander Derevyagin, and Vladislav Kurenkov. Nina: Normalizing flows in action. training vla models with normalizing flows, 2025. URL https:://axxiiii org/ abs/2505 16845.
Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montser- rat Gonzalez Arenas, Travis Armstrong. Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world. arXiv preprint arXiv:2503.20020, 2025.
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprini arXiv:2405.1 240./ 2024.
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 JEEE/RSJ international conference intelligent robots and systems, pp. 5026-5033. IEEE, 2012.
Mark Towers, Ariel Kwiatkowski, Jordan Temy, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gymnasium: standard interface for reinforcement learning environments. arXiv preprint 240. 2024.
15

---

Published as a conference paper at ICLR 2026
Linji Wang, Tong Xu, Yuanjie Lu, and Xuesu Xiao. Reward training wheels: Adaptive auxiliary rewards for robotics reinforcement learning. arXiv preprint arXiv:2503.1572, 2025a.
Yongyi Wang, Lingfeng Li, Bozhou Chen, Ang Li, Hanyu Liu, Qirui Zheng, Xionghui Yang, and Wenxin Li. Symthetic pomdps to challenge memory- rl: Memory demand structure modeling. arXiv preprint arXiv:2508. 04282, 2025b.
Junjie Wen, Yichen Zhu, Jinming Li, Zhibin Tang, Chaomin Shen, and Feifei Feng. Dexvla: Vision-language model with plug-in diffusion expert for general robot control. arXiv preprint arXiv:2502.05885, 2025a.
Junjie Wen, Yichen Zhu, Jinming Li, Minjie Zhu, Zhibin Tang, Kun Wu, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, et al. Tinyvla: Towards fast, data-efficient vision-language-actin models for robotic manipulation. IEEE Robotics and Automation Letters, 2025b.
Jason Weston, Sumit Chopra, and Antoine Bordes.
Memory networks arXiv preprint
arXiv:1410.3916, 2014.
Qingyang Wu, Zhenzhong Lan, Kun Qian, Jing Gu, Alborz Geramifard, and Zhou Yu. Memformer:
A memory-augmentedd transformer for sequence modeling. arXiv preprint arXiv:2010.06891, 2020.
Yuhuai Wu, Markus N Rabe, Del eesley Hutchins, and Christian Szegedy. Memorizing transformers. arXiv preprint arXiv:2203.089) 2022.
Zheng Wu, Wenzhao Lian, Vaibhav Unhelkar, Masayoshi Tomizuka, and Stefan Schaal. Learning dense rewards for contact-rich manipulation tasks. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 6214-6221. IEEE, 2021.
William Yue, Bo Liu, and Peter Stone. Learning memory mechanisms for decision making through demonstrations. arXiv preprint 1rivv2241..07954 2024.
Maryam Zare, Parham M Kebria, Abbas Khosravi, and Saeid Nahavandi. A survey of imitation learning: Algorithms, recent developments, and challenges. IEEE Transactions Cybernetics, 2024.
Daniil Zelezetsky, Egor Cherepanov, Alexey Kovalev, and Aleksandr I Panov. Re: Frame- retrieving experience from associative memory. arXiv preprint arXiv:2508. 1.XXi::5008.93444 2025.
Jianke Zhang, Yanjiang Guo, Xiaoyu Chen, Yen-Jen Wang, Yucheng Hu, Chengming Shi, and Jianyu Chen. Hirt: Enhancing robotic control with hierarchical robot transformers. arXiv preprint
arXiv:2410.05273, 2024.
Kun Zhang. Peng Yun. Jun Cen, Junhao Cai, Didi Zhu, Hangjie Yuan, Chao Zhao, Tao Feng, Michael Yu Wang, Qifeng Chen, et al. Generative artificial intelligence in robotic manipulation: A survey. arXiv preprint arXiv:2503.0344.
2025a.
Xinyu Zhang, Yuhan Liu, Haonan Chang, Liam Schramm, and Abdeslam Boularias. Autoregressive action sequence learning for robotic manipulation. IEEE Robotics and Automation Letters, 2025b.
Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, et al. Cot-vla: Visual chain-of-thouge reasoning for vision- language-action models. In Proceedings of the Computer Vision and Pattern Recognition Confer- ence, pp. 1702-1713, 2025.
Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint vxXiv:2344.37705 2023.
Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du Yining Hong, and Chuang Gan. 3d-vla: A 3d vision-languuag--atiion generative world model. arXiv preprint
arXiv:2403.09631, 2024.
Ruijie Zheng, Yongyuan Liang, Shuaiyi Huang, Jianfeng Gao, Hal Daumé III. Andrey Kolobov, Furong Huang, and Jianwei Yang. Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies. arXiv preprint arXiv:2412. 10345, 2024.
16

---

Published as a conference paper at ICLR 2026
Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference Robot Learning, pp. 2165-2183 PMLR, 2023.
17

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 PROOF OF EXPONENTIAL FORGETTING IN LRU UPDATES
Proposition 1 (Exponential Forgetting). Fix slot and segment index i. Suppose this slot is updated k times at segments + 1, i 4 k according to
mj
À mew
+ (1 - 1)-A)mm m
t=it t=i+
(10)
with 0 1A Then
m + (1 2) mj
Σ X(1 2)k-U
mew
f+
(11)
Consequently, the coefficient of the initial content m after k overwrites is (1- A) and the coeffi- cient of the write performed updates ago (at segment 7) X(1 A)-11
Proof. We prove Equation 11 by induction on k.
Base case k 1. By the update rule,
m
i+1
À m
++1 new
+ (1- - A) m/.
(12)
which matches Equation 11 with h: = Induction step. Assume Equation 11 holds for some h 2 For ie 1,
i+++1 A Amt
i+k+1 + (1 - A) mj i+l
(13)
Insert the induction hypothesis for m ttt:
++++ m
A m #+++
new
+ (1 (1- 2)
(1 2) 1amm + Σ A(1 A) =
men
i+u
(14)
u=1
Distribute X) and regroup terms:
k X(1 2)(b+1)-u
++++1 = (1 - 2) m
m ++ new
À new
++++1
(15)
==1
which is exactly Equation 11 with & replaced by k + 1. This completes the induction.
Finally, the coefficients Equation 11 form a convex combination:
(1 X)* + Σ X -1 (1 (1 (1- - =1,
(16)
==1
0
so it is meaningful to call them "fractions" of contribution.
Corollary (Half-life). The number overwrites ko.5 after which the contribution of m halves satisfies
In(1/2) ko.s In(1 X)
(1 Ao.s. =
(17)
Equivalently,
1
ko.s In(1 2)
(18)
Using the Maclaurin series expansion In(1 A) - as X + 0. we obtain
lim Ko.s 1n
(19)
x
Hence,
In 2
k0.5
as À + 0,
(20)
X
showing that smaller yields longer retention horizons, while larger overwrites past content more aggressively.
18

---

Published as a conference paper at ICLR 2026
A.2 BOUNDEDNESS OF MEMORY EMBEDDINGS
Fix a slot j and consider its update rule,
m ++ m new + (1 - A) mj.
OSASI.
(21)
We prove the claim by induction on i. Base case. At the assumption gives |m m S.C.
Induction step. Assume ||m]|| S C for some i 2 0. Using the update rule,
|m || || m ++1 - X) mill-
(22)
new
By the triangle inequality and the inductive hypothesis,
|| m
I ||
Al|m
m| (t || (1 - A) +
C.
(23)
Thus ||m] finft1l < C, completing the induction.
Therefore, for all i, the memory embedding satisfies |mm]| S C.
A.3 MEMORY-INTENSIVE ENVIRONMENTS
Event-recall pairs and correlation horizon. Following (Cherepanov et al., 2026b), let at denote an event of duration At starting at time le and let Br, (a²') be later decision point that must recall information from that event. Define the correlation horizon as
RemenberColor9-vv (top) and ItkeItBack-uu (bottom).
(a) MIKASA-Robo:
{ = 1, te - At +
(24)
For environment, collect all event-recall horizons into the set = = }=.
T-Maze
Recent works have proposed alternative but complementary definitions of memory- intensive environments. For instance, Wang et al. (2025b) formalize memory demand structures that capture the minimal past suffi- cient to predict future transitions and rewards, while Yue al. (2024) introduce memory dependency pairs to annotate which past observations must be recalled for correct decisions. Both perspectives emphasize controllable ways to scale memory difficulty, either by increasing order/span or by manipulating dependency graphs. In this paper, adopt the event-recall horizon framework for clarity and analytical tractability, but note that these alternative views are broadly consistent and provide useful tools for designing and benchmarking memory-intensive tasks.
(b) T-Maze.
(c) Some POPGym tasks.
Figure 7: Environments used in experiments.
Definition (memory-intensire environment). A POMDP Mp is memory-intensive if min, 3 1; i.e., every relevant decision depends on information separated by at least one intervening step, making reactive (myopic) policies insufficient. This definition cleanly separates POMDPs that gen- uinely require memory from those that are effectively MDP-like.
A.4 MEMORY-INTENSIVE ENVIRONMENTS
The memory-intensive environments used this work are presented Figure 7.
T-Maze. The T-Maze (Ni et al., 2023) features a corridor ending in junction with two goals.
At the start, one goal is randomly revealed, and the agent must recall this cue after traversing the corridor to choose the correct branch. Observations are vectors; actions are discrete. Rewards sparse, provided only upon reaching the correct goal. The task tests whether the model can retain early across long delays.
19

---

Published as a conference paper at ICLR 2026
Table 4: Normalized scores on MuJoCo tasks from the D4RL benchmark (Todorov al.,
2012). Baselines include Trajectory Transformer (TT) (Janner et al., 2021), Decision Transformer (DT) (Chen al., 2021), Conservative Q-Learning (CQL) (Kumar et al., 2020), and Behavior Cloning (BC) from (Chen et al., 2021). Top-1 and Top-2 results are highlighted. (R, o, a) and (o) denote two conditioning modes of ELMUR: conditioning on returns-to-go, observations, and actions, conditioning on observations alone, respectively.
Dataset
Environment
CQL
DT
BC
RATE
ELMUR (ft, a)
ELMUR (o)
MB
HallCheeiah
91.6
8..811..
59.9
95.01002
87.410.1
86,412.77
89.1.10.6
ME
Hopper
105.4
1.8
79.6
110.0+22.7
112.5+0.2
111.7+0.2
85.3+1.6
MR
Walker2d
108.8
108.1+0.2
36.6
101.9++6.8
108.7+0.5
108.9+0.2
108.4+0.11
M
HallCheetah
44.4
42.6.10.1
43.1
46.9110.4
43.5
43.210.11
13.1.10.1
Hopper
58.0
67.6.11.0
63.9
61.111..6
77.411.
78.010.8
78.61110
M
Walker2d
72.5
7..0
77.3
79.0:12.8
80.7
79.420.2
79.5
MR
HalfChectah
45.5
36.6:-0.8
4.3
41.942.5
39,0=0.6
40,4=0.6
39.8=100
MR
Hopper
82.747.0
27.6
91.5++3.6
83.7+8.2
64,8+1166
87.6+1.7
MR
Walker2d
77.2
6..6+
36.9
6..6
73.7+
7..0
60.8+1.7
Average
77.6
74.7
47.7
78.9
78.5
76.4
74.7
MIKASA-Robo benchmark. We further evaluate on the MIKASA-Róbe bench- mark (Cherepanov al., 2026a), which provides robotic tabletop manipulation tasks designed for memory evaluation. Each environment simulates a 7-DoF arm with two-finger gripper. Observations are paired RGB images (3 128 x 128) from static and wrist camera; actions are continuous (7 joints + gripper). Rewards are binary, given only task success. We study two families MIKASA-Robo tasks: (i) wwweeebberolloo [3, 5, 9] -vo,, where agent must recall the color of hidden cube after delay with distractors, and (ii) ItkektBackvv,, where the agent first moves a cube to a goal, then must return once the goal changes.
POPGym benchmark. POPGym benchmark (Morad et al., 2023a) is large suite 48 partially observable environments designed stress agent memory. The tasks span two categories: (i) di- agnostic memory puzzles, which require agents to remember cues or solve algorithmic sequence problems (e.g., copy, reverse, n-back, and long-horizon T-Mazes), and (ii) partially observable trol tasks, which adapt classic control benchmarks such as CartPole, MountainCar, and Lunarl annder observation-limite settings. This diversity allows POPGym test both symbolic memory skills and generalization in continuous control, providing a broad and challenging benchmark for evalu- ating memory -aggeeetd RL methods. Detailed results across all 48 POPGym tasks for each of considered baselines are presented in Table
A.5 TRAJECTORY CONDITIONING
To assess (1) how ELMUR behaves on standard high-dimensional MDP control tasks and (2) how different conditioning schemes influence performance, we evaluate it on the MuJoCo domains of the D4RL benchmark (Todorov et al., 2012). The datasets contain trajectories heterogeneous quality, including many far from expert level, which typically requires conditioning on returns-to-go and actions (as in DT (Chen et al., 2021)) to enable trajectory stitching.
We therefore test two conditioning modes for ELMUR: (R, a). analogous to DT, and o-only, as standard behavior cloning. Despite the lower quality of D4RI, data, ELMUR achieves an average normalized score of 76.4 under (R, a) conditioning, only about 2% higher than with only conditioning. This demonstrates that the model remains robust to the choice of conditioning variables and does not rely access to return-to-go action labels to perform well.
Overall, ELMUR matches or exceeds the performance of several established offline RL baselines on these MDP tasks, confirming that its strong results are not restricted to memory-centee POMDP environments. Its competitive behavior across conditioning modes further highlights its versatility and ability to leverage suboptimal trajectories effectively.
20

---

Published as a conference paper at ICLR 2026
Table 5: POPGym benchmark results. Reported scores are mean = standard error, averaged over 3 runs and evaluated with 100 random seeds. Across 21 partially observable tasks spanning puzzle and control domains, ELMUR achieves the best performance on 12 tasks, underscoring the effectiveness of its memory module for reasoning under partial observability.
POPGym-48
RATK
DT
Random
BC-MLP
BC-LSTM
KLMUR
Expert PPO-GRU
AutoencodeEass--00
0.29:0000
-047:00.00
-0.50:0..00
-0.47:00.00
-0.32:0.00
0.26:0.00
-0.26
AuteenccoaaMeclaanov
0.46:0000
0.4900000
0.50..0.01
00.00.0.00
0.4420.00
0.442.0.00
-0.43
Aucoencodelaard-v0
0..7=00.0
-0.9.0..00
-0.00.0.00
-0.1910.00
-0.17=0.00
0.16600000
-0.48
-881-002
-093-005
-:46--0011
-1000-0.00
-0.49-0.01
4.79-0002
.0.35
RattleahipMeciime
-.920001
.0.700001
00.00000
-1.00=0.000
-0.67=0001
0.79==0.11
-0.40
Ba:t_eahipHarr-vv0
.0.1=00022
.0.1100.03
0.39
.1.00:0.00
.0.81=00022
0.800::0.00
-0.43
Concent 10
4:06-002
4005=001
40.19:0.01
-0:92:0.00
-0.14:0.00
4.14:00011
Concent ionM
4:25-0000
-025500011
4:19=0000
-0.92-0.00
0.19--.01
4.22+00.0
-0.44
Concent:rttionnlardd-00
-0.84:0.00
-0.84:00000
00.4:00.00
00.8:00.00
0.84:00.0
4.88::0.00
-0.87
CountRece=11aasy-190
0.07:0.01
-0.16:0.01
-0.3:00000
00.92:00.00
0.05:0000
0.03:0001
0.22
CountRecai
0.54-0000
0.8110.02
0.9310.000
0.92....00
0.00
0.56-00.11
CountReceallaadd-00
0.17=0001
0..7500.03
0..8..0.00
-0.88.00.00
-0. 0.17
4:47:00000
-0.48
Hi #iggher2ax::2as-v0
0.5010:000
0.50110.00
0.0010.01
0.47200000
0.50110.00
0.50210.00
0.51
Higher wwgerroowwteeeiine
0.52+0..0
0.51
0.01=0.01
0.50=0.00
0.510001
0.1000.00
0.49
#igheriowezBard-v00
0.5010.00
0.50110.00
0..110..00
0.4910000
0.50100.00
0.50200.00
09540000
080+0.01
-0.39=0.00
0.72=0005
0.92+001
0.2+00.00
0.95
Tabyrint SttyrintFFaaapppaa Facaaaadd
4.56-001
-6670004
-084-00.44
-0.71-0..33
-0.69-0.02
4L54=0001
40.49
LabyrintZEscageaarcd
481:0001
0.220001
0.9400.01
-0.89=0.01
-0.86=0.000
0.82200.1
-0.44
wabyrinthxxporeeeeee
095:00.00
0.8:00.06
-0.44:0.01
0.87:0001
0.93:0.00
096200000
0.96
Labyrinthixp
D38+0000
0.6-+0.011
-0.100.00
0.45000111
0.820001
0 0.86++0.0)
0.87
LabyrinthExplereelrrdd.
0.7910.000
0.7710.01
-0.7310.00
0.261:001
0,711001
0.8110000
0.79
2ane5weepe:aasy-100
0.15100:3
0.33100.1
0.26110:03
-0.1710.01
0.20100000
0..17.0.02
sannSwaapeetbeddiii- Ki KLaesweape:tardd-0
0.20=00.00
0.37.0.02
0.39:::.01
-0.48::0.00
-0.16=00.00
0.32-00.00
.0.10
-0.44.0.00
0.4010.01
0.4310.000
00.4910.000
-0.35
0.39
-0.27
Muiti Lat.ameaandd::aaaaoooo
0.3700.01
0.2710.01
0.02-10.00
0.05100000
0.1740.02
0.43:0001
0.62
0.32+0.01
0:35+0001
0.01+0..00
0
0,14+0.00
0.32+0001
0.22+0.03
0.27+40.11
0.01=00.00
0.01=0.00
0.17+0.01
0.22+0.01
0.43
No NoisyPosite illeyoosstinnnnylaaaae how.eeeteeoooyyyaollleaayye
0.8010.03
087:30.02
0.11=00.0
0.23=0000
0.44:0001
0 0.59::0.22
0.98
No syP hta.teollllaaaaroolleeeeiiiiiiii
0:33+0.01
0.34+0.00
0.12+0031
0.18+0000
0
0.77+0.00
No NolayPoss aayoaa htno//ssttton011gaaaoooooeeeeee
0.18+0.01
0.17+66.1
0.11++0.00
0.16+0000
022:0001
No htle:PPoittionccllyeeeeellluauu
0.57.10.00
0.8110.01
0.27-10.01
0.31
0.88110000
0.8811000
0.90
No co-pyyoostiiiecclyyeemmmemeeeeee Mo../foottiannduu//eedddiiisises
0.6:00000
0.3::0.01
0.27=00.11
0.30::0.00
0. 0.72:0.00
0.72:0000
0.73
0.6010.01
0.56.10.01
0.26:10.00
0.28110.00
0.6610.00
0.6510..0
Pos Pos.::ooobbblyyaar:........._
0.9310:03
1.0010.00
0.12:10.00
0.15100000
0.1710.00
100110.00
1.00
Doal (balttioor0nlyGGrr00oaaaaedd Roo.tloobbbllyaassoo..eeaaxaa
0.07+0.00
D. 0.34+0008
0:05-40.00
0.09=0000
0.12+0.00
0.11=00.0
1.00
0.0500.01
0.03:10.00
0.04::0.00
0.05=00.00
0.06:0000
0:10:0:00
1.00
0.54+0:02
0.51+0:03
0.27=10.00
0.29=0000
0.52+0.00
Posl tieettionn]]) wwwn:mn11yendddueeeiiioo
0.49+0.01
0.55+00011
0.26-00.00
0.30+0000
0 0.890000
0 0.50+00.01
0.88
0.47+0.01
0.49+00.11
0,26==0.00
0
0.82+00.00
0.63+-0.01
0.82
RepeatFizattass--00
10010:000
0.4510.16
-011110.01
-0.5011.000
1:0110:00
100110.00
Kepeat (irp
0.9910.01
-021:0188
-0.50:00.00
-0.50:00.00
0:99:000
0:9::0.00
1.00
Repest ret
D. 0.10+0.02
0.42+0.11
00.00+0.00
-0.50=0.00
-0.50-0.00
0:99+0.01
0.99
PepeatttrevoousSass-u
100110.000
100010.00
0.4910.01
-0.52:00.00
1,0010.000
1:00100000
1.00
Repeat Previcuateddiunnn0
-0.38=0.01
-0.3110.00
-0.5010.01
-0.50:10.00
-0.38=0.00
+0.3910.00
RapeatPrevioustardee
-0.6:=0.000
0.4700.00
0.51
.0.48:20.00
.0. 00.45=0.000
40.46:00.00
-0.48
Ve Yellocityyon.yyaarppo...
1:000:000
1000:0.00
0.11=:0.00
0.99::0.00
1 1:00:0.00
1000:0.000
1.00
Va belo y yCartpo un-vv
100+00000
100+0.000
0.06+00.00
0833+001
0.00+00000
1.00+00.000
eyn1yycoooc -aadddvv
1.0000000
0.96+00022
0.04=00.00
0.63+0000
1.00+0000
1.00+00000
Sum of returns
9.54
5.80
-12.24
.6.8%
8.96
10.41
16.51
A.6 EXTENDED RELATED WORK
Transformers for manipulation. Work on transformer-based manipulation splits into character- istic families with complementary strengths and limits. Perception-centre visuomotor transformers emphasize 3D or multi-view geometry to map pixels precise end-effector actions under (near) full observability, excelling at spatial alignment but assuming short temporal credit assignment: Per- Act (Shridhar al., 2023), RVT (Goyal et al., 2023), RVT-2 (Goyal et al., 2024). Sequence/skill models distill demonstrations into reusable action chunks and long-hoiizon trajectories, improv- ing sample efficiency but remaining bottlenecked by finite context windows: ACT (Zhao et al., 2023), Skill Transformer (Huang et al., 2023), ILBiT (Kobayashi et al., 2025). Planning/RL--driver transformers blend sequence modeling with value learning or planning for closed-loop control un- der uncertainty, yet still inherit fixed-context limitations: OPTIMUS (Dalal et al., 2023), Action- Flow (Funk et al., 2024), Q-Transformer (Chebotar et al., 2023), CCT/ARP (Zhang et al., 2025b), FLaRe (Hu et al., 2025). Alternative backbones (state-space, diffusion) target long continuous hori- zons and smooth control but lack explicit persistent state: RoboMamba (Liu et al., 2024b), Diffusion Policy (Chi et al., 2023).
21

---

Published as a conference paper at ICLR 2026
VLA models for manipulation. VLA models broaden task coverage by conditioning language and scaling data, while keeping the transformer core unchanged and context-bounden Instruction- following generalists demonstrate broad skill repertoires with language prompts but no explicit long- term memory: RT-1 (Brohan et al., 2022), RT-2 (Zitkovich et al., 2023), Octo (Team et al., 2024), OpenVI.A (Kim et al., 2024), VIMA (Jiang et al., 2022). fificiiencyhoodularity variants freeze large encoders and train lightweight adapters or experts for practicality at scale: RoboFlamingo (Li al., 2023). CogACT (Li et al., 2024), FLOWER (Reuss al., 2025), NinA (Tarasov et al., 2025), Reasoning/hierarclyy extensions inject geometric priors, step-wise reasoning, or multi-level control while still relying on finite windows: 3D-VLA (Zhen al., 2024), CoT-VLA (Zhao et al., 2025), TLaceVLA (Zheng et al., 2024), HiRT (Zhang et al., 2024), DP-VLA (Han et al., 2024). Special- ized and hybrid designs tailor the backbone to domain constraints (dexterous hands, spatial priors) or mix diffusion with autoregression: HybridVL.A (Liu et al., 2025), Dex (Wen et al., 2025a), Spa- tialVLA (Qu et al., 2025), Open OpenVLA--FF (Kim et al., 2025). Generalist robot agents pursue open- world embodiment with growing breadth but inherit the same temporal limitations: VLA (Wen et al., 2025b), To (Black et al., 2024), To.s (Intelligence et al., 2025), GROOT N1 (Bjorck et al,, 2025), Gemini Roboties (Team al., 2025), NORA (Hung al., 2025).
Memory in Deep Learning. Mechanisms for long-term information fall into three broad tracks. Implicit recurrence and long-range sequence models retain information in hidden dynamics but offer limited, indirect control over storage and forgetting: LSTM (Hochreiter & Schmidhuber, 1997), xL- STM (Beck et al., 2024), linear-attention Transformers (Katharopoulos et al., 2020), RWKV (Peng et al., 2023), S4 (Gu et al., 2021), Mamba (Gu & Dao, 2023). Explicit external memory with learned read-write provides addressable storage with content- based access, trading off simplicity for opti- mization/scaling complexity: NTM (Graves et al., 2014), DNC (Graves et al., 201 6), Memory Net- works (Weston et al., 2014), differentiable memory for meta-learning (Santoro al., 2016), recent variants (Ahmadi, 2020). Transformer context extension pushes horizons via cached activations, compression, or auxiliary memory modules but typically keeps memory peripheral to the core to- ken computation: Transformer-XL. (Dai et al., 2019), Compressive Transformer (Rae et al., 2019), Memorizing Transformer (Wu et al., 2022), Ring Attention (Liu et al., 2023), Memformer (Wu et al., 2020), associative-memory Transformers (Rodkin et al., 2024), ERNIE-style memory (Ding et al., 2020), test-time memorization (TiTANS) (Behrouz et al., 2024).
Memory in RL. In partially observed decision processes, memory is not optional; it is the state estimator. Spatial/episodic buffers externalize salient facts in read-write maps associa- tive stores (navigational and episodic use-cases); Neural Map (Parisotto & Salakhutdinov, 2017), HCAM (Lampinen et al., 2021), Stable Hadamard Memory (Le et al., 2024). Sequence-model adaptations retrofit transformers for recurrence and stability in RL, improving long-horizon train- ing but leaving persistence bounded by context: DTQN (Esslinger et al., 2022), GTrXL (Parisotto et al., 2020), AGaLiTe (Pramanik et al., 2023), AMAGO-2 (Grigsby et al., 2024). Evaluation frame- works formalize memory demands and failure modes under partial observability: (Cherepanov et al., 2026b; Yue et al., 2024; Wang et al., 2025b). Trunsformers with external stores for RL insert explicit memory alongside the policy, but often as a sequence-leve attachment: RATE (Cherepanov et al., 2026c), FFM (Morad et al., 2023b), Re:Frame (Zelezetsky et al., 2025).
Memory in manipulation tasks. Long-horizon, partially observed manipulation has motivated three practical extensions. Trajectory summarization compresses the past into a short token set, trading completeness for compactness: VLA (Zheng et al., 2024). External feature banks cache recent visual features for retrieval, focusing attention but leaving distant history underrepre- sented: SAM2Act+ (Fang et al., 2025). Structured memory modules interface an explicit store with the policy to persist scene/task variables: MemoryVLA (Shi et al., 2025). Iiirarchies shift tem- poral burden to slow planners or high-level controllers, which may mask but not remove the need for persistent state at the policy layer: HiRT (Zhang et al., 2024), DP- (Han al., 2024). For evaluation, targeted benchmarks isolate memory factors: Memory Bench (Fang et al., 2025); broader suites capture multiple memory types and delays: MIKASA-Robo (Cherepanov et al., 2026a).
22

---

Published as a conference paper at ICLR 2026
Table 6: Returns on ViZDoom-Two-Coke (mean standard error over 6 runs). Top-1 and Top-2 results are highlighted.
Method
Return (mean sem)
ELMUR
55.28 0.94
RATE
59.21 1.19
DT
31.45 + 1.21
RMT
53.53 + 3.15
TEXL
49.62 0.88
BC-LSTM
52.16 2.59
CQL-MLP
12.49 + 0.19
Random
4.78
A.7 3D VISUAL NAVIGATION
To evaluate ELMUR's ability to handle 3D long-horizon navigation with visual input, addition ally experiment the ViZDoom- ViZDoom-Two-Colee Two-Colors environment (Sorokin et al., 2022). The agent moves in with acid floor that continuously drains health at every timestep. At the beginning of each episode it briefly observes single colored pole (green or red) for 45 steps. The pole then dis- appears, and the agent must navigate among green and red objects scattered on the floor and collect only those whose color matches the initial pole in order to receive rewards and restore health. The results in Table 6 show that ELMUR matches the performance of the previous best baseline within statistical uncertainty, indicating that ELMUR is effective not only on eemory-intenssvv manipula- tion tasks but also on challenging memory-dependee navigation tasks with 3D visual observations.
A.8 TRAINING PROCESS
When training ELMUR, we compute the loss on every segment processed recursively, detaching the memory state between segments (i.e., without backpropagation through time). The choice of loss depends on the task domain: for T-Maze, CartPole-vl, and a subset of POPGym tasks with dis- crete actions, with continuous actions, we use a mean- mecn--quared error (MSE) loss. The ELMUR hyperparameters used in the experiments presented in The ELMUR hyperparameters used in the experiments presented in Table 7.
apply cross-entropy loss; for MIKASA -ooAA--Robo and the remaining POPGym tasks
For data, follow consistent offline imitation- -eariiing-eriin setup. Each MIKASA RIboS environment provides a dataset of 1000 expert demonstrations generated by a PPO policy trained with oracle-level state access. For T-Maze, adopt the datasets of Morad et al. (2023a), consisting of 3000 trajectories per environment generated by PPO-GRU expert. Finally, for CartPole-vl, we use 1000 successful trajectories collected from a pre-trained PPO policy.
collect 6000 successful oracle-level trajectories. For POPGym,
Figure 8: ViZDoom-Two--olon environment.
23

---

Published as a conference paper at ICLR 2026
Parameter
RememberColor
TakeltBack
T-Maze
POPGymn-Kutoennodeeeayy
CartPole-vl
ducdel
128
128
128
Routed da
128
256
32
128
Shared
Layers
Heads
16
Esperts (MoE)
16
2
Shured Experts
2
Top-k routing
Memory
Memory init
0.1
0.001
0,001
0.01
Memory dropout
0.06
0.23
0.01
0.17
0.05
LRU blend
0.40
0,20
0.05
0,80
0.9
Dropout
0.13
0.01
0.10
0.14
0.10
Dropatt
0.30
0.17
0.10
Label smoothing
0.21
0.00
Context length
20
10
30
Batch size
64
64
128
128
512
Leaming rate
2.05c-4
2.57c-4
2.06c-4
1. 1.16
3.0c-4
Warmup steps
30000
30000
10000
50000
1000
Cosine docay
Troe
True
Trae
LR facter
0.1
Weight decay
0,001
0,01
0.0001
0.1
0.01
Epochs
200
300
1000
800
100
Grad clip
5
5
1.0
Retal
0.99
0.9
0.95
0.99
0.9
Beta2
0.99
0.999
Table 7: Hyperparameters for ELMUR. We report all architecture and training paramcters used
across tasks.
A.9 MEMORY PROBING
To examine whether ELMUR stores task-relevant information its external memory rather than relying solely increased parameter count, perform controlled memory probing study the
environment. The goal is to determine whether memory embeddings contain linearly nonlinearly decodable information about the target cube color, which serves as the minimal sufficient statistic required for correct decision making in the recall stage of the task.
Dataset construction. We first train ELMUR policy to expert-level performance Using the trained policy, we collect 1000 evaluation episodes and ex- tract, at every timestep, all layer-local memory embeddings and the corresponding ground-truth tar- get color. We retain only episodes that successfully complete the task to avoid confounding effects from incorrect trajectories and to ensure that the probed representations correspond to functional memory usage. The resulting dataset contains tuples of the form
Kememberroolor3--
(e, t, t.mm .,mm))))).
where l is the layer index, t is the timestep which the memory vector was recorded, m! e Rx is the memory embedding of layer at timestep t, and E {0, 1, 2} is the target color label.
Probing protocol. Memory in ELMUR is structured and evolves differently across layers and timesteps. To distinguish these effects, we construct a separate probing dataset for every layer E {0.... (0...., }} and every chosen timestep (we use {0, 5, 10, 20})). For each pair (e, t) we train an independent probe on the corresponding memory vectors {m: (²))
Each probe is three-layer multilayer perceptron (MLP) with hidden size 512 and ReLU nonlinear- ities. We train the probes with a cross-entropy loss to predict the target cube color solely from the memory vector. Probes trained with Adam for 200 epochs using 80/20 train-validation split. Importantly, probes are not allowed to influence the base policy; the probing procedure is strictly post-hoc.
Memory Evaluation. For every (f, pair we report the accuracy and confusion matrix the cor- responding probe. At /=0, when memory is initialized with random noise, all layers yield roughly
24

---

Published as a conference paper at ICLR 2026
Purcetoos PM
Morestogg
SIOLE
Poncatage
Dowcatage
Figure 9: Memory probing confusion matrices on RememberColor33--0 Each panel shows the confusion matrix (percentage) of the color probe trained on ELMUR memory embeddings for specific timestep (rows, E {0, 1, 5, 10, 20)) and layer (columns, é E {0. 2,3)). At t=0 (top row), when memory is still random, all probes sit near the 33% chance level, indicating that the target color not decodable from initialization. After the first write 1=1, the first layer already supports accurate decoding. with performance gradually decreasing for deeper layers. At/=5, once the cue has disappeared, probes achieve almost perfect accuracy, including 100% for the first layer. At the decision time t=10 and late in the episode at t=20, prediction remains essentially perfect in the first three layers and very high in the last layer, demonstrating stable long-horizon retention of the target color in memory.
33% accuracy per class, indicating that the target color is not decodable from the initial memory em- beddings. After the first update at t=1, the first layer already supports reliable decoding for all three colors, with accuracy gradually decreasing for deeper layers, which shows that the model writes a usable color code into memory immediately after observing the cue. By 1=5, when the colored cube
25

---

Published as a conference paper at ICLR 2026
Layer Memory Update
Memory Stot
Update Status
Stot. 0-
0.75
Slot
0
10
14
18
20
24
26
28
30
32
34
36
38
40
42
44
46
40
52
54
56
56
Timestep
Layer Memory Update
Memory Stot
Update Status
Stot. 0
Stot.
0.25
8.00
10
18
20
24
28
322
34
36
38
42
48
92
56
56
Timestep
Layer Memory Update Pattern
Memory Slot
Uplaaee Status
Stoc. 0-
Stot.
0.50
0.25
0.00
10
18
20
24
26
28
312
34
36
38
40
46
48
52
56
58
Timestep
Layer Memory Update
Memory Slot
1.00
Update Status
Stot: 0-
0.73
Stot.
0.50
Stot
0.25
0.00
10
12
14
16
18
24
26
28
312
34
36
38
42
46
48
92
58
Timestep
layers RememberColoz3 Each heatmap shows, for one ELMUR layer, which memory slot is updated at each timestep (horizontal axis: timestep [0, 60). vertical axis: slots 0-2; green indicates an update, purple indicates update). All layers exhibit the same structured schedule: early in the episode only slot is updated, then responsibility shifts to slot 1, and finally slot 2, producing clean rotation of writes enforced by the LRU policy. After slot becomes inactive it is not modified again, which implies that once the target color has been written into slot its content is preserved for the remainder of the episode without further overwriting.
Figure 10: Memory update patterns
has disappeared from the scene, prediction accuracy improves further, reaching 100% for the first layer and remaining high for the others. At/= when the agent all three cubes and must select the target, the probes achieve 100% accuracy on the first three layers and near perfect accuracy on the last layer, and the same pattern persists at t=20. Aggregated confusion matrices across layers and timesteps are shown in Figure 9.
Together, these findings confirm that ELMUR allocates memory capacity to storing the latent task variable required to solve the environment and that its superior performance arises from functional memory usage rather than increased model size. In addition, the memory update patterns Fig- ure 10 show consistent and interpretable write schedule across layers: each episode produces single dominant write into dedicated slot immediately after observing the cue, followed by long spans of preservation with no overwriting, The LRU mechanism activates only when necessary, producing sparse, well-timed updates rather than continuous churn.
The inter-slot similarity analysis in Figure 11 further supports this behavior. At the moment when the target cube is first observed, all layers exhibit a sharp transition in cosine similarity structure: single slot becomes highly similar across repeated episodes while the remaining slots remain decorrelated. This indicates that ELMUR allocates one stable slot as the canonical container for the task variable. After this write event, the similarity trajectories remain nearly constant throughout
26

---

Published as a conference paper at ICLR 2026
Lowe tod Dieen 268 scres batchet
Layer Me
514 Den Altons
dragg ti-a0
Grapg tanall
Timeslop
Timestop
Lawe 31 tele Uoter Mandarro
Sayre Me Dev Cabis
Sax Gresg bind
Bix Oriyg Sindl
Timestop
20
23
Timeslop
Timeslop
Loyer thaee
Diles 368 scres backeel
Layer's Me Der Case
b--20
Gragg tooal
Timestop
Timessop
Inter-slot similarity and variance of memory embeddings on RememberCoolo33-v. For each ELMUR layer (rows), the left panels show the cosine similarity between pairs of memory slots (0-1, 0-2, 1-2) as a function of timestep, averaged over 20 batches (shaded region: standard deviation). Around the transition to the second segment (t 20) the similarities jump from near zero to higher, stable values, indicating that one slot acquires a structured representation which becomes partially shared with the others. A second transition to the third segment appears at = 40, when another slot is written, after which the similarity profiles remain nearly constant, consistent with persistent storage. The right panels show the standard deviation of embeddings within each slot over episodes. Only the currently written slot exhibits sharp increase in variance at its corresponding write time, while the other slots remain nearly constant, confirming that the model performs localized, slot-specific writes followed by stable retention of the encoded content.
Figure 11:
the blank stage and the distractor stage, demonstrating that the stored representation is preserved without drift.
The slotwise variance plots show the same phenomenon from a complementary angle. The chosen slot exhibits a sudden rise in variance immediately after the cue is written, reflecting the encoding of task-specific information, while the unused slots retain near-zero variance for the remainder of the episode. Throughout the delay period the variance of the active slot remains stable, indicating that the stored color representation neither degraded nor rewritten.
Taken together, the update patterns. similarity dynamics, and variance profiles reveal coherent mechanism: ELMUR performs a targeted, one-shot write into a designated memory slot, then main- tains this representation with high stability until retrieval, even in the presence of visually similar distractors. This behavior aligns precisely with the probing results and provides strong causal evi- dence that ELMUR employs explicit long-term memory rather than incidental capacity effects. A more detailed, layer-wise visualization of evolution of memory embeddings across the entire episode is presented in Figure 12.
A.10 PCA MEMORY ANALYSIS
The geometric structure of memory representations is further illustrated by the Principal Component Analysis (PCA; Abdi & Williams (2010) in Figure 13. The top row shows projections colored by memory slot (slots 0, 1, and 2), while the bottom shows projections colored by layer. From left to right, we display all target colors together, then restrict to red, green, and blue targets respectively. In the slot based views, embeddings from different slots largely occupy the same manifolds for
27

---

Published as a conference paper at ICLR 2026
Figure 12: Memory evolution across layers, slots, and time on RememberColor3-vo.
Each block corresponds to snapshot of the memory state at given timestep (columns, t {0, (0.6...., (.))). Within each block, rows index memory slots (0-2) for a given layer (stacked verti- cally from Layer to Layer 3), and the horizontal axis shows embedding dimensions. Color encodes activation value. Early timesteps exhibit near-random low-amplitude activations, while after the cue observed a single slot in each layer becomes strongly structured and remains stable over subse- quent timesteps. Later in the episode additional slots are written once and then preserved, illustrating that ELMUR performs sparse, slot-specific updates followed by long-horizon retention of the en- coded representation.
fixed color, indicating that slots are not tied to particular colors and instead share common content space. In the layer based views, points from different layers form well separated clusters, especially when conditioning on a single color, which suggests that deeper layers produce progressively more structured and compact encodings of the target color while maintaining clear separation between colors in the joint view.
The temporal evolution of memory representations is illustrated in Figure 14, which shows PCA projections of all memory vectors with color indicating timestep. The first panel aggregates all target colors, while the remaining three panels condition on red, green, and blue targets respectively. In all cases the trajectories exhibit a characteristic pattern: memories start from a diffuse cloud near the initialization region (early timesteps in purple), then rapidly converge onto a small number of low- dimensional manifolds after the cue is written, and remain confined there for the rest of the episode (late timesteps in green). Conditioning on single color reveals that these manifolds are largely color specific, with each target color occupying its own stable region in PCA space and showing minimal drift over time. This behavior indicates that ELMUR quickly encodes the target color into compact representation and maintains it throughout the delay and retrieval phases without substantial degradation.
A.11 CROSS-ATTENTION MAPS
Figure 15 visualizes the cross-attention patterns of the write (tok2mem, left) and read (mem2tok, right) modules. Each row corresponds to one of the 16 attention heads, the horizontal axis shows timesteps (0-60), and the vertical axis indexes the three memory slots. In the tok2mem maps, attention is almost entirely suppressed at the beginning of the episode and then exhibits a short, high- intensity burst around the cue presentation. concentrated on single slot across many heads. After this write event, activity becomes sparse with only occasional weak updates, consistent with a one- shot allocation of the cue into a dedicated slot followed by long-term preservation. In contrast, the mem2tol maps display more sustained but still structured activity: once the cue has been written, several heads repeatedly attend to the same slot throughout the blank interval and the decision phase, with peaks near the timesteps where the agent must choose the correct cube. Other heads remain nearly inactive or attend to alternative slots, suggesting head specialization. Overall, these attention patterns corroborate the memory probing and similarity analyses, showing that ELMUR performs sharp, localized write of the target color into a single slot and then repeatedly reads from that slot during subsequent reasoning and action.
28

---

Published as a conference paper at ICLR 2026
2
Figure 13: PCA structure of memory embeddings by slot and layer on RememberColor3-vo.
Each panel shows 2D PCA projection of all memory vectors. The top colors points by memory slot (0, 1, 2), while the bottom row colors them by layer (0, 1, 2. 3). From left to right, we display embeddings for all target colors, and then restricted to red, green, and blue targets respectively. In the slot-based views, embeddings from different slots largely occupy the same manifolds for a fixed color, indicating that slots share common content space rather than being tied to particular colors. In contrast, the layer-based views form well-separated clusters, especially when conditioning on single color, showing that deeper layers produce progressively more specialized and compact representations of the same latent task variable.
2
Figure 14: PCA projection of memory embeddings over time. Each panel shows memory vectors from all layers and slots in nents and colored by timestep (see colorbar). From left to right we plot all episodes, and then subsets conditioned on the target cube being red, green, blue. Early timesteps occupy diffuse region of the space, while vectors shortly after the cue observation move into compact, color-specific clusters that remain stable for the rest of the episode, indicating that the memory state rapidly converges to persistent representation of the target color.
projected onto the first two principal compo-
A.12 LIMITATIONS
ELMUR a simple LRU rule with fixed blending, which makes its memory mechanism trans- parent and easy to analyze, though future work could explore adaptive variants. Segment-level recurrence adds a small cross-attention cost per layer, but this cost scales with the fixed number of memory slots rather than sequence length, making efficiency predictable even in long horizons. MoE-based FFNs already provide parameter efficiency at our scale, and larger architectures may fur- ther amplify this benefit. Finally, our study focuses on synthetic, POPGym, and simulated robotic tasks under IL, giving controlled and reproducible insights; extending to online RL and real-robot deployments offers promising next steps.
29

---

Published as a conference paper at ICLR 2026
Weiglts
Figure 15: Cross-attention patterns between tokens and memory slots in ELMUR. Heatmaps show attention weights for the tok2mem module (left) and the mem2tok module (right) in the RememberColor3
task. Each row corresponds to of the 16 attention heads, the hori- zontal axis is the timestep (t E [0, 60]), and the vertical axis indexes the three memory slots. Color intensity indicates the magnitude of the attention weight. tok2mem heads exhibit sparse, sharply localized write events into single slot around cue and update timesteps, whereas mem2tok heads display broader read patterns that concentrate on the slot earrying the target color during the decision phase, consistent with targeted write, once storage followed by repeated retrieval of the latent task variable.
30

---

Published as a conference paper at ICLR 2026
Table 8: Offfine RL baselines on the MIKASA- benchmark. We compare transformer-based methods (RATE, DT), behavior cloning (BC), classical offline RL algorithms (CQL), and Diffusion Policy (DP) across all 32 tasks. Scores are reported as mean sem over three seeds, with each seed averaged over 100 evaluation episodes. All models trained from RGB observations (top-view and wrist-view cameras) under sparse, success-only rewards. Even with access to memory, existing baselines (RATE, DT) fail most tasks, whereas ELMUR achieves the highest performance 21 of the 23 tasks with non zero success rates and exceeds the previous state-of-the-arr, RATE, by roughly 70% in aggregate success across all tasks.
Enviromment
RATE
DT
BC
CQL
DP
ELMUR
ShellCameTouch-vo
0.92-0.01
0.5310.07
0.2810.01
0.16:10.04
0.1810.02
0.96210.02
ShellCamePush-v0
0.78=0.06
0.62=00114
0.27+0.01
0.25=0.01
0.22+0003
0.87+00.44
ShelJGamePIck-v00
0.02=0.01
0.00=0000
0.01
0.00=0.00
0.01=0.00
0.66#0.01
Intercept:Slex-00
0.23-0.02
0.40=0002
0.37+0.06
0.25+0.01
0.33+0005
0.61
InterceeptMedium-vv0
0.32-0.02
0.56+0.01
0.31+0.14
0.03+0001
0.68+0002
0.64+0..03
InterceptFaast-v0
0.30-0.44
0.36+0.04
0.03+0.02
0.02+0.02
0.21+0.05
0.54+
Intercept:0rab51oxwww0
0.09-0.03
0.00+0000
0.28
0.03++0.00
0.03+00.1
0.08 H0.03
InterceptOOrabMeeiimm-v
0.09
0.00.0..00
0.1120.02
0.08
0.03.10.01
0.10.10.04
InteroepptGrrb/aast-00
0.14_0.03
0.11.0.03
0.0910.02
0,08
0.1810.02
0.20.10.01
kotateiJenieentPoo==>0
0.11-0.04
0.0110.01
0.15
0.16110.02
0.1110002
0,24.10.02
Rotate@eenleentsooaNeee---
0.29-0.03
0.05+0.02
0.2210001
0.12110.2
0.1410.05
0.26:10.00
Rotate@tzzict#Fos-v00
0.03-0.02
0.0510.04
0.01
0.03
0.06
0.16:10.00
KotateeBtr1cccFOsNeee--
0.08=0001
0.05=0.03
0.04
0.04
0.15#0.01
0.12400.02
TakeItBack-vv
0.42=0.24
0.08=0004
0.33+0.10
0.04
0.05=0.02
0.78+00.03
Rememberccolor3-v0
0.65=0.04
0.01=0001
0.27+0.03
0.29+0001
0.32+0.01
0.89++0.07
RememberrColor5-v0
0.13-0.03
0.07+0.05
0.12+0.01
0.15++0.02
0.10+0.02
0.19++0.03
RemembeerCollo99-v
0.09-0.02
0.01+0.01
0.12+00.2
0.15++0.01
0.17+0.01
0.23++0.02
Remppbeer
0.21=0.44
0.05
0.31
0.20
0.32:10.05
0.76:10.01
Rememberrshape5-v0
0.17_00.4
0.0410.04
0.18.0..1
0.15
0.21.10.4
0.27.10.02
Remeebberrshapee9vv0
0.05_.0.0
0.0510.02
0.10.10.2
0.14
0.11.10.22
0.17.10.03
0.14=0002
0.0410.02
0.1310002
0.11
0.1410.02
0.19.10.02
0.08-0.03
0.0610.06
0.0910002
0.0911002
0.1611001
0.19::0.02
0.07=0.02
0.01=0001
0.09
0.09:10.2
0.11
0.13+0.001
BunchOfcoleeea-v00
0.00=0.00
0.00-0000
0.00+0.00
0.00=10000
0.00=0.00
0.00+00.00
Bunchofcolcra5-v0
0.00=0.00
0.00=00.0
0.00+0.00
0.00=0.00
0.00=0.00
0.00+00.00
BunchOfColcrs77vv
0.00=0.00
0.00+0000
0.00+0.00
0.00++.00
0.00=0.00
0.00++0.00
Seq0#Colors33
0.00-0.00
0.00+0.00
0.00++..00
0.00++.000
0.00+0.00
0.00++0.00
Se:0FColors5-v0
0.00
0.00
0.00++0.00
0.00
0.00
0.00
Se:0FCcllrs7-v0
0.00
0.00
0.00.10.00
0.00
0.00
0.00
ChainofColora3-vl
0.00
0.00
0.00.10.00
0.00.10.00
0.00.0.00
0.00.10.00
ChainofColores--0
0.00-0000
0.00
0.00-10.00
0.00
0.00110.00
0.00-10.00
ChainOfColore7--v
0.00-0.00
0.00
0.00-10.00
0.00
0.0010.00
0.00-10.00
Sum Across All Tasks
5.42
3.15
3.92
2.66
4.02
9.24
31


Reviews:
[{"review_id": "ofmyOY7Yrr", "reviewer": "Reviewer_PqH3", "summary": "This paper proposes ELMUR (External Layer Memory with Update/Rewrite), a transformer policy for long-horizon control under partial observability. Each transformer layer owns a fixed-size external slot memory and two explicit pathways: mem→tok (read) and tok→mem (write) cross-attention. Memory is managed with a usage-aware LRU + convex-blend rule and temporal bias on writes, enabling controllable, analyzable forgetting via a closed-form half-life/effective horizon. Experiments demonstrate (i) retention far beyond the attention window on long-horizon probes (e.g., T-maze/key→door), (ii) strong performance across POPGym, and (iii) gains on pixel-based manipulation with sparse rewards. Ablations indicate per-layer memory and the LRU-blend update are key drivers. A direct comparison with RMT is included; I recommend adding an RMT-L diagnostic (layer-local RMT) to fully isolate where the gains come from.", "questions": "1. **Add an RMT-L diagnostic (optional but recommended).** \n2. **Clarify standpoint in Intro/Background.** Provide a compact “**Memory for long-horizon control**” subsection contrasting TXL/Compressive/Ring (history-scaled caches), RMT (shared tokens; implicit updates), RATE (recurrent state), test-time memory (e.g., Titans), and ELMUR (layer-local slots; **explicit read/write**; **LRU-blend** with temporal bias).", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "**Originality**\n* Combines per-layer fixed slot memory, explicit read/write paths, and LRU-blend with temporal write bias plus a half-life analysis—a configuration not present in prior explicit-memory policies.\n\n**Quality**\n\n* Method is precisely specified (blocks/pseudocode/hparams), and experiments span long-horizon probes, POPGym, and pixel manipulation, with ablations (M, λ, layer-local vs shared) supporting design choices.\n\n**Clarity**\n\n* Clean separation of self-attn vs mem→tok / tok→mem improves interpretability; slot contents/usage are inspectable. Figures make the two-track flow and LRU/blend easy to understand.\n\n**Significance**\n\n* Tackles long-horizon POMDP memory at fixed compute (scales with #layers×slots, not history), yielding gains on memory-heavy control and visual manipulation and aiding debugging/safety via controllable forgetting.", "weaknesses": "* **Standpoint/structure unclear.** The paper under-organizes its narrative: Introduction/Background don’t explicitly position ELMUR along the key memory axes (global vs layer-local, implicit vs explicit/usage-aware), and the method opener (“GPT + memory”) obscures the real novelty (explicit read/write, LRU-blend, temporal write bias, half-life).\n* **Memory-centric analysis is insufficient.** Although RMT is compared, there’s no RMT-L (layer-local RMT, same total memory) to isolate layer placement vs explicit management effects. More diagnostics—slot-usage/overwrite rates, write-location histograms, retention curves under capacity pressure—would clarify how memory is used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p8awNhbMph", "reviewer": "Reviewer_bpxq", "summary": "This paper introduces ELMUR, which addresses long-horizon decision-making under partial observability by augmenting each transformer layer with structured external memory updated via a Least Recently Used (LRU) mechanism with convex blending. Through offline imitation learning experiments, ELMUR achieves 100% success on T-Maze corridors up to one million steps, nearly doubles baseline performance on MIKASA-Robo visual manipulation tasks, and ranks first on half of the POPGym benchmark tasks.", "questions": "* Could the authors provide experiments on more 3D vision memory-intensive tasks (e.g., MemoryMaze)? Most memory-intensive tasks in the paper appear to be in simple observation environments.\n* How do author determine memory capacity $M$ for new tasks?\n* What is actually stored in memory slots? Could the authors provide qualitative results to demonstrate what information the memory embeddings capture?\n* In the ablation studies (Table 3), the advantages of DeepSeek MoE over MLP are not evident, with both achieving identical performance. Could the authors demonstrate whether MoE outperforms MLP across all benchmark environments, or provide quantitative results regarding computational efficiency advantages (FLOPs, training time, memory usage)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "* Provides theoretical guarantees on memory retention horizons and embedding boundedness\n* The T-Maze result (100% success at 1M steps) is genuinely impressive and demonstrates extreme memory retention", "weaknesses": "* Title claiming \"Long-Horizon RL\" while only evaluating offline imitation learning is misleading and inappropriate.\n* Lacks online RL experiments and comparisons with strong online RL baselines (e.g., R2I [1]). While the authors claim that unlimited interaction would significantly increase training time, sample efficiency should also be considered in online RL settings, which is a core evaluation metric for RL methods.\n\n[1] Mastering Memory Tasks with World Models, ICLR 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5jwFQsgP8a", "reviewer": "Reviewer_1eVV", "summary": "This paper proposes ELMUR to solve POMDPs by allowing a transformer to read and write to external memory, reminiscent of a Differentiable Neural Computer. \n\nELMUR consists of two blocks that feed each other. The token block is essentially a standard transformer with an additional cross attention module between the encoded observations and a cache. At each timestep, the token block outputs both an action and writes to the cache. The memory block consists of an LRU cache that produces memory vectors for cross attention with the token block. Importantly, the memory block is able to update its stored embeddings. These embeddings are written using a convex update $\\lambda m_{new} + (1-\\lambda)_{old}. The paper also presents a positional encoding that functions across multiple segments. The approach section ends with a short analysis on the decay rate of memories using ELMUR.\n\nThe paper evaluates ELMUR across three benchmarks and compares to other baselines such as RATE, DT, and DMamba. The results show that ELMUR achieves higher returns than other models. A T-Maze experiment demonstrates ELMUR generalizes to long trajectories. The paper performs ablation studies on the update weight $\\lambda$, random initialization of memory slots, number of slots, and segment length.", "questions": "- My understanding is the LRU eviction is based on writes, not reads. Could this result in older but important memories being evicted?\n- Why is $\\lambda$ a hyperparameter and not learned?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is organized well and pleasant to read. \n- The contributions are clearly explained.\n- The approach performs better than previous methods.", "weaknesses": "-  RQ1 and RQ4 experiments appear to be an unfair comparison, as DT does not have segment-level recurrence while the proposed method does. I would like to see a comparison between ELMUR and a DT+GTrXL method instead. This is the main weakness of the paper.\n- It is unclear why $\\lambda$ is selected rather than learned, an additional ablation would be useful.\n    - A learned, input-dependent $\\lambda$ would potentially remove the need for an LRU-eviction scheme.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "47EH3ziK7J", "reviewer": "Reviewer_AzD4", "summary": "ELMUR (External Layer Memory with Update/Rewrite) is a Transformer architecture that introduces structured, layer-wise external memory. Each layer maintains and updates memory embeddings through bidirectional cross-attention and an LRU-based update mechanism, enabling extremely long-term dependency modeling: up to 100,000× beyond the standard attention window. ELMUR achieves superior performance on long-horizon, partially observable tasks such as T-Maze, POPGym, and MIKASA-Robo, demonstrating a simple yet scalable solution for decision-making with long-range dependencies.", "questions": "1. What is the core motivation behind selecting the specific update rule presented in Eq. 8?\n\n2. Why not consider adaptive strategies that adjust the update behavior based on the relative importance of different segments?\n\n3. How generalizable is the observed improvement across different architectures？\n\n4. Why not include a visual analysis illustrating the proposed bidirectional update mechanism?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper proposes a new ELMUR mechanism and demonstrates its strong generalization ability through experiments.", "weaknesses": "1. **Table Placement and Layout:**\n\nTable 1 is placed too low and is visually disconnected from the surrounding text. It would be better to reposition the table for improved readability and consistency. The overall layout of the paper requires further optimization.\n\n2. **Ablation Study and Update Formula:**\n\nThe ablation study shows that the parameter λ is unstable when M<N. This instability might stem from the blending update formula. The motivation for choosing the specific update rule in Eq.8 should be explained in greater depth.\n\n3. **Uniform Update Weight Across Segments:**\n\nThe paper applies the same update weight to all segments, which seems somewhat unreasonable. In practice, different segments may have varying importance — for example, key frames might require longer retention (i.e., a smaller λ). This issue is not discussed in the current analysis or in the future work section.\n\n4. **Applicability and Generalization:**\n\nHow generalizable is this improvement? Is it limited to the Transformer architecture used in this paper’s experiments? The authors should apply ELMUR to more model architectures to demonstrate its generality or at least analyze its applicable scenarios.\n\n5. **Bidirectional Update Visualization:**\n\nThe bidirectional update mechanism would benefit from a visual analysis. For instance, when processing a specific frame’s token, is the most relevant portion of memory indeed assigned the highest weight? Otherwise, the observed performance gain might simply result from adding a new learnable external module.\n\n6. **Comparison with Prior Memory Methods:**\n\nA comparison table should be provided to clearly illustrate the differences between ELMUR and prior memory-related approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)